Connecting an input stream to an outputstream

JavaInputstreamOutputstream

Java Problem Overview


update in java9: https://docs.oracle.com/javase/9/docs/api/java/io/InputStream.html#transferTo-java.io.OutputStream-

I saw some similar, but not-quite-what-i-need threads.

I have a server, which will basically take input from a client, client A, and forward it, byte for byte, to another client, client B.

I'd like to connect my inputstream of client A with my output stream of client B. Is that possible? What are ways to do that?

Also, these clients are sending each other messages, which are somewhat time sensitive, so buffering won't do. I do not want a buffer of say 500 and a client sends 499 bytes and then my server holds off on forwarding the 500 bytes because it hasn't received the last byte to fill the buffer.

Right now, I am parsing each message to find its length, then reading length bytes, then forwarding them. I figured (and tested) this would be better than reading a byte and forwarding a byte over and over because that would be very slow. I also did not want to use a buffer or a timer for the reason I stated in my last paragraph — I do not want messages waiting a really long time to get through simply because the buffer isn't full.

What's a good way to do this?

Java Solutions


Solution 1 - Java

Just because you use a buffer doesn't mean the stream has to fill that buffer. In other words, this should be okay:

public static void copyStream(InputStream input, OutputStream output)
    throws IOException
{
    byte[] buffer = new byte[1024]; // Adjust if you want
    int bytesRead;
    while ((bytesRead = input.read(buffer)) != -1)
    {
        output.write(buffer, 0, bytesRead);
    }
}

That should work fine - basically the read call will block until there's some data available, but it won't wait until it's all available to fill the buffer. (I suppose it could, and I believe FileInputStream usually will fill the buffer, but a stream attached to a socket is more likely to give you the data immediately.)

I think it's worth at least trying this simple solution first.

Solution 2 - Java

How about just using

void feedInputToOutput(InputStream in, OutputStream out) {
   IOUtils.copy(in, out);
}

and be done with it?

from jakarta apache commons i/o library which is used by a huge amount of projects already so you probably already have the jar in your classpath already.

Solution 3 - Java

JDK 9 has added InputStream#transferTo(OutputStream out) for this functionality.

Solution 4 - Java

For completeness, guava also has a handy utility for this

ByteStreams.copy(input, output);

Solution 5 - Java

You can use a circular buffer :

Code

// buffer all data in a circular buffer of infinite size
CircularByteBuffer cbb = new CircularByteBuffer(CircularByteBuffer.INFINITE_SIZE);
class1.putDataOnOutputStream(cbb.getOutputStream());
class2.processDataFromInputStream(cbb.getInputStream());


Maven dependency

<dependency>
    <groupId>org.ostermiller</groupId>
    <artifactId>utils</artifactId>
    <version>1.07.00</version>
</dependency>


Mode details

http://ostermiller.org/utils/CircularBuffer.html

Solution 6 - Java

Asynchronous way to achieve it.

void inputStreamToOutputStream(final InputStream inputStream, final OutputStream out) {
    Thread t = new Thread(new Runnable() {

        public void run() {
            try {
                int d;
                while ((d = inputStream.read()) != -1) {
                    out.write(d);
                }
            } catch (IOException ex) {
                //TODO make a callback on exception.
            }
        }
    });
    t.setDaemon(true);
    t.start();
}

Solution 7 - Java

BUFFER_SIZE is the size of chucks to read in. Should be > 1kb and < 10MB.

private static final int BUFFER_SIZE = 2 * 1024 * 1024;
private void copy(InputStream input, OutputStream output) throws IOException {
    try {
        byte[] buffer = new byte[BUFFER_SIZE];
        int bytesRead = input.read(buffer);
        while (bytesRead != -1) {
            output.write(buffer, 0, bytesRead);
            bytesRead = input.read(buffer);
        }
    //If needed, close streams.
    } finally {
        input.close();
        output.close();
    }
}

Solution 8 - Java

Use org.apache.commons.io.IOUtils

InputStream inStream = new ...
OutputStream outStream = new ...
IOUtils.copy(inStream, outStream);

or copyLarge for size >2GB

Solution 9 - Java

This is a Scala version that is clean and fast (no stackoverflow):

  import scala.annotation.tailrec
  import java.io._
  
  implicit class InputStreamOps(in: InputStream) {
    def >(out: OutputStream): Unit = pipeTo(out)

    def pipeTo(out: OutputStream, bufferSize: Int = 1<<10): Unit = pipeTo(out, Array.ofDim[Byte](bufferSize))
        
    @tailrec final def pipeTo(out: OutputStream, buffer: Array[Byte]): Unit = in.read(buffer) match {
      case n if n > 0 =>
        out.write(buffer, 0, n)
        pipeTo(out, buffer)
      case _ =>
        in.close()
        out.close()
    }
  }

This enables to use > symbol e.g. inputstream > outputstream and also pass in custom buffers/sizes.

Solution 10 - Java

In case you are into functional this is a function written in Scala showing how you could copy an input stream to an output stream using only vals (and not vars).

def copyInputToOutputFunctional(inputStream: InputStream, outputStream: OutputStream,bufferSize: Int) {
  val buffer = new Array[Byte](bufferSize);
  def recurse() {
    val len = inputStream.read(buffer);
    if (len > 0) {
      outputStream.write(buffer.take(len));
      recurse();
    }
  }
  recurse();
}

Note that this is not recommended to use in a java application with little memory available because with a recursive function you could easily get a stack overflow exception error

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionjbuView Question on Stackoverflow
Solution 1 - JavaJon SkeetView Answer on Stackoverflow
Solution 2 - JavaDean HillerView Answer on Stackoverflow
Solution 3 - JavaWarren NocosView Answer on Stackoverflow
Solution 4 - JavaJesusFrekeView Answer on Stackoverflow
Solution 5 - JavaStephanView Answer on Stackoverflow
Solution 6 - JavaDaniel De LeónView Answer on Stackoverflow
Solution 7 - JavaLeandro LatorreView Answer on Stackoverflow
Solution 8 - JavaAdam111pView Answer on Stackoverflow
Solution 9 - JavapathikritView Answer on Stackoverflow
Solution 10 - JavaGeorge PligoropoulosView Answer on Stackoverflow