How to read multiple times from same io.Reader

GoReader

Go Problem Overview


I want to use request.Body(type io.ReadCloser) which is containing a image.

I dont want to use ioutil.ReadAll() as i want to write this body directly to the file as well as want to decode it, so i only want to use the reference to the content to pass to further function calls,

I tried creating multiple instances of reader for example shown below

package main

import (
	"io/ioutil"
	"log"
	"strings"
)


func main() {
	r := strings.NewReader("some io.Reader stream to be read\n")
	a := &r
	b := &r
	log.Println(ioutil.ReadAll(*a))
	log.Println(ioutil.ReadAll(*b))

}

but in second call it always results into nil.

Please help me how can i pass multiple separate reference for the same reader?

Go Solutions


Solution 1 - Go

io.Reader is treated like a stream. Because of this you cannot read it twice. Imagine an incoming TCP connection - you cannot rewind what's coming in.

But you can use the io.TeeReader to duplicate the stream:

package main

import (
	"bytes"
	"io"
	"io/ioutil"
	"log"
	"strings"
)

func main() {
	r := strings.NewReader("some io.Reader stream to be read\n")
	var buf bytes.Buffer
	tee := io.TeeReader(r, &buf)

	log.Println(ioutil.ReadAll(tee))
	log.Println(ioutil.ReadAll(&buf)) 
}

Example on Go Playground

Edit: As @mrclx pointed out: You need to read from the TeeReader first, otherwise the buffer will be empty.

Solution 2 - Go

When you call ReadAll it's going to empty the buffer, so the second call will always return nothing. What you could do is save the result of ReadAll and reuse that in your functions. For example:

bytes, _ := ioutil.ReadAll(r);
log.Println(string(bytes))

Solution 3 - Go

When you read from ioutil.ReadAll(r) then, the content is gone. You can’t read from it a second time. For an example:

var response *http.Response

//Read the content
rawBody, err := ioutil.ReadAll(response.Body)
	if err != nil {
		t.Error(err)
	}

// Restore the io.ReadCloser to it's original state
response.Body = ioutil.NopCloser(bytes.NewBuffer(rawBody))

Solution 4 - Go

Technically, on one reader, you cannot read multiple times.

  • Even if you create different references but
  • when you read once it will be same object referred by all references.
  • so what you can do is read the content and store it in one variable.
  • Then use that variable as many times as you want.

This will print twice.

package main

import (
	"io/ioutil"
	"log"
	"strings"
)

func main() {
	r := strings.NewReader("some io.Reader stream to be read\n")
	stringData, _ := ioutil.ReadAll(r)
	log.Println(stringData)
	log.Println(stringData)
}

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAbhishek SoniView Question on Stackoverflow
Solution 1 - GoTheHippoView Answer on Stackoverflow
Solution 2 - GolaurentView Answer on Stackoverflow
Solution 3 - GoSanjXView Answer on Stackoverflow
Solution 4 - GoVijay GurunaneeView Answer on Stackoverflow