From io.Reader to string in Go

Go

Go Problem Overview


I have an io.ReadCloser object (from an http.Response object).

What's the most efficient way to convert the entire stream to a string object?

Go Solutions


Solution 1 - Go

EDIT:

Since 1.10, strings.Builder exists. Example:

buf := new(strings.Builder)
n, err := io.Copy(buf, r)
// check errors
fmt.Println(buf.String())

OUTDATED INFORMATION BELOW

The short answer is that it it will not be efficient because converting to a string requires doing a complete copy of the byte array. Here is the proper (non-efficient) way to do what you want:

buf := new(bytes.Buffer)
buf.ReadFrom(yourReader)
s := buf.String() // Does a complete copy of the bytes in the buffer.

This copy is done as a protection mechanism. Strings are immutable. If you could convert a []byte to a string, you could change the contents of the string. However, go allows you to disable the type safety mechanisms using the unsafe package. Use the unsafe package at your own risk. Hopefully the name alone is a good enough warning. Here is how I would do it using unsafe:

buf := new(bytes.Buffer)
buf.ReadFrom(yourReader)
b := buf.Bytes()
s := *(*string)(unsafe.Pointer(&b))

There we go, you have now efficiently converted your byte array to a string. Really, all this does is trick the type system into calling it a string. There are a couple caveats to this method:

  1. There are no guarantees this will work in all go compilers. While this works with the plan-9 gc compiler, it relies on "implementation details" not mentioned in the official spec. You can not even guarantee that this will work on all architectures or not be changed in gc. In other words, this is a bad idea.
  2. That string is mutable! If you make any calls on that buffer it will change the string. Be very careful.

My advice is to stick to the official method. Doing a copy is not that expensive and it is not worth the evils of unsafe. If the string is too large to do a copy, you should not be making it into a string.

Solution 2 - Go

Answers so far haven't addressed the "entire stream" part of the question. I think the good way to do this is ioutil.ReadAll. With your io.ReaderCloser named rc, I would write,

Go >= v1.16

if b, err := io.ReadAll(rc); err == nil {
    return string(b)
} ...

Go <= v1.15

if b, err := ioutil.ReadAll(rc); err == nil {
    return string(b)
} ...

Solution 3 - Go

data, _ := ioutil.ReadAll(response.Body)
fmt.Println(string(data))

Solution 4 - Go

The most efficient way would be to always use []byte instead of string.

In case you need to print data received from the io.ReadCloser, the fmt package can handle []byte, but it isn't efficient because the fmt implementation will internally convert []byte to string. In order to avoid this conversion, you can implement the fmt.Formatter interface for a type like type ByteSlice []byte.

Solution 5 - Go

func copyToString(r io.Reader) (res string, err error) {
	var sb strings.Builder
	if _, err = io.Copy(&sb, r); err == nil {
		res = sb.String()
	}
	return
}

Solution 6 - Go

var b bytes.Buffer
b.ReadFrom(r)

// b.String()

Solution 7 - Go

I like the bytes.Buffer struct. I see it has ReadFrom and String methods. I've used it with a []byte but not an io.Reader.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiondjdView Question on Stackoverflow
Solution 1 - GoStephen WeinbergView Answer on Stackoverflow
Solution 2 - GoSoniaView Answer on Stackoverflow
Solution 3 - Goyakob abadaView Answer on Stackoverflow
Solution 4 - Gouser811773View Answer on Stackoverflow
Solution 5 - GoDimchanskyView Answer on Stackoverflow
Solution 6 - GoVojtech VitekView Answer on Stackoverflow
Solution 7 - GoNateView Answer on Stackoverflow