How to pipe an HTTP response to a file in Go?
GoGo Problem Overview
How do I convert the below code to use streams/pipes so that I don't need to read the full content into memory?
Something like:
http.Get("http://example.com/").Pipe("./data.txt")
package main
import ("net/http";"io/ioutil")
func main() {
resp, err := http.Get("http://example.com/")
check(err)
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
check(err)
err = ioutil.WriteFile("./data.txt", body, 0666)
check(err)
}
func check(e error) {
if e != nil {
panic(e)
}
}
Go Solutions
Solution 1 - Go
How about io.Copy()
? Its documentation can be found at: http://golang.org/pkg/io/#Copy
It's pretty simple, though. Give it an io.Reader
and an io.Writer
and it copies the data over, one small chunk at a time (e.g. not all in memory at once).
So you might try writing something like:
func main() {
resp, err := http.Get("...")
check(err)
defer resp.Body.Close()
out, err := os.Create("filename.ext")
if err != nil {
// panic?
}
defer out.Close()
io.Copy(out, resp.Body)
}
I haven't tested the above; I just hacked it together quickly from your above example, but it should be close if not on the money.
Solution 2 - Go
Another option is File.ReadFrom:
package main
import (
"net/http"
"os"
)
func main() {
r, e := http.Get("http://speedtest.lax.hivelocity.net")
if e != nil {
panic(e)
}
defer r.Body.Close()
f, e := os.Create("index.html")
if e != nil {
panic(e)
}
defer f.Close()
f.ReadFrom(r.Body)
}