Handling JSON Post Request in Go
JsonGoJson Problem Overview
So I have the following, which seems incredibly hacky, and I've been thinking to myself that Go has better designed libraries than this, but I can't find an example of Go handling a POST request of JSON data. They are all form POSTs.
Here is an example request: curl -X POST -d "{\"test\": \"that\"}" http://localhost:8082/test
And here is the code, with the logs embedded:
package main
import (
"encoding/json"
"log"
"net/http"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
req.ParseForm()
log.Println(req.Form)
//LOG: map[{"test": "that"}:[]]
var t test_struct
for key, _ := range req.Form {
log.Println(key)
//LOG: {"test": "that"}
err := json.Unmarshal([]byte(key), &t)
if err != nil {
log.Println(err.Error())
}
}
log.Println(t.Test)
//LOG: that
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
There's got to be a better way, right? I'm just stumped in finding what the best practice could be.
(Go is also known as Golang to the search engines, and mentioned here so others can find it.)
Json Solutions
Solution 1 - Json
Please use json.Decoder instead of json.Unmarshal.
func test(rw http.ResponseWriter, req *http.Request) {
decoder := json.NewDecoder(req.Body)
var t test_struct
err := decoder.Decode(&t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
Solution 2 - Json
You need to read from req.Body. The ParseForm method is reading from the req.Body and then parsing it in standard HTTP encoded format. What you want is to read the body and parse it in JSON format.
Here's your code updated.
package main
import (
"encoding/json"
"log"
"net/http"
"io/ioutil"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
body, err := ioutil.ReadAll(req.Body)
if err != nil {
panic(err)
}
log.Println(string(body))
var t test_struct
err = json.Unmarshal(body, &t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
Solution 3 - Json
There are two reasons why json.Decoder should be preferred over json.Unmarshal - that are not addressed in the most popular answer from 2013:
- February 2018,
go 1.10introduced a new method json.Decoder.DisallowUnknownFields() which addresses the concern of detecting unwanted JSON-input req.Bodyis already anio.Reader. Reading its entire contents and then performingjson.Unmarshalwastes resources if the stream was, say a 10MB block of invalid JSON. Parsing the request body, withjson.Decoder, as it streams in would trigger an early parse error if invalid JSON was encountered. Processing I/O streams in realtime is the preferred go-way.
Addressing some of the user comments about detecting bad user input:
To enforce mandatory fields, and other sanitation checks, try:
d := json.NewDecoder(req.Body)
d.DisallowUnknownFields() // catch unwanted fields
// anonymous struct type: handy for one-time use
t := struct {
Test *string `json:"test"` // pointer so we can test for field absence
}{}
err := d.Decode(&t)
if err != nil {
// bad JSON or unrecognized json field
http.Error(rw, err.Error(), http.StatusBadRequest)
return
}
if t.Test == nil {
http.Error(rw, "missing field 'test' from JSON object", http.StatusBadRequest)
return
}
// optional extra check
if d.More() {
http.Error(rw, "extraneous data after JSON object", http.StatusBadRequest)
return
}
// got the input we expected: no more, no less
log.Println(*t.Test)
Typical output:
$ curl -X POST -d "{}" http://localhost:8082/strict_test
expected json field 'test'
$ curl -X POST -d "{\"Test\":\"maybe?\",\"Unwanted\":\"1\"}" http://localhost:8082/strict_test
json: unknown field "Unwanted"
$ curl -X POST -d "{\"Test\":\"oops\"}g4rB4g3@#$%^&*" http://localhost:8082/strict_test
extraneous data after JSON
$ curl -X POST -d "{\"Test\":\"Works\"}" http://localhost:8082/strict_test
log: 2019/03/07 16:03:13 Works
Solution 4 - Json
I was driving myself crazy with this exact problem. My JSON Marshaller and Unmarshaller were not populating my Go struct. Then I found the solution at https://eager.io/blog/go-and-json:
> "As with all structs in Go, it’s important to remember that only > fields with a capital first letter are visible to external programs > like the JSON Marshaller."
After that, my Marshaller and Unmarshaller worked perfectly!
Solution 5 - Json
I found the following example from the docs really helpful (source here).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
func main() {
const jsonStream = `
{"Name": "Ed", "Text": "Knock knock."}
{"Name": "Sam", "Text": "Who's there?"}
{"Name": "Ed", "Text": "Go fmt."}
{"Name": "Sam", "Text": "Go fmt who?"}
{"Name": "Ed", "Text": "Go fmt yourself!"}
`
type Message struct {
Name, Text string
}
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var m Message
if err := dec.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Printf("%s: %s\n", m.Name, m.Text)
}
}
The key here being that the OP was looking to decode
type test_struct struct {
Test string
}
...in which case we would drop the const jsonStream, and replace the Message struct with the test_struct:
func test(rw http.ResponseWriter, req *http.Request) {
dec := json.NewDecoder(req.Body)
for {
var t test_struct
if err := dec.Decode(&t); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
log.Printf("%s\n", t.Test)
}
}
Update: I would also add that this post provides some great data about responding with JSON as well. The author explains struct tags, which I was not aware of.
Since JSON does not normally look like {"Test": "test", "SomeKey": "SomeVal"}, but rather {"test": "test", "somekey": "some value"}, you can restructure your struct like this:
type test_struct struct {
Test string `json:"test"`
SomeKey string `json:"some-key"`
}
...and now your handler will parse JSON using "some-key" as opposed to "SomeKey" (which you will be using internally).
Solution 6 - Json
type test struct {
Test string `json:"test"`
}
func test(w http.ResponseWriter, req *http.Request) {
var t test_struct
body, _ := ioutil.ReadAll(req.Body)
json.Unmarshal(body, &t)
fmt.Println(t)
}
Solution 7 - Json
I like to define custom structs locally. So:
// my handler func
func addImage(w http.ResponseWriter, r *http.Request) {
// define custom type
type Input struct {
Url string `json:"url"`
Name string `json:"name"`
Priority int8 `json:"priority"`
}
// define a var
var input Input
// decode input or return error
err := json.NewDecoder(r.Body).Decode(&input)
if err != nil {
w.WriteHeader(400)
fmt.Fprintf(w, "Decode error! please check your JSON formating.")
return
}
// print user inputs
fmt.Fprintf(w, "Inputed name: %s", input.Name)
}