In Go, how can I convert a struct to a byte array?
ArraysStructGoByteArrays Problem Overview
I have an instance of a struct that I defined and I would like to convert it to an array of bytes. I tried []byte(my_struct), but that did not work. Also, I was pointed to the http://golang.org/pkg/encoding/binary/">binary package, but I am not sure which function I should use and how I should use it. An example would be greatly appreciated.
Arrays Solutions
Solution 1 - Arrays
One possible solution is the "encoding/gob" standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.
As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.
I've included some code and a play.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type P struct {
X, Y, Z int
Name string
}
type Q struct {
X, Y *int32
Name string
}
func main() {
// Initialize the encoder and decoder. Normally enc and dec would be
// bound to network connections and the encoder and decoder would
// run in different processes.
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
// Encode (send) the value.
err := enc.Encode(P{3, 4, 5, "Pythagoras"})
if err != nil {
log.Fatal("encode error:", err)
}
// HERE ARE YOUR BYTES!!!!
fmt.Println(network.Bytes())
// Decode (receive) the value.
var q Q
err = dec.Decode(&q)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}
Solution 2 - Arrays
I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.
Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.
EDIT:
Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct)) like so: http://play.golang.org/p/yY8mSdZ_kf
Solution 3 - Arrays
I know this thread is old, but none of the answers were accepted, and there's a pretty simple way to do this.
https://play.golang.org/p/TedsY455EBD
important code from playground
import (
"bytes"
"fmt"
"encoding/json"
)
type MyStruct struct {
Name string `json:"name"`
}
testStruct := MyStruct{"hello world"}
reqBodyBytes := new(bytes.Buffer)
json.NewEncoder(reqBodyBytes).Encode(testStruct)
reqBodyBytes.Bytes() // this is the []byte
Solution 4 - Arrays
Just use json marshal， this is a very simple way.
newFsConfig := dao.ConfigEntity{EnterpriseId:"testing"}
newFsConfigBytes, _ := json.Marshal(newFsConfig)
Solution 5 - Arrays
Serialization is likely proper answer.
But if you consent to unsafety and actually need to read struct as bytes, then relying on byte array memory representation might be a bit better than relying on byte slice internal structure.
type Struct struct {
Src int32
Dst int32
SrcPort uint16
DstPort uint16
}
const sz = int(unsafe.SizeOf(Struct{}))
var asByteSlice []byte = (*(*[sz]byte)(unsafe.Pointer(&struct_value)))[:]
Works and provides read-write view into struct, zero-copy. Two "unsafe" should hint enough that it may break badly.
Solution 6 - Arrays
You should use a bytes buffer instead of a string, the other suggested methods create a SHA1 of variable length, the SHA1 standard length must be 20 bytes (160 bits)
package main
import (
"crypto/sha1"
"fmt"
"encoding/binary"
"bytes"
)
type myStruct struct {
ID string
Data string
}
func main() {
var bin_buf bytes.Buffer
x := myStruct{"1", "Hello"}
binary.Write(&bin_buf, binary.BigEndian, x)
fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}
Try it yourself: http://play.golang.org/p/8YuM6VIlLV
It's a really easy method and it works great.
Solution 7 - Arrays
package main
import (
"crypto/sha1"
"fmt"
"encoding/binary"
"bytes"
)
type myStruct struct {
ID [10]byte
Data [10]byte
}
func main() {
var bin_buf bytes.Buffer
x := myStruct{"1", "Hello"}
binary.Write(&bin_buf, binary.BigEndian, x)
fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}
binary.Write takes a struct which has fixed length memory allocated datatype.
Solution 8 - Arrays
json.Marshal is the best option to convert a struct to []byte, see example below:
package main
import (
"encoding/json"
"fmt"
)
type ExampleConvertToByteArray struct {
Name string
SurName string
}
func main() {
example := ExampleConvertToByteArray{
Name: "James",
SurName: "Camara",
}
var exampleBytes []byte
var err error
exampleBytes, err := json.Marshal(example)
if err != nil {
print(err)
return
}
fmt.Println(string(exampleBytes))
}
Go playground -> https://play.golang.org/p/mnB9Cxy-2H3
Solution 9 - Arrays
Take a look at https://blog.golang.org/go-slices-usage-and-internals Specifically slice internals. The idea is to mimic slice's internal structure and point to our struct instead of a byte sequence:
package main
import (
"fmt"
"unsafe"
)
// our structure
type A struct {
Src int32
Dst int32
SrcPort uint16
DstPort uint16
}
// that is how we mimic a slice
type ByteSliceA struct {
Addr *A
Len int
Cap int
}
func main() {
// structure with some data
a := A{0x04030201,0x08070605,0x0A09, 0x0C0B}
// create a slice structure
sb := &ByteSliceA{&a, 12, 12} // struct is 12 bytes long, e.g. unsafe.Sizeof(a) is 12
// take a pointer of our slice mimicking struct and cast *[]byte on it:
var byteSlice []byte = *(*[]byte)(unsafe.Pointer(sb))
fmt.Printf("%v\n", byteSlice)
}
Output:
[1 2 3 4 5 6 7 8 9 10 11 12]
Solution 10 - Arrays
Have you considered serializing it to bson? http://labix.org/gobson