go.pennock.tech/swallowjson

Golang: Support for preserving unknown fields from JSON when unmarshalling instead of discarding

Current public git hosting (GitHub)
Go package documentation

swallowjson

The swallowjson Golang library provides a simple-to-use support function to use to implement an interface method in your type, to aid in JSON decoding.

When decoding JSON into a struct, the default Golang handling is to discard the JSON fields encountered which have not been declared as struct fields. The UnmarshalWith function lets you implement the UnmarshalJSON method as a one-liner, declaring a map within the struct which should swallow all JSON fields not otherwise accepted by the struct.

A simple example:

type MyType struct {
	Foo  string                 `json:"foo"`
	Bar  int                    `json:"bar"`
	Rest map[string]interface{} `json:"-"`
}

func (mt *MyType) UnmarshalJSON(raw []byte) error {
	return swallowjson.UnmarshalWith(mt, "Rest", raw)
}

You can then decode as normal for Golang, letting the JSON decoder dispatch to your overridden UnmarshalJSON method whenever it expects to decode a MyType (whether at the top-level, or nested inside other types, etc):

var myData MyType
if err := json.Unmarshal(rawBytes, &myData); err != nil {
	processError(err)
}

When invoked on mt, which should already exist as a struct, swallowjson.UnmarshalWith will populate Foo and Bar from JSON fields foo and bar respectively, per normal Golang decoding rules. But if the JSON also contains fields baz and bat then those will end up as keys, holding their child data, in the Rest map.

This library was written as a fairly quick proof-of-concept for a friend and later gained sufficient tests as to be something which can be relied upon as a core component, with no other dependencies. The library is released in the hopes that it might prove useful to others.

Behavior notes:

The Golang encoding/json library has a bunch of legacy case-insensitivity handling in the default unmarshaller; while swallowjson builds upon that library (it uses json.NewDecoder() under the hood) our API is a new API, implemented in new code and is not a drop-in replacement. Thus there is no need to preserve backwards behavior here, so I didn’t implement that case-insensitivity.
The Rest map will be created on-demand; if no unexpected keys are seen and the map is nil going in, then it will still be nil afterwards.
The Rest map can have arbitrary value types, but if the content won’t parse then you’ll get an error. Sensible choices for generic usage include interface{} and json.RawMessage.

Canonical import path is: go.pennock.tech/swallowjson