Decode always clobbers existing values in destination map for nested struct fields.

[skaggmannen]

Description

We are using mapstructure to deal with JSON payloads where our application only knows about a sub-set of the fields. We do this by unmarshalling the JSON to a map[string]any, and then use mapstructure.Decode() to decode to a struct containing only the fields we know about.

Then when we need to commit changes made to the struct we use mapstructure.Decode() to copy the fields to the map[string]any, which we then marshal into JSON.

So very simplified, this is what we're doing:

someJSON := readDataFromSomewhere()

var m map[string]any
json.Unmarshal(someJSON, &m)

var s struct {
	A int
	B struct {
		C int
	}
}
mapstructure.Decode(m, &s)

s.B.C = 42;

mapstructure.Decode(s, &m)
someJSON, _ = json.Marshal(m)

writeDataSomeWhere(someJSON)

The issue

What we noticed is that any nested structs used will "erase" all other fields for the nested map in the destination.

So if the input in the example above looks like this:

{
  "A": 1,
  "B": {
    "C": 2,
    "D": "we don't know about this field"
  },
  "E": "another field we don't know about" 
}

The output would look like this:

{
  "A": 1,
  "B": {
    "C": 42
  }
  "E": "another field we don't know about"
}

This erases the D field from the final output, which is less than desirable in our use case!

Proposed solution

I've created a pull request with a proposed new config flag called MergeIntoExisting which will make mapstructure.Decoder recursively merge the input into the destination for all nested structs: #61

I have tested that the changes works for our use case but I also purposefully limited it to only be applied when decoding structs to maps, since I'm not sure how generally applicable this is to all other cases (e.g. map-to-map or map-to-struct).