JSON Serialization – Array or Record

I came across a recent support case, and I thought it was interesting because the issue was due to external products and it was a good example of finding information related to the case in external resources.

The way the issue presented itself was that a -8109 error was occurring with a util.JSON.parse() call.  This method takes a string that contains JSON formatted data and fills the destination member variables by name.  The destination variable is expected to have the same structure as the formatted JSON string and there is a set of conversion rules that dictate the parsing.

If you get the -8109 error, it is normally a case of comparing the structure of the JSON string with the destination variable and seeing where it differs.  The proposeType method is handy for this and chances are you used it as part of the development.  With a string that causes the error, you can use the result of the proposeType call on this string and compare it against your expected type.

What we discovered for this case was that there was an inconsistency in a child element of the destination record.  Sometimes the JSON string would have this child element as a record whilst in other times the JSON string would have this child element as an array.   There is no “sometimes” argument in programming languages, and there is normally a logical reason for any difference, and it was then noted that there were no array elements of length 1.  What was happening was that every time an array of length 1 was expected to be received, the JSON string had it as a record.

So normally the JSON String might contain an array with multiple elements … "name":[{"x":1,"y":2},{"x":3,"y":4}]} … which if there was only one element in the array should look like this … “name":[{"x":1,"y":2}] ….instead a record was being received … "name":{"x":1,"y":2} … If you look closely what is missing is the [] to signify an array.

So the problem ultimately was in the external product that was responsible for the JSON, it was not following standards.  It is not that easy to change what is coming from an external source  and so the solution for the developer was either to 1) parse in smaller bits via util.JSONObject and util.JSONArray methods,  or 2)  if there was only  a handful of such arrays in the destination record then they could use TRY/CATCH on a number of alternative destination records until one matched.

When I was reading the case, my thought was that surely we cannot be the only one with this problem and found a number of cases where developers in other languages encountered a similar issue upon receipt of a JSON String.  Examples included https://stackoverflow.com/questions/13575280/jersey-json-array-with-1-element-is-serialized-as-object and https://stackoverflow.com/questions/17003823/make-jackson-interpret-single-json-object-as-array-with-one-element.

Following links in these articles and others I found this article which I thought summed up the issue quite well and gave a plausible explanation how it was that these external products ended up producing these inconsistent JSON strings.  The comments in that article were interesting as well, particularly the one that said

Interestingly, more than one person on twitter has suggested that its a bug in the API, and therefore we should make the API owner fix their API.
This is a blinkered answer. If you’re in the business of writing software for clients, you’ll know that you often have to work with whatever is presented to you.

If you believe you are encountering this, use the above articles to help explain what is going to.  Consider the alternative techniques to parse the string, and ask your support consultant for the code example mentioned in the Issue Tracker .  Make sure you get added as a requestor to this case.  If the issue is more widespread than we think, then we should not rule out adding an option to cater for this inconsistent formatting of JSON strings.