json export consistency

[donelliot]

I'm trying to export results as json as part of an integration, but can't seem to enforce the generated json to always be arrays for certain fields which sometimes only has one value.. in those case it reverts to a string and causes headaches downstream. 

The data has been stored in a kvstore and I'm wanting to fix it pre or post storage in the kvstore before executing the json generating command..

one thing i found is that if I act on the field with mvcombine the functions tojson and makejson seems to realize how to covert it.. but if I do that on fields that are multiple value it messes them up.. and puts a \n in between my values

[donelliot]

Ok I was able to get a work around of sorts... 

I suspect it would be resolved if i used rest to create the item in the first place, as I then would be in charge of explicitly declaring that some of these fields are arrays ( and then I wouldn't have to rely on the magic of tojson to deduce what it can't possibly tell).  (I';m using outputlookup to populate the very simple structure)

The workaround - was just to accept that the kvstore stores the values, but realizing that reissuing a stats(value) command generates true mv fields which tojson pays attention to allows me to force the array json generation in this context.. and it does that even if there is only one value in the field which is destined to be sent as an array (in this requirement)

so before generating sending the json on it's way I issue a stats command for the record to be sent....

inputlookup kvstoreresource where ....
stats latest(action_type) as action_type,latest(asset_plugin_count) as asset_plugin_count, values(asset_uuid) as asset_uuid...

here I explicitly am giving the tojson command which follows a heads up which fields are to be treated as arrays (the ones where I uses an mv generating command like list, values) and the fields which are just to be treated as single values

so when I issue a tojson auto(*) command it takes the queues from the stats command

Not sure if this is ideal. but it works and the volume of the integration is tiny - one record per day if that 🙂 

I was hoping that kvstore would have understood when it was populated with outputlookup that the single value fields generated with multi values were to be stored and exported as arrays avoiding this overhead