(I previously asked this in a more vague CSV context.) I am using file monitor to ingest data from an API that returns JSON. I have to split the returns into individual files with host name in file path so I can use host_regex to force host field at index time.
Is there an easier way to persuade the monitor so I can write returns into the same file? (There are advantages of writing smaller files. But there are also disadvantages of writing numerous files and having numerous "sources".) The last time I tried with CSV, setting a field with name "host" doesn't seem to be much of a persuasion, as the indexer renamed "host" field as "detected_host" with that value, instead of setting "host" directly to the source "host" value.
With JSON, the "host" field value is coalesced into indexed "host" field as a second value (whether the value is the same as the "other"/"default" value or not). Multivalue "host" can be an even bigger problem if the original JSON happens to contain a field named "host". (Not in the APIs that I am testing but there could be.)
Or is this caused by something wrong with my test method?
