topic Re: Json Indexing in Getting Data In

Json Indexing

nsiva23 — Mon, 28 Sep 2020 16:22:00 GMT

We are trying parse a Json file for indexing. While parsing we have two events in the json file mentioned below

[
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "Windows",
"host": "10.10.10.10",
"hardwareRevision": "0200"
},
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "LINUX",
"host": "10.10.10.20",
"hardwareRevision": "NA"
}
]

I am declaring the following params in props.conf

KV_MODE=json
NO_BINARY_CHECK=1
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N

But indexer recognizes only one event instead of two. It doesn't recognizes second time stamp. I don't see any examples of props.conf for Json on the internet. Is there is any samples of props.conf how it should look like for the Json file.

Thanks in advance.

Re: Json Indexing

hkeebler — Mon, 28 Sep 2020 19:40:22 GMT

Did anyone ever answer your question? I have the same issue now. I have something that is all json and includes a json array "outages: [{event1},{event2},....] ". I cannot get it to recognize separate events, even with the BREAK_ONLY_BEFORE = clause.

Re: Json Indexing

fdi01 — Fri, 24 Apr 2015 15:00:56 GMT

Since you have multiple time stamps in the event, you need select the right one for Splunk to recognize.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition

Your format should be the :

timestamp": "2014-04-07 00:00:36.297",

Portion from your json.

So in your props, your timestamp format should be something like:

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
 TIME_PREFIX= \t

I'm not sure if there is a tab, you could also try

TIME_PREFIX = (AM\s|PM\s)

That looks for AM OR PM (not its followed by a space. That might be a tab in your json..)

Re: Json Indexing

hkeebler — Mon, 28 Sep 2020 19:40:33 GMT

Thanks so much for the response. This still doesn't work for me. My sample data is below. I don't really care if any of the dates are used as timestamps. Today's date as default would be fine. I just want separate events to be recognized.

{"respId":"search","status":"0","events":[{"name":"event1","startDate":"20150404:1600",...move stuff,"dateReceived":"20150405:0000"},{"name":"event2",.........}],"count":"500"}

If I use the default _json (Indexed_extractions = json) I end up with fields respId, status, events{}:name, events{}:startDate, events{}:dateReceived etc. where all the data for these fields is now crammed into the one field - i.e., one event.
My props.conf fields are
INDEXED_EXTRACTIONS = json
TIME_FORMAT = %Y%m%d:%H%M
BREAK_ONLY_BEFORE = {"name"
TIME_PREFIX = dateReceived":