Getting Data In

Json Indexing

nsiva23
Explorer

We are trying parse a Json file for indexing. While parsing we have two events in the json file mentioned below

[
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "Windows",
"host": "10.10.10.10",
"hardwareRevision": "0200"
},
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "LINUX",
"host": "10.10.10.20",
"hardwareRevision": "NA"
}
]

I am declaring the following params in props.conf

KV_MODE=json
NO_BINARY_CHECK=1
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N

But indexer recognizes only one event instead of two. It doesn't recognizes second time stamp. I don't see any examples of props.conf for Json on the internet. Is there is any samples of props.conf how it should look like for the Json file.

Thanks in advance.

Tags (2)

fdi01
Motivator

Since you have multiple time stamps in the event, you need select the right one for Splunk to recognize.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition

Your format should be the :

timestamp": "2014-04-07 00:00:36.297",

Portion from your json.

So in your props, your timestamp format should be something like:

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
 TIME_PREFIX= \t

I'm not sure if there is a tab, you could also try

TIME_PREFIX = (AM\s|PM\s)

That looks for AM OR PM (not its followed by a space. That might be a tab in your json..)

hkeebler
Engager

Thanks so much for the response. This still doesn't work for me. My sample data is below. I don't really care if any of the dates are used as timestamps. Today's date as default would be fine. I just want separate events to be recognized.

{"respId":"search","status":"0","events":[{"name":"event1","startDate":"20150404:1600",...move stuff,"dateReceived":"20150405:0000"},{"name":"event2",.........}],"count":"500"}

If I use the default _json (Indexed_extractions = json) I end up with fields respId, status, events{}:name, events{}:startDate, events{}:dateReceived etc. where all the data for these fields is now crammed into the one field - i.e., one event.
My props.conf fields are
INDEXED_EXTRACTIONS = json
TIME_FORMAT = %Y%m%d:%H%M
BREAK_ONLY_BEFORE = {"name"
TIME_PREFIX = dateReceived":

0 Karma

hkeebler
Engager

Did anyone ever answer your question? I have the same issue now. I have something that is all json and includes a json array "outages: [{event1},{event2},....] ". I cannot get it to recognize separate events, even with the BREAK_ONLY_BEFORE = clause.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...