Why am I getting sourcetype errors after trying to configure Splunk to parse json and proper timestamp recognition?

[tomeumir]

Hi,
I am trying Splunk and try to evaluate it as a tool for managing the logs of our in-house applications. I am uploading a file with thousands of lines like the following ones (2 sample lines):
{"time_stamp":"2014-10-06 18:00:00.025956","message_id":2459316836,"rq_id":8036007057,"rs_id":8036007059,"status":"OK","message_type":"OTA_HotelAvailRQ","host":"0.0.0.0","elapsed_time":155,"subsystem":"Player","connectorhost"":localhost","resstatus":"null","requestorid":"TRPS"}
{"time_stamp":"2014-10-06 18:00:00.041755","message_id":2459316838,"rq_id":8036007025,"rs_id":8036007010,"status":"OK","message_type":"OTA_HotelAvailRQ","host":"0.0.0.0","elapsed_time":31,"subsystem":"Player","connectorhost""localhost","resstatus":"null","requestorid":"TREP"}

But after uploading the file I cannot set the proper SourceType. I am choosing _json and setting
- TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N %Z
- TIME_FIELD= time_stamp

But I get the following message: No results found. Please change Sourcetype, adjust Sourcetype settings, or check your source file.

I will appreciate any help.

Thanks a lot in advanced.

Tom.

[aweitzman]

First of all, this needs to be correct as you upload the file, so changing anything here will not change what's in Splunk unless you delete it out of Splunk and re-upload your file.

Second, your TIME_FORMAT differs from your actual input in the following three ways:

1. You have a "T" separating the date from the time in the TIME_FORMAT, but it's not there in the data.
2. You have a "%Z" for the timezone in your TIME_FORMAT, but it's not there in your input.
3. You have a "%3N" for milliseconds in your TIME_FORMAT, but you appear to have microseconds there, which would mean you want to use "%6N" instead.

Third, you can't use TIME_FIELD here since that's used for search-time parsing, not index-time parsing, which is when the sourcetype is assigned. At the point that indexing happens, the JSON parsing has not taken place, and you don't have a time_stamp field yet. You need to use the TIME_PREFIX line in your stanza, like this:

TIME_PREFIX = time_stamp":

Read this for more details: http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Configuretimestamprecognition

What happens when you fix those?

[tomeumir · Accepted solution]

Hi there,
It seems that the time stamp field must be the first one. If so, then choosing just _json as source type all fields are parsed well automatically and _time is calculated based on my time stamp.
Rgds,
T.

[aweitzman]

Glad it works!