JSON time field not parsed correctly

wsw70 — Wed, 03 Dec 2014 12:03:39 GMT

Hello

I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not.

My props.conf file:

[nessusjson]
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = 'N_scantime'
TIME_FORMAT = '%s'
MAX_TIMESTAMP_LOOKAHEAD = -1

I also tried KV_MODE = json.

Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT.

11/27/14 
3:22:38.000 PM  
{ [-] 
   N_exploit: false 
   N_exploit_malware: UNKNOWN 
   N_exploit_metasploit: UNKNOWN 
   N_exploit_metasploit_name: UNKNOWN 
   N_nettype: wazaatype 
   N_scantime: 1417079828 
   N_subnetname: wazaa name 
   N_timeduration: 680 
   N_timeend: Thu Nov 27 14:33:58 2014 
   N_timeend_epoch: 1417098838 
   N_timestart: Thu Nov 27 14:22:38 2014 
   N_timestart_epoch: 1417098158 
   N_vendor: java 
}

The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time...).
I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD). I restarted the server for each test.

It looks like the N_timestart field was parsed instead.

Is there something I am still missing?

Thank you for any pointers!

Re: JSON time field not parsed correctly

wsw70 — Wed, 03 Dec 2014 13:38:19 GMT

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

topic Re: JSON time field not parsed correctly in Getting Data In

JSON time field not parsed correctly

Re: JSON time field not parsed correctly