Getting Data In

JSON time field not parsed correctly

wsw70
Communicator

Hello

I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not.

My props.conf file:

[nessusjson]
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = 'N_scantime'
TIME_FORMAT = '%s'
MAX_TIMESTAMP_LOOKAHEAD = -1

I also tried KV_MODE = json.

Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT.

11/27/14 
3:22:38.000 PM  
{ [-] 
   N_exploit: false 
   N_exploit_malware: UNKNOWN 
   N_exploit_metasploit: UNKNOWN 
   N_exploit_metasploit_name: UNKNOWN 
   N_nettype: wazaatype 
   N_scantime: 1417079828 
   N_subnetname: wazaa name 
   N_timeduration: 680 
   N_timeend: Thu Nov 27 14:33:58 2014 
   N_timeend_epoch: 1417098838 
   N_timestart: Thu Nov 27 14:22:38 2014 
   N_timestart_epoch: 1417098158 
   N_vendor: java 
}

The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time...).
I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD). I restarted the server for each test.

It looks like the N_timestart field was parsed instead.

Is there something I am still missing?

Thank you for any pointers!

0 Karma
1 Solution

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

View solution in original post

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...