Getting Data In

JSON time field not parsed correctly

wsw70
Communicator

Hello

I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not.

My props.conf file:

[nessusjson]
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = 'N_scantime'
TIME_FORMAT = '%s'
MAX_TIMESTAMP_LOOKAHEAD = -1

I also tried KV_MODE = json.

Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT.

11/27/14 
3:22:38.000 PM  
{ [-] 
   N_exploit: false 
   N_exploit_malware: UNKNOWN 
   N_exploit_metasploit: UNKNOWN 
   N_exploit_metasploit_name: UNKNOWN 
   N_nettype: wazaatype 
   N_scantime: 1417079828 
   N_subnetname: wazaa name 
   N_timeduration: 680 
   N_timeend: Thu Nov 27 14:33:58 2014 
   N_timeend_epoch: 1417098838 
   N_timestart: Thu Nov 27 14:22:38 2014 
   N_timestart_epoch: 1417098158 
   N_vendor: java 
}

The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time...).
I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD). I restarted the server for each test.

It looks like the N_timestart field was parsed instead.

Is there something I am still missing?

Thank you for any pointers!

0 Karma
1 Solution

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

View solution in original post

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

Get Updates on the Splunk Community!

What the End of Support for Splunk Add-on Builder Means for You

Hello Splunk Community! We want to share an important update regarding the future of the Splunk Add-on Builder ...

Solve, Learn, Repeat: New Puzzle Channel Now Live

Welcome to the Splunk Puzzle PlaygroundIf you are anything like me, you love to solve problems, and what ...

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...