DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event Warning is coming in a JSON data load for EPOCH TIMESTAMP

[pbankar]

Hello, I'm trying to get the timestamp from the below JSON data. I have two type of JSON data to be indexed in Splunk as below.
NOTE Below JSON is copied from the Splunk event by clicking the - "Show as raw text"

JSON DATA for

sourcetype = [lastFound]

{"vulnerability": null, "lastFound": "1511257640614", "os": null, "authType": ["UNIX_AUTH"], "supportedBy": ["VM", "CA-Linux Agent", "CA-Mac Agent"], "discoveryType": ["AUTHENTICATED"], "port": null, "firstFound": "1511257640614"}

JSON DATA for

sourcetype = [lastScanned]

{"size": 520227288, "created": "1432662156000", "vulnerabilities": {"severity2Count": 2, "severity5Count": 11, "severity3Count": 38, "severity1Count": 0, "severity4Count": 30}, "lastScanned": "1516151267981"}

My prop.conf is

[lastFound]
SHOULD_LINEMERGE=false
KV_MODE=json
INDEXED_EXTRACTIONS= JSON
DATETIME_CONFIG = CURRENT
MAX_EVENTS = 50000
TIME_PREFIX=lastFound:
TIME_FORMAT=%s%3N
MAX_TIMESTAMP_LOOKAHEAD=13
TZ=UTC

[lastScanned]
SHOULD_LINEMERGE=false
KV_MODE=json
INDEXED_EXTRACTIONS= JSON
DATETIME_CONFIG = CURRENT
MAX_EVENTS = 50000
TIME_PREFIX=lastScanned:
TIME_FORMAT=%s%3N
MAX_TIMESTAMP_LOOKAHEAD=13
TZ=UTC

I'm curious sometime it works to give me the eventdate as per the "lastFound" but sometime it dosen't. Is it related to the WARNING I'm getting?

P.S. "lastScanned" never worked!!! Please do suggest me if I'm doing anything unexpected.

[p_gurav · Accepted solution]

hi ,

Try for last found:
TIME_PREFIX = \"lastFound\":\s
TIME_FORMAT = %s

Try for last scanned:
TIME_PREFIX = \"lastScanned\":\s
TIME_FORMAT = %s

[pbankar]

Hi @p_gurav,

Thanks for your comment. I tried below as your sugesstion was good but didn't worked for me.

[lastFound]
 SHOULD_LINEMERGE=false
 KV_MODE=json
 INDEXED_EXTRACTIONS= JSON
 DATETIME_CONFIG = CURRENT
 MAX_EVENTS = 50000
 TIME_PREFIX=\"lastFound\":\s
 TIME_FORMAT=%s%3N
 #MAX_TIMESTAMP_LOOKAHEAD=13 # commented this line. So it worked for the lastFound
 TZ=UTC

 [lastScanned]
 SHOULD_LINEMERGE=false
 KV_MODE=json
 INDEXED_EXTRACTIONS= JSON
 DATETIME_CONFIG = CURRENT
 MAX_EVENTS = 50000
 TIME_PREFIX=\"lastScanned\":\s
 TIME_FORMAT=%s%3N
# MAX_TIMESTAMP_LOOKAHEAD=13 # commented same but not working here. Not sure Why?
 TZ=UTC

[p_gurav]

Hi,

Can you try :

[sample_json]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = lastFound
TIME_FORMAT = %s%3N
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

[sample_json1]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = lastScanned
TIME_FORMAT = %s%3N
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

[pbankar]

@p_gurav I need one more help from you.

In both of the JSON, there are 2 epoch time fields.

sourcetype = [lastFound]

{"vulnerability": null, "lastFound": "1511257640614", "os": null, "authType": ["UNIX_AUTH"], "supportedBy": ["VM", "CA-Linux Agent", "CA-Mac Agent"], "discoveryType": ["AUTHENTICATED"], "port": null, "firstFound": "1511257640614"}

JSON DATA for

sourcetype = [lastScanned]

{"size": 520227288, "created": "1432662156000", "vulnerabilities": {"severity2Count": 2, "severity5Count": 11, "severity3Count": 38, "severity1Count": 0, "severity4Count": 30}, "lastScanned": "1516151267981"}

I need to provide an OR case for the TIMESTAMP_FIELDS.
I tried this but it didn't work.
TIMESTAMP_FIELDS=lastScanned|created

Could you suggest something?

[pbankar]

@p_gurav

Below is working for me:

[lastScanned]
INDEXED_EXTRACTIONS=json
KV_MODE=json
MAX_EVENTS=50000
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIMESTAMP_FIELDS=lastScanned
TIME_FORMAT=%s%3N
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true

[p_gurav]

Great. Happy to help!!!

[pbankar]

Thanks @p_gurav