Getting Data In
Highlighted

props.conf timestamp clarification

Path Finder

I have json data that can vary greatly in size with the timestamp field coming at the end of each event. I'm able to parse all the timestamps correctly using the config TIMEPREFIX="timestamp":+ except for the events that are very large. My question is, in order to parse the timestamp for the very large events, do I need to add a MAXTIMESTAMPLOOKAHEAD? Or if I added a larger TRUNCATE would the TIMEPREFIX config still need the MAXTIMESTAMPLOOKAHEAD?

props.conf
[mysourcetype]
CHARSET=UTF-8
INDEXEDEXTRACTIONS=json
KV
MODE=none
LINEBREAKER=([\r\n]+)
NO
BINARYCHECK=true
SHOULD
LINEMERGE=true
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldowntype=true
TIME
PREFIX="timestamp":+

0 Karma
Highlighted

Re: props.conf timestamp clarification

SplunkTrust
SplunkTrust

The MAX_TIMESTAMP_LOOKAHEAD settings starts at TIME_PREFIX so changing it won't help. It's likely you're running into your TRUNCATE limit. Try increasing that after you make sure events are breaking correctly.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

Highlighted

Re: props.conf timestamp clarification

Communicator

As @richgalloway rightly pointed, you should look into increasing the value of TRUNCATE (Defaults to 10,000). Splunk logs it's complain regarding the truncate issues in splunkd.log inside $SPLUNK_HOME/var/log/splunk. You can check it, to make sure you're facing the same issue.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.