Getting Data In

How do I configure line breaking on the timestamp in my data?

tkwaller
Builder

Need to line break on timestamp in the following log record, but the underscore in the timestamp is throwing me off. Any help would be appreciated and thank you!

2015-07-28_09:51:02 INFO App=EXTRACT_HIT_DATA Message=::' Checking for backlog file/files to process '
2015-07-28_09:51:02 INFO App=EXTRACT_HIT_DATA Message=:' Number of backlog files : 48'
2015-07-28_09:51:02 INFO App=EXTRACT_HIT_DATA Message=:' Start processing all available backlog files '
2015-07-28_09:51:02 WARNING App=EXTRACT_HIT_DATA Message=:' File /nas/abc/myx/myarchive/omniture/feed/.../xyz.fin still not available, will try to process the file again during next run.'
2015-07-28_09:51:02 WARNING App=EXTRACT_HIT_DATA Message=:' File /nas/abc/myx/myarchive/omniture/feed/.../xyz.fin still not available, will try to process the file again during next run.'
2015-07-28_09:51:02 WARNING App=EXTRACT_HIT_DATA Message=:' File /nas/abc/myx/myarchive/omniture/feed/.../xyz.fin still not available, will try to process the file again during next run.'
Tags (2)
0 Karma
1 Solution

jnussbaum_splun
Splunk Employee
Splunk Employee

you'll want to create a props.conf file that sits with your inputs.conf file. - Change "yoursourcetype" to the sourcetype you've defined in inputs.conf
[yoursourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 20
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d_%H:%M:%S
TIME_PREFIX = ^

View solution in original post

jnussbaum_splun
Splunk Employee
Splunk Employee

you'll want to create a props.conf file that sits with your inputs.conf file. - Change "yoursourcetype" to the sourcetype you've defined in inputs.conf
[yoursourcetype]
MAX_TIMESTAMP_LOOKAHEAD = 20
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d_%H:%M:%S
TIME_PREFIX = ^

tkwaller
Builder

Will that break at timestamp?
I think the lookahead and no linemerge would allow it to break properly is that correct?

0 Karma

wdsjon
Engager

Yep - Splunk will automatically break on the timestamp unless you manually define a line breaker.

0 Karma

bmacias84
Champion

Within your props.conf you need to specify the ** TIME_FORMAT**.

#props.conf
[mysource]
TIME_FORMAT =  < strptime >

Use this reference Configuretimestamprecognition

tkwaller
Builder

So what would that make LINE_BREAKER look like then or should I use it? Considering I need to break on timestamp.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...