Getting Data In
Highlighted

Why isn't the timestamp being recognized ?

Ultra Champion

We have a case that looks like this -

alt text

So, the events are not broken by the timestamp. Is it because of the underscore between the date and time?

Tags (1)
0 Karma
Highlighted

Re: Why isn't the timestamp being recognized ?

SplunkTrust
SplunkTrust

Most likely the underscores are throwing it off. What's your props.conf look like?

0 Karma
Highlighted

Re: Why isn't the timestamp being recognized ?

SplunkTrust
SplunkTrust

Splunk has a number of built in time formats, I would recommend you manually parse your timestamps to improve indexing performance and ensure that you have the timestamps consistently recognised.

In this case in your props.conf

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N

Also refer to configure timestamp recognition. I agree with skoelpin it's probably the _ throwing the default timestamp parsing off.

View solution in original post

0 Karma
Highlighted

Re: Why isn't the timestamp being recognized ?

Ultra Champion

Gorgeous - thank you @garethatiagg

Highlighted

Re: Why isn't the timestamp being recognized ?

SplunkTrust
SplunkTrust

Remember that this is a start and you should apply line breaking to your props.conf aswell.. This will take some load off your indexers

Highlighted

Re: Why isn't the timestamp being recognized ?

SplunkTrust
SplunkTrust

The documentation also covers line breaking

This is a slightly more advanced topic but potentially worth discussing here, the default for the MAXEVENTS in props.conf is 256 lines at the time of writing, if you always have single line events you could switch SHOULDLINEMERGE=false on and never line merge (this entry would go in the props.conf)

Alternatively if you have very large events (>1000 lines) you could make a more complicated LINEBREAKER setting and change SHOULDLINEMERGE to false such as:

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}_\d

Of course the above would require testing, and it would only make sense if you have larger multi-line events (I've used examples like the above for events >1000 lines), doing this can remove pressure from the aggregation queue as the merging will move to the parsing queue...

Highlighted

Re: Why isn't the timestamp being recognized ?

Ultra Champion

Btw, this by itself did it ; -) with no line breaking

0 Karma