Getting Data In

Why isn't the timestamp being recognized ?

Ultra Champion

We have a case that looks like this -

alt text

So, the events are not broken by the timestamp. Is it because of the underscore between the date and time?

Tags (1)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

Splunk has a number of built in time formats, I would recommend you manually parse your timestamps to improve indexing performance and ensure that you have the timestamps consistently recognised.

In this case in your props.conf

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N

Also refer to configure timestamp recognition. I agree with skoelpin it's probably the _ throwing the default timestamp parsing off.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Splunk has a number of built in time formats, I would recommend you manually parse your timestamps to improve indexing performance and ensure that you have the timestamps consistently recognised.

In this case in your props.conf

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N

Also refer to configure timestamp recognition. I agree with skoelpin it's probably the _ throwing the default timestamp parsing off.

View solution in original post

0 Karma

Ultra Champion

Btw, this by itself did it ; -) with no line breaking

0 Karma

Ultra Champion

Gorgeous - thank you @garethatiagg

SplunkTrust
SplunkTrust

Remember that this is a start and you should apply line breaking to your props.conf aswell.. This will take some load off your indexers

SplunkTrust
SplunkTrust

The documentation also covers line breaking

This is a slightly more advanced topic but potentially worth discussing here, the default for the MAX_EVENTS in props.conf is 256 lines at the time of writing, if you always have single line events you could switch SHOULD_LINEMERGE=false on and never line merge (this entry would go in the props.conf)

Alternatively if you have very large events (>1000 lines) you could make a more complicated LINE_BREAKER setting and change SHOULD_LINEMERGE to false such as:

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}_\d

Of course the above would require testing, and it would only make sense if you have larger multi-line events (I've used examples like the above for events >1000 lines), doing this can remove pressure from the aggregation queue as the merging will move to the parsing queue...

SplunkTrust
SplunkTrust

Most likely the underscores are throwing it off. What's your props.conf look like?

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!