topic Re: Regex/timestamp question in Splunk Search

Regex/timestamp question

Branden — Tue, 18 Sep 2012 14:25:38 GMT

Hello!

Given an event like this:

 PSMONITORSRV.32876010 (0) [09/15/12 09:16:20](3) PSJNI: Created a Java VM instance

I have two questions:

What is the regex for extracting just the message? The message in this case is "PSJNI: Created a Java VM instance". I can sorta get it to work, but it keeps capture the timestamp in the message as well.
How do I tell Splunk to index the event based on the time in the event's timestamp, and not on the time it was actually indexed? 99.9% of the time the two are the same, but once in a while there's a difference, and I'd rather Splunk index based on what the event says.

Thanks!

MHibbin — Tue, 18 Sep 2012 14:33:07 GMT

Regex (assuming general format stays the same): "\[\d+/\d+/\d+\s+\d+:\d+:\d+\]$\d+$(?P<yourField>.*)$"

You can test regular expressions on you data using the following web-based tool: http://gskinner.com/RegExr/

So for example for you data sourcetype you could configure something like:

[yourSourcetype]
MAX_TIMESTAMP_LOOKAHEAD = <integer>
TIME_PREFIX = <regular expression>
TIME_FORMAT = <strptime-style format>

External resource for strptime format: http://linux.die.net/man/3/strptime

kristian_kolb — Tue, 18 Sep 2012 14:44:03 GMT

Just given one event, it's not easy to give a regex that will always work. Try rex to find a regex that will match the correct part of the event. The following might work for you.

... | rex "^(?:\S+\s+){3}(?<msg>.*)$"
If Splunk does not capture the timestamp in the event correctly, you may have to edit the props.conf on the splunk server where the PARSING takes place. Normally that would be the Indexer, but if you have Heavy Forwarders, that's where you would make the configuration.

props.conf

[your_sourcetype]
TIME_PREFIX = \[
TIME_FORMAT = %m/%d/%y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 40

Hope this helps,

Kristian

kristian_kolb — Tue, 18 Sep 2012 14:44:33 GMT

Oh dear..... not again...

MHibbin — Tue, 18 Sep 2012 14:51:10 GMT

🙂 we should stop meeting like this

kristian_kolb — Tue, 18 Sep 2012 14:58:18 GMT

🙂 indeed

Branden — Tue, 18 Sep 2012 16:25:03 GMT

Thank you for your response. I will give this a try and report back!

Branden — Tue, 18 Sep 2012 16:25:46 GMT

Thank you as well for your response. I plan to give these a try later today. Thanks again!