Splunk Search

Regex/timestamp question

Branden
Builder

Hello!

Given an event like this:

 PSMONITORSRV.32876010 (0) [09/15/12 09:16:20](3) PSJNI: Created a Java VM instance

I have two questions:

  1. What is the regex for extracting just the message? The message in this case is "PSJNI: Created a Java VM instance". I can sorta get it to work, but it keeps capture the timestamp in the message as well.
  2. How do I tell Splunk to index the event based on the time in the event's timestamp, and not on the time it was actually indexed? 99.9% of the time the two are the same, but once in a while there's a difference, and I'd rather Splunk index based on what the event says.

Thanks!

Tags (2)
1 Solution

kristian_kolb
Ultra Champion
  1. Just given one event, it's not easy to give a regex that will always work. Try rex to find a regex that will match the correct part of the event. The following might work for you.

    ... | rex "^(?:\S+\s+){3}(?<msg>.*)$"

  2. If Splunk does not capture the timestamp in the event correctly, you may have to edit the props.conf on the splunk server where the PARSING takes place. Normally that would be the Indexer, but if you have Heavy Forwarders, that's where you would make the configuration.

props.conf

[your_sourcetype]
TIME_PREFIX = \[
TIME_FORMAT = %m/%d/%y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 40

Hope this helps,

Kristian

View solution in original post

kristian_kolb
Ultra Champion
  1. Just given one event, it's not easy to give a regex that will always work. Try rex to find a regex that will match the correct part of the event. The following might work for you.

    ... | rex "^(?:\S+\s+){3}(?<msg>.*)$"

  2. If Splunk does not capture the timestamp in the event correctly, you may have to edit the props.conf on the splunk server where the PARSING takes place. Normally that would be the Indexer, but if you have Heavy Forwarders, that's where you would make the configuration.

props.conf

[your_sourcetype]
TIME_PREFIX = \[
TIME_FORMAT = %m/%d/%y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 40

Hope this helps,

Kristian

Branden
Builder

Thank you as well for your response. I plan to give these a try later today. Thanks again!

0 Karma

kristian_kolb
Ultra Champion

🙂 indeed

MHibbin
Influencer

🙂 we should stop meeting like this

MHibbin
Influencer

Regex (assuming general format stays the same): "\[\d+/\d+/\d+\s+\d+:\d+:\d+\]\(\d+\)(?P<yourField>.*)$"

You can test regular expressions on you data using the following web-based tool: http://gskinner.com/RegExr/

For timestamp recognition you should look at the following docs: http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

So for example for you data sourcetype you could configure something like:

[yourSourcetype]
MAX_TIMESTAMP_LOOKAHEAD = <integer>
TIME_PREFIX = <regular expression>
TIME_FORMAT = <strptime-style format>

External resource for strptime format: http://linux.die.net/man/3/strptime

Branden
Builder

Thank you for your response. I will give this a try and report back!

0 Karma

kristian_kolb
Ultra Champion

Oh dear..... not again...

Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...