Splunk Search

Is it possible to extract the date from two different spots in the same sourcetype?

tmeader
Contributor

Haven't been able to find any other questions anywhere asking about this, so I was wondering if anyone has tried it before? Basically, we have a firewall log that we're ingesting, which, for 95% of the events indexes on the field of "start_time=" (this is forced at index time in props.conf). This works great, except for the other 5% that aren't actually traffic messages, but rather adminstrivia type stuff generated by the firewall. These messages do however have a time-stamp at the end of the message included in parenthesis. Since this 5% of the messages is generating "Unable to extract time-stamp" error messages on the indexers whenever they're encountered, I'd like to be able to have this sourcetype ("netscreen") be able to extract the date from the "start_time" field in almost all cases, but, when that doesn't exist, key in on that time stamp at the end of the message. Is something like this possible? We already use very specific transforms for the netscreen sourcetype (called in props.conf to do the extractions)... would it be possible to include the multiple time-stamp recognition in there somehow? Or are we just out of luck (and will have to accept the error messages on the indexers; letting Splunk index on the time the event was received)?

Thanks in advance, and sorry for any incomprehensibility of this... it's late.

0 Karma

southeringtonp
Motivator

If you're using TIME_PREFIX, it's just a regex, so you should be able to build in an "or" condition, e.g.:

TIME_PREFIX=(prefix1)|(prefix2)

and make sure that MAX_TIMESTAMP_LOOKAHEAD is set high enough to get through the longer longer messages.

Alternately, if the netscreen events are coming in via syslog, you could have the syslog server write them to two separate files (based on pattern match, syslog facility, or whatever works), and then create two distinct sourcetypes within Splunk.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...