Splunk Search

Is it possible to extract the date from two different spots in the same sourcetype?

tmeader
Contributor

Haven't been able to find any other questions anywhere asking about this, so I was wondering if anyone has tried it before? Basically, we have a firewall log that we're ingesting, which, for 95% of the events indexes on the field of "start_time=" (this is forced at index time in props.conf). This works great, except for the other 5% that aren't actually traffic messages, but rather adminstrivia type stuff generated by the firewall. These messages do however have a time-stamp at the end of the message included in parenthesis. Since this 5% of the messages is generating "Unable to extract time-stamp" error messages on the indexers whenever they're encountered, I'd like to be able to have this sourcetype ("netscreen") be able to extract the date from the "start_time" field in almost all cases, but, when that doesn't exist, key in on that time stamp at the end of the message. Is something like this possible? We already use very specific transforms for the netscreen sourcetype (called in props.conf to do the extractions)... would it be possible to include the multiple time-stamp recognition in there somehow? Or are we just out of luck (and will have to accept the error messages on the indexers; letting Splunk index on the time the event was received)?

Thanks in advance, and sorry for any incomprehensibility of this... it's late.

0 Karma

southeringtonp
Motivator

If you're using TIME_PREFIX, it's just a regex, so you should be able to build in an "or" condition, e.g.:

TIME_PREFIX=(prefix1)|(prefix2)

and make sure that MAX_TIMESTAMP_LOOKAHEAD is set high enough to get through the longer longer messages.

Alternately, if the netscreen events are coming in via syslog, you could have the syslog server write them to two separate files (based on pattern match, syslog facility, or whatever works), and then create two distinct sourcetypes within Splunk.

0 Karma
Get Updates on the Splunk Community!

Wrapping Up Cybersecurity Awareness Month

October might be wrapping up, but for Splunk Education, cybersecurity awareness never goes out of season. ...

🌟 From Audit Chaos to Clarity: Welcoming Audit Trail v2

🗣 You Spoke, We Listened  Audit Trail v2 wasn’t written in isolation—it was shaped by your voices.  In ...

What's New in Splunk Observability - October 2025

What’s New?    We’re excited to announce the latest enhancements to Splunk Observability Cloud and share ...