Splunk Search

Is it possible to extract the date from two different spots in the same sourcetype?

tmeader
Contributor

Haven't been able to find any other questions anywhere asking about this, so I was wondering if anyone has tried it before? Basically, we have a firewall log that we're ingesting, which, for 95% of the events indexes on the field of "start_time=" (this is forced at index time in props.conf). This works great, except for the other 5% that aren't actually traffic messages, but rather adminstrivia type stuff generated by the firewall. These messages do however have a time-stamp at the end of the message included in parenthesis. Since this 5% of the messages is generating "Unable to extract time-stamp" error messages on the indexers whenever they're encountered, I'd like to be able to have this sourcetype ("netscreen") be able to extract the date from the "start_time" field in almost all cases, but, when that doesn't exist, key in on that time stamp at the end of the message. Is something like this possible? We already use very specific transforms for the netscreen sourcetype (called in props.conf to do the extractions)... would it be possible to include the multiple time-stamp recognition in there somehow? Or are we just out of luck (and will have to accept the error messages on the indexers; letting Splunk index on the time the event was received)?

Thanks in advance, and sorry for any incomprehensibility of this... it's late.

0 Karma

southeringtonp
Motivator

If you're using TIME_PREFIX, it's just a regex, so you should be able to build in an "or" condition, e.g.:

TIME_PREFIX=(prefix1)|(prefix2)

and make sure that MAX_TIMESTAMP_LOOKAHEAD is set high enough to get through the longer longer messages.

Alternately, if the netscreen events are coming in via syslog, you could have the syslog server write them to two separate files (based on pattern match, syslog facility, or whatever works), and then create two distinct sourcetypes within Splunk.

0 Karma
Get Updates on the Splunk Community!

OpenTelemetry for Legacy Apps? Yes, You Can!

This article is a follow-up to my previous article posted on the OpenTelemetry Blog, "Your Critical Legacy App ...

UCC Framework: Discover Developer Toolkit for Building Technology Add-ons

The Next-Gen Toolkit for Splunk Technology Add-on Development The Universal Configuration Console (UCC) ...

.conf25 Community Recap

Hello Splunkers, And just like that, .conf25 is in the books! What an incredible few days — full of learning, ...