Splunk Search

Is it possible to extract the date from two different spots in the same sourcetype?

tmeader
Contributor

Haven't been able to find any other questions anywhere asking about this, so I was wondering if anyone has tried it before? Basically, we have a firewall log that we're ingesting, which, for 95% of the events indexes on the field of "start_time=" (this is forced at index time in props.conf). This works great, except for the other 5% that aren't actually traffic messages, but rather adminstrivia type stuff generated by the firewall. These messages do however have a time-stamp at the end of the message included in parenthesis. Since this 5% of the messages is generating "Unable to extract time-stamp" error messages on the indexers whenever they're encountered, I'd like to be able to have this sourcetype ("netscreen") be able to extract the date from the "start_time" field in almost all cases, but, when that doesn't exist, key in on that time stamp at the end of the message. Is something like this possible? We already use very specific transforms for the netscreen sourcetype (called in props.conf to do the extractions)... would it be possible to include the multiple time-stamp recognition in there somehow? Or are we just out of luck (and will have to accept the error messages on the indexers; letting Splunk index on the time the event was received)?

Thanks in advance, and sorry for any incomprehensibility of this... it's late.

0 Karma

southeringtonp
Motivator

If you're using TIME_PREFIX, it's just a regex, so you should be able to build in an "or" condition, e.g.:

TIME_PREFIX=(prefix1)|(prefix2)

and make sure that MAX_TIMESTAMP_LOOKAHEAD is set high enough to get through the longer longer messages.

Alternately, if the netscreen events are coming in via syslog, you could have the syslog server write them to two separate files (based on pattern match, syslog facility, or whatever works), and then create two distinct sourcetypes within Splunk.

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...