Splunk Search

Is it possible to extract the date from two different spots in the same sourcetype?

Contributor

Haven't been able to find any other questions anywhere asking about this, so I was wondering if anyone has tried it before? Basically, we have a firewall log that we're ingesting, which, for 95% of the events indexes on the field of "start_time=" (this is forced at index time in props.conf). This works great, except for the other 5% that aren't actually traffic messages, but rather adminstrivia type stuff generated by the firewall. These messages do however have a time-stamp at the end of the message included in parenthesis. Since this 5% of the messages is generating "Unable to extract time-stamp" error messages on the indexers whenever they're encountered, I'd like to be able to have this sourcetype ("netscreen") be able to extract the date from the "start_time" field in almost all cases, but, when that doesn't exist, key in on that time stamp at the end of the message. Is something like this possible? We already use very specific transforms for the netscreen sourcetype (called in props.conf to do the extractions)... would it be possible to include the multiple time-stamp recognition in there somehow? Or are we just out of luck (and will have to accept the error messages on the indexers; letting Splunk index on the time the event was received)?

Thanks in advance, and sorry for any incomprehensibility of this... it's late.

0 Karma

Motivator

If you're using TIME_PREFIX, it's just a regex, so you should be able to build in an "or" condition, e.g.:

TIME_PREFIX=(prefix1)|(prefix2)

and make sure that MAX_TIMESTAMP_LOOKAHEAD is set high enough to get through the longer longer messages.

Alternately, if the netscreen events are coming in via syslog, you could have the syslog server write them to two separate files (based on pattern match, syslog facility, or whatever works), and then create two distinct sourcetypes within Splunk.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!