topic Re: extracting fields from "event_message" "context: " portion of splunkd events on indexers? in Getting Data In

extracting fields from "event_message" "context: " portion of splunkd events on indexers?

mitag — Mon, 13 Jul 2020 20:47:07 GMT

in splunkd events on indexers such as this:

07-13-2020 11:42:03.337 -0700 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Mon Jul 13 11:42:02 2020). Context: source=/Library/Application Support/Symantec/Silo/NFM/LiveUpdate/Logs/lux.log|host=mac_mini04|symantec:silo:NFM=LiveUpdate:lux|233394

host = splunk_indexer_01 | source = /opt/splunk/var/log/splunk/splunkd.log | sourcetype = splunkd

... it does not look like fields in the "Context: " portion of the events are extracted:

Context: source=/Library/Application Support/Symantec/Silo/NFM/LiveUpdate/Logs/lux.log|host=bpa-mit-mini04|symantec:silo:NFM=LiveUpdate:lux|233394

Do I need to manually extract them via rex? If so - has anyone done this and could perhaps share a template rex command for this event type? If not, what's the best practice?

Thank you!

P.S. Something like this?

index=_internal sourcetype=splunkd "Context: " | rex field=event_message "Context\:\s+(?P<Context>source\=(?P<context_source>\S+?)?[\||$]host\=(?P<context_host>\S+?)(?:\|(?P<context_tail>.*))$)"

Re: extracting fields from "event_message" "context: " portion of splunkd events on indexers?

isoutamo — Mon, 13 Jul 2020 19:24:17 GMT

Basically this error means that splunk couldn’t recognize valid time stamp on your event within 128 character from beginning of event.
Can you send sample events and your inputs, props and possible transformations.conf files?

r. Ismo

Re: extracting fields from "event_message" "context: " portion of splunkd events on indexers?

mitag — Mon, 13 Jul 2020 20:44:45 GMT

Thanks @isoutamo! The question is about a field extraction for a specific sourcetype, not the root cause of the event. I have yet to dig into it - but looks like these "Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD" errors are happening in several places (hosts, sourcetypes) and will have to be dealt with separately and individually. For now, I just need a "best practice" on how to get those fields out of the "Context" portion of those errors. Thanks again.

Re: extracting fields from "event_message" "context: " portion of splunkd events on indexers?

isoutamo — Mon, 13 Jul 2020 22:11:31 GMT

Sorry that ;-( I should read the whole message and think little bit more before answer.

you could try the next one

...
| rex "Context:\s+(?<context>.*)"
| rex field=context max_match=0 "source=(?<context_source>[^\|]+)|host=(?<context_host>[^\|]+)"

Re: extracting fields from "event_message" "context: " portion of splunkd events on indexers?

mitag — Mon, 13 Jul 2020 22:32:00 GMT

Thank you! Much cleaner than mine.