Getting Data In

extracting fields from "event_message" "context: " portion of splunkd events on indexers?

mitag
Contributor

in splunkd events on indexers such as this:

 

07-13-2020 11:42:03.337 -0700 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Mon Jul 13 11:42:02 2020). Context: source=/Library/Application Support/Symantec/Silo/NFM/LiveUpdate/Logs/lux.log|host=mac_mini04|symantec:silo:NFM=LiveUpdate:lux|233394

 

 

host = splunk_indexer_01 | source = /opt/splunk/var/log/splunk/splunkd.log | sourcetype = splunkd

 

... it does not look like fields in the "Context: " portion of the events are extracted:

 

Context: source=/Library/Application Support/Symantec/Silo/NFM/LiveUpdate/Logs/lux.log|host=bpa-mit-mini04|symantec:silo:NFM=LiveUpdate:lux|233394

 

Do I need to manually extract them via rex? If so - has anyone done this and could perhaps share a template rex command for this event type? If not, what's the best practice?

Thank you!

P.S. Something like this?

 

index=_internal sourcetype=splunkd  "Context: "
| rex field=event_message "Context\:\s+(?P<Context>source\=(?P<context_source>\S+?)?[\||$]host\=(?P<context_host>\S+?)(?:\|(?P<context_tail>.*))$)"

 

Labels (1)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Sorry that ;-( I should read the whole message and think little bit more before answer.

you could try the next one

  

...
| rex "Context:\s+(?<context>.*)"
| rex field=context max_match=0 "source=(?<context_source>[^\|]+)|host=(?<context_host>[^\|]+)"

View solution in original post

isoutamo
SplunkTrust
SplunkTrust

Basically this error means that splunk couldn’t recognize valid time stamp on your event within 128 character from beginning of event. 
Can you send sample events and your inputs, props and possible transformations.conf files?

r. Ismo

0 Karma

mitag
Contributor

Thanks @isoutamo! The question is about a field extraction for a specific sourcetype, not the root cause of the event. I have yet to dig into it - but looks like these "Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD" errors are happening in several places (hosts, sourcetypes) and will have to be dealt with separately and individually. For now, I just need a "best practice" on how to get those fields out of the "Context" portion of those errors. Thanks again.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Sorry that ;-( I should read the whole message and think little bit more before answer.

you could try the next one

  

...
| rex "Context:\s+(?<context>.*)"
| rex field=context max_match=0 "source=(?<context_source>[^\|]+)|host=(?<context_host>[^\|]+)"

View solution in original post

mitag
Contributor

Thank you! Much cleaner than mine.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!