topic Re: Problem extracting field from a field in Splunk Search

Problem extracting field from a field

chengka — Wed, 10 Dec 2014 20:10:12 GMT

Splunk 6.2
I used the Field Extractor app to extract a field from an previous field. The resulting extraction tested perfectly from with the app. Here is an excerpt from the sourcetype stanza of props.conf.

EXTRACT-ErrorMessageText = (?ims)^\w+.+AMQ\d\d\d\d:(?P.+)EXPLANATION
EXTRACT-ErrorMessageChannel = [cC]hannel[\s']+(?P[\w\d.]+) in ErrorMessageText

The problem is the field ErrorMessageChannel field never appears in the field list for an event. The 6.2 field extractor does not even list it as an Existing Field (ErrorMessageText is listed). The new field IS listed in the Settings Field Extractions.

Example event:

12/10/2014 03:54:45 PM - Process(5136.740878) User(mqm) Program(amqrmppa)
Host(hostname) Installation(Installation_2)
VRMF(7.5.0.3) QMgr(abcdef)
AMQ9999: Channel 'ABCD.EFG' to host 'xxx-xxx (10.1.1.1)' ended abnormally. EXPLANATION: The channel program running under process ID 5136 for channel 'ABCD.EFG' ended abnormally. The host name is 'xxx-xxx (10.1.1.1)'; in some cases the host name cannot be determined and so is shown as '????'. ACTION: Look at previous error messages for the channel program in the error logs to determine the cause of the failure. Note that this message can be excluded completely or suppressed by tuning the "ExcludeMessage" or "SuppressMessage" attributes under the "QMErrorLog" stanza in qm.ini. Further information can be found in the System Administration Guide.

The ErrorMessageText works properly and resolves to
"AMQ9999: Channel 'ABCD.EFG' to host 'xxx-xxx (10.1.1.1)'
ended abnormally."

The regex for ErrorMessageChannel works fine with the Field Extractor app as well. For this event the ErrorMessageText would be "ABCD.EFG"

What am I missing here?

Re: Problem extracting field from a field

jhupka — Wed, 10 Dec 2014 20:15:50 GMT

Can you provide a sample event? If it is listed in the Field Extractions settings that means Splunk knows it is defined...but if you're not seeing it within the Existing Field list on the left of your search results there might be some other issue in your RegEx that isn't working right so nothing ever gets extracted.

As a side note, it looks like maybe the less-than and greater-than portions of your field extractions got left out by the website - you might want to try editing it and markup your EXTRACT lines as blocks of code.

Re: Problem extracting field from a field

somesoni2 — Wed, 10 Dec 2014 20:38:30 GMT

Try adding "| extract reload=t" at the end of your search. Also, see if restarting your search head is an option and try doing that.

Re: Problem extracting field from a field

lguinn2 — Wed, 10 Dec 2014 22:11:08 GMT

I think this is your problem: when you specify the EXTRACT, Splunk uses the class that you provide to sequence the order of evaluation. This is particularly important when you consider that Splunk merges all the props.conf files into one run-time specification.

Since ErrorMessageChannel precedes ErrorMessageText, it is performed first and extracts nothing.

Do this instead:

 EXTRACT-e1 = (?ims)^\w+.+AMQ\d\d\d\d:(?P<ErrorMessageText>.+)EXPLANATION
 EXTRACT-e2 = [cC]hannel[\s']+(?P[\w\d.]+) in ErrorMessageTex

and I think it will work. BTW, the class name is not at all related to the field name.

Re: Problem extracting field from a field

chengka — Wed, 10 Dec 2014 22:18:35 GMT

I can see the field when I add "| extract reload=t". When I remove it, the field disappears. Also I adding ErrorMessageChannel=* or any of the Top 10 values returns nothing.

Re: Problem extracting field from a field

lguinn2 — Wed, 10 Dec 2014 22:41:32 GMT

If it was actually a reload problem, then you should only need to run this (by itself, not at the end of a search) one time

| reload extract=t

Can you show us the actual search, not just the props.conf entries?

Re: Problem extracting field from a field

chengka — Thu, 11 Dec 2014 15:21:22 GMT

Thank you, changing the class name did the trick.