How to find the origin and regular expression of an extracted field I use in Splunk?

[skoelpin]

We have a field called Response_Size which we cannot find. I looked in the Settings>Fields>Field Extractions and selected ALL for App context and Any for Owner and still cannot find the field. I also looked in Calculated fields with no luck. The field is active in Splunk and I'm able to use the field, but cannot find it's origin and the regular expression that extracts it. Any ideas how I can find it?

[skoelpin · Accepted solution]

Thanks for this. I looked in splunk/etc/apps/endeca/localand found the field in transforms.conf..

After opening transforms.conf I saw this stanza which contains the field i'm looking for (Response_Size)

[requestLogFields]
FIELDS = "Timestamp", "Client_IP", "Agraph_Transaction_ID", "HTTP_Exchange_ID", "Response_Size", "Total_Request_Time", "Total_Processing_Time", "HTTP_Status_Code"
DELIMS = " "

I then opened the props.conf file and saw

REPORT-requestLogFields = requestLogFields

I don't see an inputs.conf in this directory where the regex is defined. I'm also using the Endeca app so I suspect the regex was defined by the guy who made the app.. Any idea where I can find it? We're getting close!

[somesoni2]

The field extraction has nothing to do with inputs.conf. Based on your transforms.conf entry, there is no regex used, instead space is used a delimiter to extract fields. Why you do you need to regex for this field (based on your transforms.conf, your can try this ^(\S+\s){4}(?<Response_Size>\S+)for the field)?

[skoelpin]

We recently upgraded our ATG framework and after upgrading, our Response_Size is 10 times smaller according to Splunk which is not believable. I want to find the Response_Size field so I can see how it's extracted to find the root cause and fix it. So do you think it would be a good idea to extract the field rather than delim it by a space?

If I did this then I guess it would be a dead give away that deliming it by a space was the issue

[somesoni2]

Well, it will depend on if the log format has changed after ATG framework upgrade. If this space delimited field extraction was working/correct earlier and if the log format has not changed, then the value should be correct only.

[richgalloway]

It could be extracted automatically. What is the sourcetype? Can you share a sample event?

[deepthi5]

Hello skoelpin

You can see the extracted fields in the app folder where ever the extraction is done for example app\search\local\props.conf file and inputs.conf u can find your extracted field and its regular expression

Example:

[XXXXXXXXX (Sourcetype)]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = true
EXTRACT-server = ^(?P\d+).*\s+msg\s+execution
EXTRACT-Msg_Time = ^[^\]\n]*\]\s+msg+\s+execution\s+time+\s+=\s+(?P.+)