topic Re: Why are delimited field extractions not working? in Splunk Search

Why are delimited field extractions not working?

richardAtOmni — Tue, 31 Jan 2017 22:12:43 GMT

Hello, we are inputting data via the HTTP Event collector. The "event" member has this format, which we are trying to split into fields with the pipe delimiter:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

As I work through the field extraction definition tool, the delimiter properly splits out the fields. I work through and I rename each field the way I want. Then, I save the field extraction. I get a message saying that this was successful. Then I click on the link "explore the fields that I just extracted" (I'm paraphrasing from memory), then it takes me to a search with a filter on the sourcetype that I just defined the field extraction for.

The problem is the search results do not show the new fields that I just defined. It only shows the first one. And as it's value it has the entire row as the value, as though none of the delimiters were recognized at all.

For example, on the row above, if my first field was named "start", then it would have a value of:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

What am I missing?

Thanks for any insight you can provide.

Re: Why are delimited field extractions not working?

mpreddy — Tue, 31 Jan 2017 23:54:16 GMT

Hi,

Use props and transforms in search time it will extract | seperated.
props.conf

[sourcetype]
REPORT-fields = pipefields

transforms.conf

[pipefields]
DELIMS = "|"
FIELDS = field1, field2, field3, field4, field5, field6, field7, field8, field9

Re: Why are delimited field extractions not working?

richardAtOmni — Thu, 02 Feb 2017 17:19:53 GMT

I checked these files, and the field extractor tool I used to define the delimited fields pretty much generated the same config that you suggest. However, it still doesn't work for some reason.

Re: Why are delimited field extractions not working?

richardAtOmni — Thu, 02 Feb 2017 17:24:04 GMT

The delimited fields worked as expected after we changes our input format to be just a straight string, instead of a nested JSON object.

So instead of this:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"}

We have this:
"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"

I'm not sure, but there seems to have been something about the nested JSON which prevented the parsing from working as expected?

The original question of why this didn't work isn't quite answered, but we're good for our use cases to proceed, so I'll mark it closed.

Thanks!

Re: Why are delimited field extractions not working?

jawaharas — Mon, 21 Jan 2019 05:12:45 GMT

Your suggestion helped me. Thank you. 🙂