Why are delimited field extractions not working?

[richardAtOmni]

Hello, we are inputting data via the HTTP Event collector. The "event" member has this format, which we are trying to split into fields with the pipe delimiter:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

As I work through the field extraction definition tool, the delimiter properly splits out the fields. I work through and I rename each field the way I want. Then, I save the field extraction. I get a message saying that this was successful. Then I click on the link "explore the fields that I just extracted" (I'm paraphrasing from memory), then it takes me to a search with a filter on the sourcetype that I just defined the field extraction for.

The problem is the search results do not show the new fields that I just defined. It only shows the first one. And as it's value it has the entire row as the value, as though none of the delimiters were recognized at all.

For example, on the row above, if my first field was named "start", then it would have a value of:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

What am I missing?

Thanks for any insight you can provide.

[richardAtOmni · Accepted solution]

The delimited fields worked as expected after we changes our input format to be just a straight string, instead of a nested JSON object.

So instead of this:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"}

We have this:
"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"

I'm not sure, but there seems to have been something about the nested JSON which prevented the parsing from working as expected?

The original question of why this didn't work isn't quite answered, but we're good for our use cases to proceed, so I'll mark it closed.

Thanks!

[mpreddy]

Hi,

Use props and transforms in search time it will extract | seperated.
props.conf

[sourcetype]
REPORT-fields = pipefields

transforms.conf

[pipefields]
DELIMS = "|"
FIELDS = field1, field2, field3, field4, field5, field6, field7, field8, field9

[jawaharas]

Your suggestion helped me. Thank you. 🙂

[richardAtOmni]

I checked these files, and the field extractor tool I used to define the delimited fields pretty much generated the same config that you suggest. However, it still doesn't work for some reason.