topic Re: Field Extraction from event in Getting Data In

Field Extraction from event

CsinadyIoan — Mon, 28 Sep 2020 15:55:48 GMT

I have this Event : (A|0000349541000123126400200|A2C12312091142|A2C58063752|||01|004730343236303305002A0043|53427677011118600001|1|v1_09_111020||1202|02010||04|A2C53425053|CCR1130601630003|01 H02010 S1202 CXXX|11.11.02|28512312721R ---|0|0||||||)(B|2|tmarep1s|8|A2C58063751_BOT|Renlt_1.011:v3.06|110926171939||102|||T2801-8 limit error:|Classification Station|||||||||)(||||||+0.1824000E+01|+0.1093000E+03|||||||)

how do I split this on "|" in separate fields so that I may search in multiple events like this one? I can\t find solutions i'm new to splunk it's first time I worked with this program I I don't know to much. please tell me how to create a regex that I can apply to multiple events 🙂

Re: Field Extraction from event

sc0tt — Wed, 19 Feb 2014 14:01:47 GMT

I believe something like this should work

search... | eval first_group = mvindex(split(_raw,"|"),0) | eval second_group = mvindex(split(_raw,"|"),1)

Re: Field Extraction from event

CsinadyIoan — Wed, 19 Feb 2014 14:06:37 GMT

could you please be more explicit on this:) as I told you i'm new... and splunk commands are still a mistery for me. I just tried adding this on search and nothing happened.

Re: Field Extraction from event

sc0tt — Mon, 28 Sep 2020 15:55:50 GMT

In my example, I am using mvindex/split to create a new field called first_group and second_group based on the position of | in the raw data. You can then use those new fields in your search. For example, a search may look like

which would count the number of events grouped by the first and second group. I'm not exactly sure what you want to accomplish, but hopefully this gives you some idea how to use the search.

Re: Field Extraction from event

somesoni2 — Wed, 19 Feb 2014 15:03:12 GMT

Another option: If you know what all different fields are present in your logs (separated by "|"), you can use below to get them as fields.

index=yourindex sourcetype=yoursourcetype source=yoursource | rex "(?i).*?(?P<field1>\w+)" | rex "(?i)^(?:[^\|]*\|){1}(?P<field2>[^\|]+)" | rex "(?i)^(?:[^\|]*\|){2}(?P<field3>[^\|]+)"...

For other fields (based on the order in which they appear in your event), your can change the integer value in "{}" and update field name.
e.g.

field 2 ("0000349541000123126400200" and "2" in your example) 
| rex "(?i)^(?:[^\|]*\|){1}(?P<field2>[^\|]+)"

field 3 ("A2C12312091142" and "tmarep1s" in your example) 
| rex "(?i)^(?:[^\|]*\|){1}(?P<field3>[^\|]+)"

and so on

Re: Field Extraction from event

martin_mueller — Wed, 19 Feb 2014 15:09:29 GMT

You may also want to take a look at delimiter-based field extractions: http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles#Configuring_delimiter-based_field_extraction

For the config details, see the DELIMS and FIELDS keys in transforms.conf: http://docs.splunk.com/Documentation/Splunk/latest/Admin/transformsconf

Your config is going to look something like this:

props.conf

[your_sourcetype]
...
REPORT-pipedfields = pipedfields
...

transforms.conf

[pipedfields]
DELIMS = "|"
FIELDS = field1, field2, ...