Splunk Search

Field Extraction

cauhe
Explorer

Hi Experts,

I'm having some difficulties to extract the correct information from a file that was add to splunk.

I tried to read/understand as much as I could but still struggling to correctly extract the information.

Here is a snip of my file:

call_type: "I" alert_id: "8626530 " data_center: "XYZ2 " memname: "QWERTPX " order_id: "1OOUZ" severity: "R" status: "Not_Noticed " send_time: "20210928070008" last_user: " " last_time: " " message: "ASDFGH STARTUP OF REGION QWERTPX" run_as: "USER01 " sub_application: "QWERT " application: "HOUSEKEEPING " job_name: "JOBASDF " host_id: " " alert_type: "R" closed_from_em: " " ticket_number: " " run_counter: " " notes: " "
call_type: "I" alert_id: "8626531 " data_center: "XYZ2 " memname: "QWERTZD " order_id: "1OOVH" severity: "R" status: "Not_Noticed " send_time: "20210928070009" last_user: " " last_time: " " message: "ASDFGH STARTUP OF REGION QWERTZD" run_as: "USER01 " sub_application: "QWERT " application: "HOUSEKEEPING " job_name: "JOBASDF " host_id: " " alert_type: "R" closed_from_em: " " ticket_number: " " run_counter: " " notes: " "
call_type: "I" alert_id: "8626533 " data_center: "XYZ2 " memname: "QWERTZU " order_id: "1OOVV" severity: "R" status: "Not_Noticed " send_time: "20210928070009" last_user: " " last_time: " " message: "ASDFGH STARTUP OF REGION QWERTZU" run_as: "USER01 " sub_application: "QWERT " application: "HOUSEKEEPING " job_name: "JOBASDF " host_id: " " alert_type: "R" closed_from_em: " " ticket_number: " " run_counter: " " notes: " "
call_type: "I" alert_id: "8626532 " data_center: "XYZ2 " memname: "QWERTZE " order_id: "1OOVJ" severity: "R" status: "Not_Noticed " send_time: "20210928070009" last_user: " " last_time: " " message: "ASDFGH STARTUP OF REGION QWERTZE" run_as: "USER01 " sub_application: "QWERT " application: "HOUSEKEEPING " job_name: "JOBASDF " host_id: " " alert_type: "R" closed_from_em: " " ticket_number: " " run_counter: " " notes: " "

What I need is have this 21 fields extracted properly, at moment I tried the delimiters but it doesn't work with :  

I believe I will have to write an regular expression (this is where I got stuck as I have no clue how...)

Basically what I need is the below fields extracted from the file so I could run dashbords, reports, alerts etc...

Field_1 - all_type: "I"
Field_2 - alert_id: "0000007 "
Field_3 - data_center: "XYZ2 "
Field_4 - memname: "ABCABC01 "
Field_5 - order_id: "1OO59"
Field_6 - severity: "R"
Field_7 - status: "Not_Noticed "
Field_8 - send_time: "20210923210008"
Field_9 - last_user: " "
Field_10 - last_time: " "
Field_11 - message: "MSG SHUTDOWN OF REGION ABCDEF"
Field_12 - run_as: "USER01 "
Field_13 - sub_application: "QWERT "
Field_14 - application: "HOUSEKEEPING "
Field_15 - job_name: "JOBASDF "
Field_16 - host_id: " "
Field_17 - alert_type: "R"
Field_18 - closed_from_em: " "
Field_19 - ticket_number: " "
Field_20 - run_counter: " "
Field_21 - notes: " "

Really appreciate any help to achieve this 

Thank you !! 

 

Labels (2)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

Maybe indeed those spaces confused me a bit and it looked as if the field names were in quotes.

Anyway, if the field set and sequence is constant, it's relatively easy to write a regex to parse such string.

call_type:\s"(?<call_type>.*)\s?"\salert_id:\s"(?<alert_id>.*)\s?"\sdata_center:\s(?<data_center>.*\s?")...

and so on.

The "\s?" at the end of a capturing group should account for the possible space at the end of your field value. I assumed that you would like to capture the value without that space.

It could be more complicated if the set and/or order of fields was not constant.

EDIT: the last capture group should have ended after .* - the \s?" should have been outside the parentheses

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

It's interesting because some of the field names inyour example are in quotes and some aren't. Is it really so? Or is it a mistake in preparing example?

cauhe
Explorer

Hi Mate, 

The fields have a pattern like for example

The first field call_type --> call_type: "I"

The second field alert_id --> alert_id: "8626530 "

if you look closely you will find that all field names will finish with : and them the values will be inside the " " 

Funny thing is some values inside " " have a blank space at the end of the value like this "DATA " 

call_type: "I" alert_id: "8626530 " data_center: "XYZ2 " memname: "QWERTPX " order_id: "1OOUZ" severity: "R" status: "Not_Noticed " send_time: "20210928070008" last_user: " " last_time: " " message: "ASDFGH STARTUP OF REGION QWERTPX" run_as: "USER01 " sub_application: "QWERT " application: "HOUSEKEEPING " job_name: "JOBASDF " host_id: " " alert_type: "R" closed_from_em: " " ticket_number: " " run_counter: " " notes: " "

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Maybe indeed those spaces confused me a bit and it looked as if the field names were in quotes.

Anyway, if the field set and sequence is constant, it's relatively easy to write a regex to parse such string.

call_type:\s"(?<call_type>.*)\s?"\salert_id:\s"(?<alert_id>.*)\s?"\sdata_center:\s(?<data_center>.*\s?")...

and so on.

The "\s?" at the end of a capturing group should account for the possible space at the end of your field value. I assumed that you would like to capture the value without that space.

It could be more complicated if the set and/or order of fields was not constant.

EDIT: the last capture group should have ended after .* - the \s?" should have been outside the parentheses

0 Karma

cauhe
Explorer

Mate,

Have no words to thank you it works (: 

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...