How to extract fields that are recursively present between two tags in an Event

[saikumar009]

Hi All,

I  am new to Splunk and i have been trying to extract field that are present in between two tags in an Event.

From the below event snippet i would like to extract loglevel information field ( i.e INFO / ERROR/WARN) and log_msg* fields. 

[BEGIN LOGGING AT 2021-05-18 21:35:20,002] , [END LOGGING] are start and end tags.

Please help with the regular expression.

[BEGIN LOGGING AT 2021-05-18 21:35:20,002]
2011-05-19 02:31:05,371 INFO - log_msg1.
2011-05-19 03:35:09,897 WARN - log_msg2.
2011-05-19 03:35:13,756 ERROR - log_msg3.
2011-05-19 03:35:27,272 WARN - log_msg4.

2011-05-19 03:35:31,069 INFO - log_msg5.

[END LOGGING]

[acharlieh]

Without additional context it seems like you should adjust the ingestion so that instead of one event, this winds up as multiple events, as opposed to one compound event. 

How you do that is a function of how you are ingesting the data... Often times this is adjusting props.conf for your sourcetype at the point of parsing the data into events, however if you're sending in via HTTPEventCollector, the adjustment could be on the side that's sending events (lots of caveats of course)

These docs are relevant: 
https://docs.splunk.com/Documentation/Splunk/8.2.0/Data/Configureeventlinebreaking
https://docs.splunk.com/Documentation/Splunk/8.2.0/Data/Modifyeventprocessing

With separate events, field extraction then becomes much easier. 

[kamlesh_vaghela]

@saikumar009 

Can you please try this?

YOUR_SEARCH
| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel

My Sample Search :

| makeresults 
| eval d="2011-05-19 02:31:05,371 INFO - log_msg1.|2011-05-19 03:35:09,897 WARN - log_msg2.|2011-05-19 03:35:13,756 ERROR - log_msg3.|2011-05-19 03:35:27,272 WARN - log_msg4.|2011-05-19 03:35:31,069 INFO - log_msg5.", d=split(d,"|") 
| mvexpand d
| rename d  as _raw
|rename comment as "Upto Now is sample data only" 
| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel

 Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.