Building for the Splunk Platform

How to extract fields that are recursively present between two tags in an Event


Hi All,


I  am new to Splunk and i have been trying to extract field that are present in between two tags in an Event.

From the below event snippet i would like to extract loglevel information field ( i.e INFO / ERROR/WARN) and log_msg* fields. 

[BEGIN LOGGING AT 2021-05-18 21:35:20,002] , [END LOGGING] are start and end tags.

Please help with the regular expression.

[BEGIN LOGGING AT 2021-05-18 21:35:20,002]
2011-05-19 02:31:05,371 INFO - log_msg1.
2011-05-19 03:35:09,897 WARN - log_msg2.
2011-05-19 03:35:13,756 ERROR - log_msg3.
2011-05-19 03:35:27,272 WARN - log_msg4.

2011-05-19 03:35:31,069 INFO - log_msg5.


Labels (1)
Tags (1)
0 Karma


Without additional context it seems like you should adjust the ingestion so that instead of one event, this winds up as multiple events, as opposed to one compound event. 

How you do that is a function of how you are ingesting the data... Often times this is adjusting props.conf for your sourcetype at the point of parsing the data into events, however if you're sending in via HTTPEventCollector, the adjustment could be on the side that's sending events (lots of caveats of course)

These docs are relevant:

With separate events, field extraction then becomes much easier. 

0 Karma



Can you please try this?


| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel


My Sample Search :

| makeresults 
| eval d="2011-05-19 02:31:05,371 INFO - log_msg1.|2011-05-19 03:35:09,897 WARN - log_msg2.|2011-05-19 03:35:13,756 ERROR - log_msg3.|2011-05-19 03:35:27,272 WARN - log_msg4.|2011-05-19 03:35:31,069 INFO - log_msg5.", d=split(d,"|") 
| mvexpand d
| rename d  as _raw
|rename comment as "Upto Now is sample data only" 
| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel


If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Get Updates on the Splunk Community!

Security Highlights: September 2022 Newsletter

 September 2022 The Splunk App for Fraud Analytics (SFA) is now Splunk SupportedUse your existing Splunk ...

Platform Highlights | September 2022 Newsletter

 September 2022 What’s New in 9.0 and How to UpgradeGet a walk through of what is new Splunk Enterprise 9.0 ...

Observability Highlights | September 2022 Newsletter

 September 2022 Splunk Observability SuiteAccess to "Classic" SignalFx Interface Will be Removed on Sept 30, ...