Building for the Splunk Platform

How to extract fields that are recursively present between two tags in an Event


Hi All,


I  am new to Splunk and i have been trying to extract field that are present in between two tags in an Event.

From the below event snippet i would like to extract loglevel information field ( i.e INFO / ERROR/WARN) and log_msg* fields. 

[BEGIN LOGGING AT 2021-05-18 21:35:20,002] , [END LOGGING] are start and end tags.

Please help with the regular expression.

[BEGIN LOGGING AT 2021-05-18 21:35:20,002]
2011-05-19 02:31:05,371 INFO - log_msg1.
2011-05-19 03:35:09,897 WARN - log_msg2.
2011-05-19 03:35:13,756 ERROR - log_msg3.
2011-05-19 03:35:27,272 WARN - log_msg4.

2011-05-19 03:35:31,069 INFO - log_msg5.


Labels (1)
Tags (1)
0 Karma


Without additional context it seems like you should adjust the ingestion so that instead of one event, this winds up as multiple events, as opposed to one compound event. 

How you do that is a function of how you are ingesting the data... Often times this is adjusting props.conf for your sourcetype at the point of parsing the data into events, however if you're sending in via HTTPEventCollector, the adjustment could be on the side that's sending events (lots of caveats of course)

These docs are relevant:

With separate events, field extraction then becomes much easier. 

0 Karma



Can you please try this?


| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel


My Sample Search :

| makeresults 
| eval d="2011-05-19 02:31:05,371 INFO - log_msg1.|2011-05-19 03:35:09,897 WARN - log_msg2.|2011-05-19 03:35:13,756 ERROR - log_msg3.|2011-05-19 03:35:27,272 WARN - log_msg4.|2011-05-19 03:35:31,069 INFO - log_msg5.", d=split(d,"|") 
| mvexpand d
| rename d  as _raw
|rename comment as "Upto Now is sample data only" 
| rex field=_raw "^.{24}(?<loglevel>\w+)\s" 
| table loglevel


If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...