Splunk Search

Is there a way to generalize these in Splunk?

Nicholas_Key
Splunk Employee
Splunk Employee

I would like to know if there is a way to generalize the following EXTRACT regexes in my props.conf? The configuration looks like the following:

[activityLog]
LINE_BREAKER = [-]{8,}([\r\n]+)
SHOULD_LINEMERGE = false
EXTRACT-ComponentId = ComponentId:\t(?.*)
EXTRACT-ProcessId = ProcessId:\t(?.*)
EXTRACT-ThreadId = ThreadId:\t(?.*)
EXTRACT-ThreadName = ThreadName:\t(?.*)
EXTRACT-SourceId = SourceId:\t(?.*)
EXTRACT-ClassName = ClassName:\t(?.*)
EXTRACT-MethodName = MethodName:\t(?.*)
EXTRACT-Manufacturer = Manufacturer:\t(?.*)
EXTRACT-Product = Product:\t(?.*)
EXTRACT-Version = Version:\t(?.*)
EXTRACT-ServerName = ServerName:\t(?.*)
EXTRACT-TimeStamp = TimeStamp:\t(?.*)
EXTRACT-UnitOfWork = UnitOfWork:\t(?.*)
EXTRACT-Severity = Severity:\t(?.*)
EXTRACT-Category = Category:\t(?.*)
EXTRACT-PrimaryMessage = PrimaryMessage:\t(?.*)
EXTRACT-ExtendedMessage = ExtendedMessage:\t(?.*)

And the content have a similar pattern

ComponentId:     Application Server
ProcessId:   5316
ThreadId:    00000000
ThreadName:  P=901265:O=0:CT
SourceId:    com.ibm.ws.runtime.WsServerImpl
ClassName:   
MethodName:  
Manufacturer:    IBM
Product:     WebSphere
Version:     Platform 7.0.0.7 [BASE 7.0.0.7 cf070942.55]
ServerName:  sfeserv36Node01Cell\sfeserv36Node01\server1
TimeStamp:   2010-04-27 09:15:57.671000000
UnitOfWork:  
Severity:    3
Category:    AUDIT
PrimaryMessage:  WSVR0001I: Server server1 open for e-business
ExtendedMessage: 
Tags (2)
0 Karma
1 Solution

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

View solution in original post

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

Nicholas_Key
Splunk Employee
Splunk Employee

Thank you for your helpful hint, Simeon! It works with DELIMS = "\n", ":\t" because first, the lines are separated by a "\n" and second, the key-value pairs are separated by a ":\t".

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...