Splunk Search

Is there a way to generalize these in Splunk?

Nicholas_Key
Splunk Employee
Splunk Employee

I would like to know if there is a way to generalize the following EXTRACT regexes in my props.conf? The configuration looks like the following:

[activityLog]
LINE_BREAKER = [-]{8,}([\r\n]+)
SHOULD_LINEMERGE = false
EXTRACT-ComponentId = ComponentId:\t(?.*)
EXTRACT-ProcessId = ProcessId:\t(?.*)
EXTRACT-ThreadId = ThreadId:\t(?.*)
EXTRACT-ThreadName = ThreadName:\t(?.*)
EXTRACT-SourceId = SourceId:\t(?.*)
EXTRACT-ClassName = ClassName:\t(?.*)
EXTRACT-MethodName = MethodName:\t(?.*)
EXTRACT-Manufacturer = Manufacturer:\t(?.*)
EXTRACT-Product = Product:\t(?.*)
EXTRACT-Version = Version:\t(?.*)
EXTRACT-ServerName = ServerName:\t(?.*)
EXTRACT-TimeStamp = TimeStamp:\t(?.*)
EXTRACT-UnitOfWork = UnitOfWork:\t(?.*)
EXTRACT-Severity = Severity:\t(?.*)
EXTRACT-Category = Category:\t(?.*)
EXTRACT-PrimaryMessage = PrimaryMessage:\t(?.*)
EXTRACT-ExtendedMessage = ExtendedMessage:\t(?.*)

And the content have a similar pattern

ComponentId:     Application Server
ProcessId:   5316
ThreadId:    00000000
ThreadName:  P=901265:O=0:CT
SourceId:    com.ibm.ws.runtime.WsServerImpl
ClassName:   
MethodName:  
Manufacturer:    IBM
Product:     WebSphere
Version:     Platform 7.0.0.7 [BASE 7.0.0.7 cf070942.55]
ServerName:  sfeserv36Node01Cell\sfeserv36Node01\server1
TimeStamp:   2010-04-27 09:15:57.671000000
UnitOfWork:  
Severity:    3
Category:    AUDIT
PrimaryMessage:  WSVR0001I: Server server1 open for e-business
ExtendedMessage: 
Tags (2)
0 Karma
1 Solution

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

View solution in original post

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

Nicholas_Key
Splunk Employee
Splunk Employee

Thank you for your helpful hint, Simeon! It works with DELIMS = "\n", ":\t" because first, the lines are separated by a "\n" and second, the key-value pairs are separated by a ":\t".

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...