Splunk Search

How to split event lines into multiple fields using regex?

vamseepotluri
New Member

HI,

I am trying to write a regex to split these event lines into multiple fields.
Can some one please help me how to achieve this for these events mentioned below?

Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceDmgrNode on   - Deployment manager
Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceNode01 on  
Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceNode02 on 

Tue Mar 29 15:20:16 EDT 2016    DEPLOYMENT_MANAGER dmgr            RUNNING
Tue Mar 29 15:20:16 EDT 2016    APPLICATION_SERVER MDMServiceAppServer01 RUNNING
Tue Mar 29 15:20:16 EDT 2016    NODE_AGENT         nodeagent       RUNNING
Tue Mar 29 15:20:16 EDT 2016    APPLICATION_SERVER MDMServiceAppServer02 RUNNING
Tue Mar 29 15:20:16 EDT 2016    NODE_AGENT         nodeagent       RUNNING
Tue Mar 29 15:20:16 EDT 2016    WEB_SERVER         MDMServiceHTTPServer01 RUNNING
Tue Mar 29 15:20:16 EDT 2016    WEB_SERVER         MDMServiceHTTPServer02 RUNNING

Thanks, Much Appreciated.

Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is the intended purpose of the first rex command?
The second rex command doesn't extract any fields. Try this

... | rex field=line "(?<Eventtime>\w+\s\w+\s\d+\s\d+:\d+:\d+\s\w+\s\d+)\s+(?<ServerType>[\w_]+)\s+(?<ServerName>\w+)\s+(?<Status>\w+)" | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma

somesoni2
Revered Legend

Based on your sample search, you've multiline event. I would suggest to configure line breaking to split those multiple events as separate events as I can see they all have timestamp and good candidates for line breaking. It will also reduce load on indexers.

Meanwhile, Try this (check the line 1 and update according your current search)

index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line
  | rex field=line "^(?<Eventtime>(\S+\s+){6})(?<ServerType>\S+)\s+(?<ServerName>\S+)\s+(?<Status>\S+)"

If above doesn't work try this as well

index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line | rex mode=sed field=line "s/^\s+(.+)/\1/"   | rex field=line "^(?<Eventtime>(\S+\s+){6})(?<ServerType>\S+)\s+(?<ServerName>\S+)\s+(?<Status>\S+)"

yuanliu
SplunkTrust
SplunkTrust

Agree that multiline events are your enemy, especially when every line has a valid timestamp.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the fields?

---
If this reply helps you, Karma would be appreciated.
0 Karma

vamseepotluri
New Member

Fields should be something like this.

Eventtime ServerType ServerName Status
Tue Mar 29 15:20:16 EDT 2016 DEPLOYMENT_MANAGER dmgr RUNNING

0 Karma

yuanliu
SplunkTrust
SplunkTrust

You want to post what have you been trying, and explain what particular expectation is not met by your method.

0 Karma

vamseepotluri
New Member
index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?\V+)$" max_match=10000 | table line | mvexpand line | rex field=line "^(?%w\s%b\s%d\s%H:%M:%S\s%Z\s%Y\d+)\s+(?[a-zA-Z]\d+)\s+(?[a-z0-9#]\d+)\s+(?[a-zA-Z]\d+)$" | table eventtime, SERVERTYPE, SERVERNAME, STATUS
0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...