topic Re: How to split event lines into multiple fields using regex? in Splunk Search

How to split event lines into multiple fields using regex?

vamseepotluri — Tue, 29 Mar 2016 19:25:22 GMT

HI,

I am trying to write a regex to split these event lines into multiple fields.
Can some one please help me how to achieve this for these events mentioned below?

Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceDmgrNode on   - Deployment manager
Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceNode01 on  
Tue Mar 29 15:20:16 EDT 2016 NODE MDMServiceNode02 on 

Tue Mar 29 15:20:16 EDT 2016    DEPLOYMENT_MANAGER dmgr            RUNNING
Tue Mar 29 15:20:16 EDT 2016    APPLICATION_SERVER MDMServiceAppServer01 RUNNING
Tue Mar 29 15:20:16 EDT 2016    NODE_AGENT         nodeagent       RUNNING
Tue Mar 29 15:20:16 EDT 2016    APPLICATION_SERVER MDMServiceAppServer02 RUNNING
Tue Mar 29 15:20:16 EDT 2016    NODE_AGENT         nodeagent       RUNNING
Tue Mar 29 15:20:16 EDT 2016    WEB_SERVER         MDMServiceHTTPServer01 RUNNING
Tue Mar 29 15:20:16 EDT 2016    WEB_SERVER         MDMServiceHTTPServer02 RUNNING

Thanks, Much Appreciated.

Re: How to split event lines into multiple fields using regex?

richgalloway — Tue, 29 Mar 2016 19:57:34 GMT

What are the fields?

Re: How to split event lines into multiple fields using regex?

vamseepotluri — Tue, 29 Mar 2016 20:51:46 GMT

Fields should be something like this.

Eventtime ServerType ServerName Status
Tue Mar 29 15:20:16 EDT 2016 DEPLOYMENT_MANAGER dmgr RUNNING

Re: How to split event lines into multiple fields using regex?

yuanliu — Tue, 29 Mar 2016 20:55:16 GMT

You want to post what have you been trying, and explain what particular expectation is not met by your method.

Re: How to split event lines into multiple fields using regex?

vamseepotluri — Tue, 29 Mar 2016 21:00:43 GMT

index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?\V+)$" max_match=10000 | table line | mvexpand line | rex field=line "^(?%w\s%b\s%d\s%H:%M:%S\s%Z\s%Y\d+)\s+(?[a-zA-Z]\d+)\s+(?[a-z0-9#]\d+)\s+(?[a-zA-Z]\d+)$" | table eventtime, SERVERTYPE, SERVERNAME, STATUS

Re: How to split event lines into multiple fields using regex?

somesoni2 — Tue, 29 Mar 2016 22:45:27 GMT

Based on your sample search, you've multiline event. I would suggest to configure line breaking to split those multiple events as separate events as I can see they all have timestamp and good candidates for line breaking. It will also reduce load on indexers.

Meanwhile, Try this (check the line 1 and update according your current search)

index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line
  | rex field=line "^(?<Eventtime>(\S+\s+){6})(?<ServerType>\S+)\s+(?<ServerName>\S+)\s+(?<Status>\S+)"

If above doesn't work try this as well

index="myindex" sourcetype="mysourcetype" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line | rex mode=sed field=line "s/^\s+(.+)/\1/"   | rex field=line "^(?<Eventtime>(\S+\s+){6})(?<ServerType>\S+)\s+(?<ServerName>\S+)\s+(?<Status>\S+)"

Re: How to split event lines into multiple fields using regex?

richgalloway — Tue, 29 Mar 2016 22:47:51 GMT

What is the intended purpose of the first rex command?
The second rex command doesn't extract any fields. Try this

... | rex field=line "(?<Eventtime>\w+\s\w+\s\d+\s\d+:\d+:\d+\s\w+\s\d+)\s+(?<ServerType>[\w_]+)\s+(?<ServerName>\w+)\s+(?<Status>\w+)" | ...

Re: How to split event lines into multiple fields using regex?

yuanliu — Tue, 29 Mar 2016 22:54:00 GMT

Agree that multiline events are your enemy, especially when every line has a valid timestamp.