topic Re: How do I extract fields with Rex? in Splunk Search

How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 12:37:10 GMT

Hello
I have these events :

copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
going to copy total size of:  25.1 MiB
starting time:  Tue May 28 12:24:40 UTC 2019

I want to extract from the first line the string completed (it can be also failed).
I'd like to name it Status, and StatusTime for the the first date and time.
From second line, the size and name it TotalSize.
From third line I want the date, time, and to name it StartingTime.

How do I do this?

Thanks.

Re: How do I extract fields with Rex?

cmakepeace_nfcu — Tue, 28 May 2019 12:52:15 GMT

The below SPL should show the desired end status. This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

| makeresults 
| eval test="copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019~ 
going to copy total size of:  25.1 MiB~
starting time:  Tue May 28 12:24:40 UTC 2019" 
| fields - _time 
| rex field=test "(\w*\s){3}(?<Status>\w*)[^:]*:\s+(?<StatusTime>[^\n]*)\n[^:]*:\s+(?<TotalSize>[^\n]*)\n[^:]*:\s+(?<StartingTime>.*)"

This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

Re: How do I extract fields with Rex?

harsmarvania57 — Tue, 28 May 2019 13:00:30 GMT

Hi,

Please try below regex

<yourBaseSearch>
| rex field=_raw "^(?:[^\s]*[\s]){3}(?<Status>\w+)[^\:]*\:\s+(?<StatusTime>[^\v]*)\v[^\:]*\:\s+(?<TotalSize>[^\v]*)\v[^\:]*\:\s+(?<StartingTime>[^\v]*)"

Regex101 URL with sample data you have provided : https://regex101.com/r/a5Rbki/1

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 15:43:23 GMT

it returns empty results 😞

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 15:43:28 GMT

it returns empty results 😞

Re: How do I extract fields with Rex?

harsmarvania57 — Tue, 28 May 2019 15:47:31 GMT

Is it multiline event? If yes then you need to provide whole sample event, part of event will not work.

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 15:50:46 GMT

each event in separet line, just as i wrote at my questions
i think it should be rex for each line

 copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
 going to copy total size of:  25.1 MiB
 starting time:  Tue May 28 12:24:40 UTC 2019

Re: How do I extract fields with Rex?

DavidHourani — Tue, 28 May 2019 16:51:36 GMT

Hi @sarit_s,

If you have three different events, then use the following for each of the lines.
Line 1: ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"

Line 2: ...| rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"

Line 3: ...| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

You could also combine them as follows:

...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

Let me know how that works out for you.

Cheers,
David

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 18:40:34 GMT

Hi David,
this is how the results looks like :

Status StatusTime TotalSize StartingTime
Tue May 28 12:24:40 UTC 2019 Tue May 28 12:24:40 UTC 2019
Tue May 28 11:27:29 UTC 2019 Tue May 28 11:27:29 UTC 2019
completed 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019
completed 2019-05-27 2019-05-27 2019-05-27
total 25.1 MiB 25.1 MiB 25.1 MiB
total 5.1 MiB 5.1 MiB 5.1 MiB

Re: How do I extract fields with Rex?

DavidHourani — Tue, 28 May 2019 18:59:00 GMT

Hi @sarit_s, updated the answer, have a look and try again !

Re: How do I extract fields with Rex?

DavidHourani — Tue, 28 May 2019 19:04:48 GMT

This should do it :

 ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 19:08:10 GMT

Hi @DavidHourani
Thanks !!
it is almost perfect

just the rex for StatusTime should be separeted to 2 fields :

this is the result:
2019-05-27 Tue May 28 12:24:40 UTC 2019
and i need 2019-05-27 to be one part and the rest as second part

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 19:10:53 GMT

i did something like this:

| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)(?<StatusTime>.+)"

is that ok ?

Re: How do I extract fields with Rex?

DavidHourani — Tue, 28 May 2019 19:16:40 GMT

Looks good, you can add the space between the two fields as well to avoid having it in StatusTime :

 | rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)\s(?<StatusTime>.+)"

Re: How do I extract fields with Rex?

sarit_s — Tue, 28 May 2019 19:20:10 GMT

perfect
thanks !