Splunk Search

How do I extract fields with Rex?

Communicator

Hello
I have these events :

copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
going to copy total size of:  25.1 MiB
starting time:  Tue May 28 12:24:40 UTC 2019

I want to extract from the first line the string completed (it can be also failed).
I'd like to name it Status, and StatusTime for the the first date and time.
From second line, the size and name it TotalSize.
From third line I want the date, time, and to name it StartingTime.

How do I do this?

Thanks.

0 Karma
1 Solution

Super Champion

Hi @sarit_s,

If you have three different events, then use the following for each of the lines.
Line 1: ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"

Line 2: ...| rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"

Line 3: ...| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

You could also combine them as follows:

...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

Let me know how that works out for you.

Cheers,
David

View solution in original post

Super Champion

Hi @sarit_s,

If you have three different events, then use the following for each of the lines.
Line 1: ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"

Line 2: ...| rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"

Line 3: ...| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

You could also combine them as follows:

...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

Let me know how that works out for you.

Cheers,
David

View solution in original post

Communicator

Hi David,
this is how the results looks like :

Status StatusTime TotalSize StartingTime
Tue May 28 12:24:40 UTC 2019 Tue May 28 12:24:40 UTC 2019
Tue May 28 11:27:29 UTC 2019 Tue May 28 11:27:29 UTC 2019
completed 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019
completed 2019-05-27 2019-05-27 2019-05-27
total 25.1 MiB 25.1 MiB 25.1 MiB
total 5.1 MiB 5.1 MiB 5.1 MiB

Super Champion

Hi @sarit_s, updated the answer, have a look and try again !

0 Karma

Communicator

Hi @DavidHourani
Thanks !!
it is almost perfect

just the rex for StatusTime should be separeted to 2 fields :

this is the result:
2019-05-27 Tue May 28 12:24:40 UTC 2019
and i need 2019-05-27 to be one part and the rest as second part

0 Karma

Communicator

i did something like this:

| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)(?<StatusTime>.+)"

is that ok ?

0 Karma

Super Champion

Looks good, you can add the space between the two fields as well to avoid having it in StatusTime :

 | rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)\s(?<StatusTime>.+)"
0 Karma

Communicator

perfect
thanks !

0 Karma

Super Champion

This should do it :

 ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"
0 Karma

SplunkTrust
SplunkTrust

Hi,

Please try below regex

<yourBaseSearch>
| rex field=_raw "^(?:[^\s]*[\s]){3}(?<Status>\w+)[^\:]*\:\s+(?<StatusTime>[^\v]*)\v[^\:]*\:\s+(?<TotalSize>[^\v]*)\v[^\:]*\:\s+(?<StartingTime>[^\v]*)"

Regex101 URL with sample data you have provided : https://regex101.com/r/a5Rbki/1

0 Karma

Communicator

it returns empty results 😞

0 Karma

SplunkTrust
SplunkTrust

Is it multiline event? If yes then you need to provide whole sample event, part of event will not work.

0 Karma

Communicator

each event in separet line, just as i wrote at my questions
i think it should be rex for each line

 copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
 going to copy total size of:  25.1 MiB
 starting time:  Tue May 28 12:24:40 UTC 2019
0 Karma

New Member

The below SPL should show the desired end status. This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

| makeresults 
| eval test="copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019~ 
going to copy total size of:  25.1 MiB~
starting time:  Tue May 28 12:24:40 UTC 2019" 
| fields - _time 
| rex field=test "(\w*\s){3}(?<Status>\w*)[^:]*:\s+(?<StatusTime>[^\n]*)\n[^:]*:\s+(?<TotalSize>[^\n]*)\n[^:]*:\s+(?<StartingTime>.*)"

This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

0 Karma

Communicator

it returns empty results 😞

0 Karma