Splunk Search

Multi-line event field extraction

Path Finder

I have logs being indexed that look like:

/some/filesystem/path 1234567890 1500

/some/filesystem/path2 1256320145 452633

/some/filesystem2/path 2365800125 1122233

/some/filesystem2/path2 6539025630 553222

...

I am trying to get each line extracted into 3 fields (for example fs, fs_total, fs_used). Some logs will have 1 line, and some logs will have 400+ lines.

I have tried rex like

index=disk sourcetype="disk_logs"| rex field=_raw "(?i)(?m)[0-9]{1,15}(?<filesystem_used>.*)"

and that will give me the last column, but I'm not sure how to get all 3 columns into 3 fields for all the lines per log.

Tags (2)
0 Karma
1 Solution

Influencer

Assuming that all lines are seperate events, you can use such a regex:

index=disk sourcetype="disk_logs" | rex "^(?<fs>.+?)\s+(?<fs_total>\d+)\s+(?<fs_used>\d+)$"

Otherwise you can split the events into seperate results:

index=disk sourcetype="disk_logs" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line | rex field=line "^(?<fs>.+?)\s+(?<fs_total>\d+)\s+(?<fs_used>\d+)$"

View solution in original post

Influencer

Assuming that all lines are seperate events, you can use such a regex:

index=disk sourcetype="disk_logs" | rex "^(?<fs>.+?)\s+(?<fs_total>\d+)\s+(?<fs_used>\d+)$"

Otherwise you can split the events into seperate results:

index=disk sourcetype="disk_logs" | rex "(?m)^(?<line>\V+)$" max_match=10000 | table line | mvexpand line | rex field=line "^(?<fs>.+?)\s+(?<fs_total>\d+)\s+(?<fs_used>\d+)$"

View solution in original post

Influencer

Great! Please accept the answer.

0 Karma

Path Finder

makes the perfect table, thank you so much! now i just need to figure out how to make some pretty graphs with each fs_total and fs_used as the y-axis and time as the x-axis (each event has a unix m-time which splunk seems to respect) and make a graph per fs

0 Karma

Influencer

even if you did not generate the logs, you can still configure how splunk is doing the line breaking

transforms.conf

[disk_logs]
SHOULD_LINEMERGE = false
0 Karma

Influencer

Is this by intention?

0 Karma

Path Finder

works great! except for the events that have multiple lines. the largest is 442 lines in one event. not sure how to go about reading each line and extracting the 3 fields.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes and swag!