topic Splitting output and field extraction in Splunk Search

Splitting output and field extraction

Branden — Fri, 27 Aug 2010 01:49:16 GMT

I'm using Subsystem Device Drivers (SDD) on an AIX system to monitor SAN LUNs. When I run "datapath query devstats" command, I get output that looks like this:

Device #:  35
=============
                Total Read  Total Write  Active Read  Active Write   Maximum
I/O:             301295802      7081834            0             0        40
SECTOR:          967435533   1003883755            0             0     11424
Transfer Size:      <= 512        <= 4k       <= 16K        <= 64K     > 64K
                     13566        32412    301058990       4143978   3128690

(The spacing isn't coming out correctly on this form, but you get the idea).

It lists X number of the above depending on how many LUNs I have assigned. In this case, there will be 36 entries (0-35, #35 being the one I just pasted).

What I need is to capture the device IDs as well as "Total Read" and "Total Write" fields for both I/O and SECTOR for each device.

This command would be running every hour, its output sent directly to Splunk.

The trick here will be telling Splunk to split the output up per Device, and then extracting the needed fields across multiple lines.

Is this possible? If so, how do I tell Splunk to break the output up into chunks divided in a certain spot?

Thanks!

Re: Splitting output and field extraction

Branden — Fri, 27 Aug 2010 02:14:34 GMT

Thanks for the edit, looks great!
One thing for people to note: the "Device #: 35" and the equal signs below it are part of the output as well.

Re: Splitting output and field extraction

Brian_Osburn — Fri, 27 Aug 2010 02:25:37 GMT

I'd modify the linebreak to include the whole event.

In addition, I was able to extract to fields using something like this:

|file /tmp/test.txt | rex field=_raw "Device #:\s+(?P<device_id>[\d]+)" | rex field=_raw "I\/O:\s+(?P<total_read_io>[\d]+)\s+(?P<total_write_io>[\d]+)\s+" | rex field=_raw "SECTOR:\s+(?P<sector_write_io>[\d]+)\s+(?P<sector_read_io>[\d]+)\s+"

Brian

Re: Splitting output and field extraction

Branden — Fri, 27 Aug 2010 02:50:13 GMT

Brian,

Thank you for your comment.
How do I go about modifying the link break to include the whole event? Can you provide some clarification on that?
Thanks!

Re: Splitting output and field extraction

gkanapathy — Mon, 28 Sep 2020 09:16:47 GMT

You can change how events are broken up (on input, so you may need to reindex data until it's right) using the BREAK_ONLY_BEFORE parameter in props.conf. That's probably the easiest way to configure it, something like BREAK_ONLY_BEFORE = ^Device #: \d+