What is a good approach to extract fields from a single event that captures a structured data table?

[wcooper003]

We have events coming in from stdout, such as the top command, where a single event captures a multi-line structured data output, e.g., this is a single Splunk event:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11981 root      20   0 2121m 860m 6996 S  0.3  1.8  36:17.82 python
12149 root      20   0 19.1g 1.0g 6556 S  0.3  2.2  45:00.03 java
13744 root      20   0 4959m 207m 5676 S  0.3  0.4  22:26.91 java
    1 root      20   0 19364 1232 1064 S  0.0  0.0   3:43.65 init

What is a good approach to do field extractions on this type of data, where a single event is a structured data table?

Thanks

[bmacias84 · Accepted solution]

You will want to use multikv. You will see Splunk using this in app like *nix.

https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Multikv
http://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/Multikvconf

I you may need to edit the multikv.conf file for your data, but should be relatively simple.

Cheers

[wcooper003]

Excellent thank you! I figured there was a built in way to do this.

[gcusello]

Hi,
The easier approach is mvexpand command:
Your_search | mvexpand | table your_fields_list.
Bye.
Giuseppe

[wcooper003]

Hi Guiseppe,

Thanks for the reply. Unfortunately there are no fields extracted since it's coming in as this structured format. For example, one of the source is 'top' which is just the 'top' command output. A typical event has 100s of lines in it. If I try mvexpand on the _raw field:

index=mihealth source=top | mvexpand _raw | table _raw

Here is a snippet of the output (12 columns for each row, 100's of rows):

PID USER PR NI VIRT RES SHR S pctCPU pctMEM cpuTIME COMMAND 4186 tomcat2 20 0 7714m 450m 7692 S 2.0 1.4 38:34.88 java 7114 root 20 0 15160 1200 808 R 2.0 0.0 0:00.01 top 7360 tomcat 20 0 21.8g 5.9g 7744 S 2.0 18.8 359:51.46 java 1 root 20 0 19344 1076 848 S 0.0 0.0 0:07.64 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd 3 root RT 0 0 0 0 S 0.0 0.0 0:04.54 migration/0 4 root 20 0 0 0 0 S 0.0 0.0 0:11.94 ksoftirqd/0 5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0 6 root RT 0 0 0 0 S 0.0 0.0 0:02.92 watchdog/0 7 root RT 0 0 0 0 S 0.0 0.0 0:06.73 migration/1 8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/1 9 root 20 0 0 0 0 S 0.0 0.0 0:06.23 ks

It seems like i need some type of mvexpand, but it doesn't appear to expand based on return characters for each row in the raw data.