Splunk Search
Highlighted

What is a good approach to extract fields from a single event that captures a structured data table?

Communicator

We have events coming in from stdout, such as the top command, where a single event captures a multi-line structured data output, e.g., this is a single Splunk event:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11981 root      20   0 2121m 860m 6996 S  0.3  1.8  36:17.82 python
12149 root      20   0 19.1g 1.0g 6556 S  0.3  2.2  45:00.03 java
13744 root      20   0 4959m 207m 5676 S  0.3  0.4  22:26.91 java
    1 root      20   0 19364 1232 1064 S  0.0  0.0   3:43.65 init

What is a good approach to do field extractions on this type of data, where a single event is a structured data table?

Thanks

0 Karma
Highlighted

Re: What is a good approach to extract fields from a single event that captures a structured data table?

Legend

Hi,
The easier approach is mvexpand command:
Yoursearch | mvexpand | table yourfields_list.
Bye.
Giuseppe

0 Karma
Highlighted

Re: What is a good approach to extract fields from a single event that captures a structured data table?

Communicator

Hi Guiseppe,

Thanks for the reply. Unfortunately there are no fields extracted since it's coming in as this structured format. For example, one of the source is 'top' which is just the 'top' command output. A typical event has 100s of lines in it. If I try mvexpand on the _raw field:

index=mihealth source=top | mvexpand _raw | table _raw

Here is a snippet of the output (12 columns for each row, 100's of rows):

PID USER PR NI VIRT RES SHR S pctCPU pctMEM cpuTIME COMMAND 4186 tomcat2 20 0 7714m 450m 7692 S 2.0 1.4 38:34.88 java 7114 root 20 0 15160 1200 808 R 2.0 0.0 0:00.01 top 7360 tomcat 20 0 21.8g 5.9g 7744 S 2.0 18.8 359:51.46 java 1 root 20 0 19344 1076 848 S 0.0 0.0 0:07.64 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd 3 root RT 0 0 0 0 S 0.0 0.0 0:04.54 migration/0 4 root 20 0 0 0 0 S 0.0 0.0 0:11.94 ksoftirqd/0 5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0 6 root RT 0 0 0 0 S 0.0 0.0 0:02.92 watchdog/0 7 root RT 0 0 0 0 S 0.0 0.0 0:06.73 migration/1 8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/1 9 root 20 0 0 0 0 S 0.0 0.0 0:06.23 ks

It seems like i need some type of mvexpand, but it doesn't appear to expand based on return characters for each row in the raw data.

0 Karma
Highlighted

Re: What is a good approach to extract fields from a single event that captures a structured data table?

Champion

You will want to use multikv. You will see Splunk using this in app like *nix.

https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Multikv
http://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/Multikvconf

I you may need to edit the multikv.conf file for your data, but should be relatively simple.

Cheers

View solution in original post

Highlighted

Re: What is a good approach to extract fields from a single event that captures a structured data table?

Communicator

Excellent thank you! I figured there was a built in way to do this.

0 Karma