topic How to dynamically extract fields from Events(without internal fields like source,sourcetype,host etc..) and pipe it as fieldlist to table command. in Splunk Search

How to dynamically extract fields from Events(without internal fields like source,sourcetype,host etc..) and pipe it as fieldlist to table command.

jackson1990 — Wed, 17 Dec 2014 08:18:02 GMT

I need to create table with fields present in Events result,excluding internal fields.

Example:

Indexed Data: A=xxx B=yyy C=zzz A=fhf B=vvv C=bbb A=xer B=yqw C=zui D=yuu E=hgjhg A=clk B=viu C=bfg F=hh G=hjkh A=asd B=viu C=mnnmn F=hj G=yuyu A=rtr B=viu C=bfg F=hh G=hjkh H=ghg

Scenario 1:

SearchResult:

A=xxx B=yyy C=zzz
A=fhf B=vvv C=bbb

Expected Table:

A B C
xxx yyy zzz
fhf vvv bbb

Scenario 2:

SearchResult:

A=xer B=yqw C=zui D=yuu E=hgjhg

Expected Table:

A B C D E
xer yqw zui yuu hgjhg

Scenario 3:

SearchResult:
A=clk B=viu C=bfg F=hh G=hjkh
A=asd B=viu C=mnnmn F=hj G=yuyu
Expected Table:

A B C F G
clk viu bfg hh hjkh
asd viu mnnmn hj yuyu

I tried the below command to exclude internal fields and display the event fields alone in table

table * | fields - *Time source* host _raw date_* index linecount punct eventtype time*pos splunk_server _time

But the result i am getting(scenario 3) is

A     B   C      D           E         F    G        H
clk viu bfg   (empty)       (empty) hh  hjkh    (empty)
asd viu mnnmn   (empty)     (empty) hj  yuyu    (empty)

The same case applies for other above mentioned scenarios too.
Is it possible to remove empty fields,showing only the fields with values.Also the empty fields shouldn't be filled with null or any other static values.
or Is there any possibility to extract the fields(excluding internal fields) alone from events,so that it can be piped to table command as fieldlist.

Restrictions:
The fields in event are not static.It changes dynamically depending on the logs getting forwarded.
Fieldlist to table command shouldn't be specified in static way(like table A,B,C or table Q,A,Z,X).
It has to be generated through some other commands.

Thanks in advance...

Re: How to dynamically extract fields from Events(without internal fields like source,sourcetype,host etc..) and pipe it as fieldlist to table command.

stephane_cyrill — Wed, 17 Dec 2014 09:31:14 GMT

Hi Jackson,
To fill an empty field with null or other value you can use the commande fillnull

... | fillnull value=NULL
you can also specified the fields to work on if you like
... | fillnull value=NULL fieldx fieldy .....
you can replace NULL by a string of your choise

Re: How to dynamically extract fields from Events(without internal fields like source,sourcetype,host etc..) and pipe it as fieldlist to table command.

capnjosh — Thu, 05 Feb 2015 18:00:50 GMT

You have to filter down to only the specific event types you want before your do any automatic field extractions or "table *" commands.

I've hit this problem myself, and you have to prefilter.

If you have to filter on extracted fields (or if you have a kv_mode=json in your props.conf), you could remove all but the _raw field and then re-extract all the fields; when you do "table *" after that you should see only fields that actually show up in the results.