How do I find all the possible fields from our raw logs for a particular index, excluding internal fields generated by Splunk?

[ZacEsa]

Hi all,

I'm trying to create a guide for my colleagues regarding the raw logs on Splunk, but I'm stuck as I'm not sure what which fields are generated by Splunk (e.g. date_hour, date_mday, linecount, etc.) and which are the fields Splunk gets from the logs.

So, as the question states, is it possible for me to get all possible fields from a certain index excluding those fields generated by Splunk?

[muebel · Accepted solution]

Hi ZacEsa, here's a good breakdown on the default fields : http://docs.splunk.com/Documentation/Splunk/6.4.2/Data/Aboutdefaultfields

For a full list of fields, fieldsummary could be used like:

index=yourIndex | fieldsummary | table field

Please let me know if this answers your question!

[ZacEsa]

I believe this still shows the Splunk default fields right? Does the link you gave me contain all the fields that Splunk generate?

[somesoni2]

It would but you can use the field names from the link provided and exclude them, may be put in a lookup called splunk_fields.csv and use it like this

index=yourIndex | fieldsummary | table field | search NOT [| inputlookup splunk_fields.csv | table fields]

Where, splunk_fields.csv is

fields
_raw
_time
_indextime,
 _cd
..and so on

[ZacEsa]

Possible to do this without putting it in a csv? Sorry if this seems like a basic question. Haha.

[somesoni2]

You can actually create a macro and put something like this in the macro

search NOT ( field="_raw" OR field=_time OR field=source....)

and use the macro like this

index=yourIndex | fieldsummary | table field | `filtersplunkfields`

[ZacEsa]

Okay, thanks!

[sundareshr]

Very Nice!