Splunk Enterprise

Does Splunk have the ability to determine if fields & field counts are substantially deviating for a particular sourcetype?


Need the ability to perform a hygiene check on certain data types by comparing fields & field quantities every time a new source is ingested for a particular sourcetype. This would be an extension of fieldsummary in a way, by showing fieldsummary over time and highlighting, for example if a field changes from FieldX to fieldX.

Tags (1)


Set this up as an alert to run right after the new data gets injested everyday

base search earliest=@d | table * | untable _time today_fields data | table today_fields | append [ search base search earliest=-1d@d latest=@d | table * | untable _time yesterday_fields data | fields yesterday_fields ] | stats values(*) AS * | eval z=mvzip(today_fields, yesterday_fields) | mvexpand z | rex field=z "(?<today>[^\,]+)\,(?<yesterday>.*)" | eval diff=if(match(yesterday, today), "match", "no") | table yesterday today diff
0 Karma

Super Champion

maybe, we can manually create an alert and if the hourly or daily change is more than a particular amount, we can send an alert.

sourcetype=ActiveDirectory earliest=-1h@h latest=@h | stats count as Today
| appendcols [search index=main earliest=-25h@h latest=-24h@h | 
 stats count as Yesterday ] | 
 eval Change = Today - Yesterday | table Change

alt text

0 Karma

Esteemed Legend

Could you give more information?
it seems as you have to search in an index for a sourcetype showing al the fields values in a table, inserting some filters

0 Karma


I have a source that updates once a day. The fields are as follows:

Field1, Field2, Field3, Field4, Field5

The next day, the source is slightly changed, and now the fields are coming in as follows:

field1, Field2, Field3, Field4, Field5

How can I easily detect that the first field changed from Field1 to field1? When this happens, it breaks the dashboards that refer to Field1 (uppercase).

0 Karma
Get Updates on the Splunk Community!

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...

Splunk Life | Happy Pride Month!

Happy Pride Month, Splunk Community! &#x1f308; In the United States, as well as many countries around the ...

SplunkTrust | Where Are They Now - Michael Uschmann

The Background Five years ago, Splunk published several videos showcasing members of the SplunkTrust to share ...