Splunk Enterprise

Does Splunk have the ability to determine if fields & field counts are substantially deviating for a particular sourcetype?


Need the ability to perform a hygiene check on certain data types by comparing fields & field quantities every time a new source is ingested for a particular sourcetype. This would be an extension of fieldsummary in a way, by showing fieldsummary over time and highlighting, for example if a field changes from FieldX to fieldX.

Tags (1)


Set this up as an alert to run right after the new data gets injested everyday

base search earliest=@d | table * | untable _time today_fields data | table today_fields | append [ search base search earliest=-1d@d latest=@d | table * | untable _time yesterday_fields data | fields yesterday_fields ] | stats values(*) AS * | eval z=mvzip(today_fields, yesterday_fields) | mvexpand z | rex field=z "(?<today>[^\,]+)\,(?<yesterday>.*)" | eval diff=if(match(yesterday, today), "match", "no") | table yesterday today diff
0 Karma

Super Champion

maybe, we can manually create an alert and if the hourly or daily change is more than a particular amount, we can send an alert.

sourcetype=ActiveDirectory earliest=-1h@h latest=@h | stats count as Today
| appendcols [search index=main earliest=-25h@h latest=-24h@h | 
 stats count as Yesterday ] | 
 eval Change = Today - Yesterday | table Change

alt text

0 Karma

Esteemed Legend

Could you give more information?
it seems as you have to search in an index for a sourcetype showing al the fields values in a table, inserting some filters

0 Karma


I have a source that updates once a day. The fields are as follows:

Field1, Field2, Field3, Field4, Field5

The next day, the source is slightly changed, and now the fields are coming in as follows:

field1, Field2, Field3, Field4, Field5

How can I easily detect that the first field changed from Field1 to field1? When this happens, it breaks the dashboards that refer to Field1 (uppercase).

0 Karma
Get Updates on the Splunk Community!

Splunk APM & RUM | Upcoming Planned Maintenance

There will be planned maintenance of the streaming infrastructure for Splunk APM and Splunk RUM in the coming ...

Part 2: Diving Deeper With AIOps

Getting the Most Out of Event Correlation and Alert Storm Detection in Splunk IT Service Intelligence   Watch ...

User Groups | Upcoming Events!

If by chance you weren't already aware, the Splunk Community is host to numerous User Groups, organized ...