Does Splunk have the ability to determine if fields & field counts are substantially deviating for a particular sourcetype?

[katelynengel]

Need the ability to perform a hygiene check on certain data types by comparing fields & field quantities every time a new source is ingested for a particular sourcetype. This would be an extension of fieldsummary in a way, by showing fieldsummary over time and highlighting, for example if a field changes from FieldX to fieldX.

[sundareshr]

Set this up as an alert to run right after the new data gets injested everyday

base search earliest=@d | table * | untable _time today_fields data | table today_fields | append [ search base search earliest=-1d@d latest=@d | table * | untable _time yesterday_fields data | fields yesterday_fields ] | stats values(*) AS * | eval z=mvzip(today_fields, yesterday_fields) | mvexpand z | rex field=z "(?<today>[^\,]+)\,(?<yesterday>.*)" | eval diff=if(match(yesterday, today), "match", "no") | table yesterday today diff

[inventsekar]

maybe, we can manually create an alert and if the hourly or daily change is more than a particular amount, we can send an alert.

sourcetype=ActiveDirectory earliest=-1h@h latest=@h | stats count as Today
| appendcols [search index=main earliest=-25h@h latest=-24h@h | 
 stats count as Yesterday ] | 
 eval Change = Today - Yesterday | table Change

[gcusello]

Could you give more information?
it seems as you have to search in an index for a sourcetype showing al the fields values in a table, inserting some filters
Bye.
Giuseppe

[katelynengel]

I have a source that updates once a day. The fields are as follows:

Field1, Field2, Field3, Field4, Field5

The next day, the source is slightly changed, and now the fields are coming in as follows:

field1, Field2, Field3, Field4, Field5

How can I easily detect that the first field changed from Field1 to field1? When this happens, it breaks the dashboards that refer to Field1 (uppercase).