Alerting

Generate alert listing broken Field extractions

splunk_zen
Builder

We get fed quite a wide range of formats.
This makes it hard for us to keep track of log formatting changes.

From time to time we get mission critical search alerts which rely on Field extractions

stopping to work to work because a line format has changed.

Those searches do start generating:

error in 'eval' command: The expression is malformed

so I'll be grateful if someone suggests a search picking on that,

but is it possible to limit it to Field extractions/scheduled alerts ?
We'll inevitably get spammed everytime a user makes a typo/is testing a new search otherwise.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If your critical logs each have a known regex-testable format you could define alerts that fire if events are indexed that don't match their respective regex.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Field extractions in and of themselves don't produce errors when the events don't match... they just don't match, it's normal behaviour.

If your fields are all required you could create an alert for events that don't have all fields, ie must have a changed format.

splunk_zen
Builder

Thank you Martin, but wouldn't that double the maintenance overhead?
We'd essentially have to clone the Field extrations expresssions and define search alerts.
Isn't it possible to define a single search alert which would catch all Field extractions erroring out ?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Agent Mode Engaged! Enchaining Agentic Operations with Splunk AI Assistant 2.0

    Are you ready to transform how your team handles complex data requests? We invite you to our upcoming ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...