Alerting

Generate alert listing broken Field extractions

splunk_zen
Builder

We get fed quite a wide range of formats.
This makes it hard for us to keep track of log formatting changes.

From time to time we get mission critical search alerts which rely on Field extractions

stopping to work to work because a line format has changed.

Those searches do start generating:

error in 'eval' command: The expression is malformed

so I'll be grateful if someone suggests a search picking on that,

but is it possible to limit it to Field extractions/scheduled alerts ?
We'll inevitably get spammed everytime a user makes a typo/is testing a new search otherwise.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If your critical logs each have a known regex-testable format you could define alerts that fire if events are indexed that don't match their respective regex.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Field extractions in and of themselves don't produce errors when the events don't match... they just don't match, it's normal behaviour.

If your fields are all required you could create an alert for events that don't have all fields, ie must have a changed format.

splunk_zen
Builder

Thank you Martin, but wouldn't that double the maintenance overhead?
We'd essentially have to clone the Field extrations expresssions and define search alerts.
Isn't it possible to define a single search alert which would catch all Field extractions erroring out ?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...