Getting Data In

Can you sanity check my transforms.conf config?

daniel333
Builder

All,

Can I have a quick sanity check on this transforms.conf? Basically I want to keep any log which has fatal, critical and error and drop the rest. THEN turn around and drop some of those logs which are "known".

# transforms.conf

[XYZ_keep]
  REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
  DEST_KEY=queue
  FORMAT=nullQueue

[XYZ_nullqueue]
  REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
  DEST_KEY=queue
  FORMAT=nullQueue
0 Karma
1 Solution

woodcock
Esteemed Legend

You need something like this in props.conf:

TRANSFORMS-this_string_does_not_matter = XYZ_null_all, XYZ_keep, XYZ_null_some

Then in transforms.conf:

[XYZ_null_all]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[XYZ_keep]
REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
DEST_KEY=queue
FORMAT=indexQueue

[XYZ_null_some]
REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
DEST_KEY=queue
FORMAT=nullQueue

View solution in original post

0 Karma

woodcock
Esteemed Legend

This will NOT work, it will send everything to nullQueue.

0 Karma

woodcock
Esteemed Legend

You need something like this in props.conf:

TRANSFORMS-this_string_does_not_matter = XYZ_null_all, XYZ_keep, XYZ_null_some

Then in transforms.conf:

[XYZ_null_all]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[XYZ_keep]
REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
DEST_KEY=queue
FORMAT=indexQueue

[XYZ_null_some]
REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
DEST_KEY=queue
FORMAT=nullQueue
0 Karma

richgalloway
SplunkTrust
SplunkTrust

I recommend avoiding the double-negative logic in XYZ_keep to prevent mis-understandings by future maintainers. I added the case-insensitive flag to allow for "Error", "ERROR", and "error".

[XYZ_keep]
   REGEX=(?i)(error)|(fatal)|(crit)|(splunk)
   DEST_KEY=queue
   FORMAT=indexQueue
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...