Splunk Search

What would be the correct regular expression to capture lines that include these messages in my data?

lacrosse1991
Explorer

Hello,

I'm trying to set up my Splunk instance so that it filters out some lines and then leaves everything else. The lines that I'd like to remove contain one of the following values:

  • %ASA-6-302013
  • %ASA-6-302014
  • %ASA-6-302015
  • %ASA-6-302016

To filter out these lines, I added the following sections to my props.conf and transform.conf files

props.conf

 [cisco:asa]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
REGEX = %ASA-6-3020(13|14|15|16|20|21)
DEST_KEY = queue
FORMAT = nullQueue 

But for some reason, syslog lines containing values other than what was listed above are being removed as well. Is there a change that I should make to my regular expression in order to get things working correctly?

Thank you

0 Karma
1 Solution

MuS
Legend

Hi lacrosse1991,

Your config must be applied on the parsing layer, so if there are heavy weight forwarder along the way the props.conf and transforms.conf must be put on them,. See the docs for details about this http://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurationparametersandthedatapipeline#P...

Another thing to remember is that you need to restart your Splunk instance after you changed those config files.

Regarding the regex one thing that will speed it up, is to put the last two digits in a non-capturing group like this:

  %ASA-6-3020(?:13|14|15|16|20|21)

Works faster on regex101.com.

Hope this helps ...

cheers, MuS

View solution in original post

MuS
Legend

Hi lacrosse1991,

Your config must be applied on the parsing layer, so if there are heavy weight forwarder along the way the props.conf and transforms.conf must be put on them,. See the docs for details about this http://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurationparametersandthedatapipeline#P...

Another thing to remember is that you need to restart your Splunk instance after you changed those config files.

Regarding the regex one thing that will speed it up, is to put the last two digits in a non-capturing group like this:

  %ASA-6-3020(?:13|14|15|16|20|21)

Works faster on regex101.com.

Hope this helps ...

cheers, MuS

woodcock
Esteemed Legend

Also, only post-deploy, post-restart events will be effected (old events will stay broken).

0 Karma

lacrosse1991
Explorer

sorry about taking so long to respond, but thanks! things seem to be working properly now

0 Karma

maciep
Champion

a long shot, but is it possible that you have another stanza named setnull in a different transforms.conf that is winning the conflict? Maybe btool could be a quick check

splunk btool transforms list setnull --debug
0 Karma

alemarzu
Motivator

Nothing weird but try adding another rule to your configuration to see what happens.

[cisco:asa]
 TRANSFORMS-null= setnull, not_filtered

[setnull]
REGEX = %ASA-6-3020(13|14|15|16|20|21)
DEST_KEY = queue
FORMAT = nullQueue 

[not_filtered]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

Hope it helps.

0 Karma

lacrosse1991
Explorer

Thanks for your feedback. What advantage would this type of setup have over what I'm currently using?

0 Karma

koshyk
Super Champion

I can't find anything major wrong in your regex. Except may be escape % using \%

 REGEX = \%ASA-6-3020(13|14|15|16)
0 Karma

lacrosse1991
Explorer

thanks! I'm going to see if this will work

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...