Getting Data In

ASA filter not letting logs through?

Svill321
Path Finder

Hello everyone,

One of the projects I worked on was to build a filter for ASA logs in Splunk so logs we were not interested in would not be indexed, thus preserving the license. I did that, and it seems to have worked too well. Were we once had maybe 1.5 million events in 24 hours, Splunk now shows less than three hundred. I made the filter with a props.conf and transforms.conf to search for and allow only events that have one of the listed ID numbers, as shown here:

#props.conf
[sourcetype_name]
TRANSFORMS-set=setnull,setparsing

#transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \d{4}\-\d{2}-\d{2}.+?%ASA\-\d{1}\-(106100|106023|106015)
DEST_KEY = queue
FORMAT = indexQueue

I think I noticed a typo in the original regex (which I can't find now) where one of the numbers had an apostrophe in it. Given that I'm using pipes, though, that shouldn't matter. Indeed, I'm still getting the events I'm filtering for, just much fewer of them. Any ideas?

Edit: To clarify, the problem is not that I'm getting no events, or that the filter is not working, but that I am getting far fewer events than expected of those logs that are indexed. I found a typo in the original file where "DEST_KEY = queue" under [setparsing] is "DSET_KEY = queue". I'm not sure if this is the issue though, as my understanding is that this would set all events to NULL.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

In order to identify the error in the regex, we need examples of logs that were parsed correctly, and of logs that were not parsed correctly but which were not supposed to be blacklisted.

The way I read that regex, you have a datestamp followed by a bunch of stuff, then a percent sign, ASA, a dash, a number, one more dash, and then the message number. The escaping on the dashes doesn't seem to be needed, but it probably isn't hurting anything.

 2017-08-10xxxxxxxxxxxxxx%ASA-6-106015: more stuff more stuff

Look to verify that your incoming events timestamps are consistent, and always have two-digit days and months. If sometimes the months and days are 1-digit, then change the beginning of the regex to pick it up, such as...

REGEX =   \d{4}\-\d{1,2}-\d{1,2}.+?%ASA\-\d{1}\-(106100|106023|106015)

If there are no percent signs in the zone marked xxxx, then you could even simplify your regex to ...

REGEX = ^[^%]*%ASA-\d-(106100|106023|106015)
0 Karma

Svill321
Path Finder

I don't think the regex is the issue - I did further testing and was able to match all events in the filter with raw events I extracted. I did manage to get access to the server it was deployed on (my permissions are fairly limited), and I saw that I misspelled "DEST_KEY = queue" under [setparsing] as "DSET_KEY = queue". Wouldn't this cause there to be NO events returned, though, as everything is being set to null?

Also, I edited the question for more clarification.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...