Getting Data In

ASA filter not letting logs through?

Path Finder

Hello everyone,

One of the projects I worked on was to build a filter for ASA logs in Splunk so logs we were not interested in would not be indexed, thus preserving the license. I did that, and it seems to have worked too well. Were we once had maybe 1.5 million events in 24 hours, Splunk now shows less than three hundred. I made the filter with a props.conf and transforms.conf to search for and allow only events that have one of the listed ID numbers, as shown here:

#props.conf
[sourcetype_name]
TRANSFORMS-set=setnull,setparsing

#transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \d{4}\-\d{2}-\d{2}.+?%ASA\-\d{1}\-(106100|106023|106015)
DEST_KEY = queue
FORMAT = indexQueue

I think I noticed a typo in the original regex (which I can't find now) where one of the numbers had an apostrophe in it. Given that I'm using pipes, though, that shouldn't matter. Indeed, I'm still getting the events I'm filtering for, just much fewer of them. Any ideas?

Edit: To clarify, the problem is not that I'm getting no events, or that the filter is not working, but that I am getting far fewer events than expected of those logs that are indexed. I found a typo in the original file where "DEST_KEY = queue" under [setparsing] is "DSET_KEY = queue". I'm not sure if this is the issue though, as my understanding is that this would set all events to NULL.

0 Karma

SplunkTrust
SplunkTrust

In order to identify the error in the regex, we need examples of logs that were parsed correctly, and of logs that were not parsed correctly but which were not supposed to be blacklisted.

The way I read that regex, you have a datestamp followed by a bunch of stuff, then a percent sign, ASA, a dash, a number, one more dash, and then the message number. The escaping on the dashes doesn't seem to be needed, but it probably isn't hurting anything.

 2017-08-10xxxxxxxxxxxxxx%ASA-6-106015: more stuff more stuff

Look to verify that your incoming events timestamps are consistent, and always have two-digit days and months. If sometimes the months and days are 1-digit, then change the beginning of the regex to pick it up, such as...

REGEX =   \d{4}\-\d{1,2}-\d{1,2}.+?%ASA\-\d{1}\-(106100|106023|106015)

If there are no percent signs in the zone marked xxxx, then you could even simplify your regex to ...

REGEX = ^[^%]*%ASA-\d-(106100|106023|106015)
0 Karma

Path Finder

I don't think the regex is the issue - I did further testing and was able to match all events in the filter with raw events I extracted. I did manage to get access to the server it was deployed on (my permissions are fairly limited), and I saw that I misspelled "DEST_KEY = queue" under [setparsing] as "DSET_KEY = queue". Wouldn't this cause there to be NO events returned, though, as everything is being set to null?

Also, I edited the question for more clarification.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!