Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

[pnazario]

I've created this filter and placed them in the config files mentioned below in the following directory:

D:\Program Files (x86)\Splunk\etc\system\local

props.conf

[cisco:asa]
TRANSFORMS-null = setnull

transforms.conf

[setnull]
REGEX = (?=.*ASA-4-106100)(?=.\b(Built|Teardown|permitted)\b)
DEST_KEY = queue
FORMAT = nullQueue

The filter doesn't seem to work. Anyone have any suggestions?

Thanks

[anshu · Accepted solution]

I was not able to get your expression to work with the data that was provided. Please try the following value for the "REGEX" attribute.

REGEX = %ASA-\d+-106100.*(Built|Teardown|permitted)

If needed, you should be able to make the expression case-insensitive. Please refer to the following post: https://answers.splunk.com/answers/25305/case-insensitive-transforms-conf.html

After modifying the configuration files please make sure to restart the Splunk service before testing.

[duckcluck]

Thanks for that. Switching my stanza to default worked for me too.

[pnazario]

Anshu,

Thanks the regular expression you provided does work when I test it but my filter still doesn't seem to work this is what I did

props.conf

[cisco:asa]
TRANSFORMS-null = setnull

transforms.conf

[setnull]
REGEX = (%ASA-\d+-106100.*(Built|Teardown|permitted))
DEST_KEY = queue
FORMAT = nullQueue

In the following directory D:\Program Files (x86)\Splunk\etc\system\local

Does this look right?

[anshu]

That looks good. Did you restart the Splunk service? What does the configuration look like for the cisco:asa data input, this can be found in inputs.conf. What search are you running to verify the results?

[pnazario]

Anshu,

I rebooted the server which should of restarted the service.

inputs.conf in the local directory only contains
[default]
host = SPLUNK01

in the default inputs.conf I don't see anything for ASA either. Unfortunately I've violoated my license and am waiting on a license violation reset key but most of the data being indexed is from my ASA's so I would of expected a significant drop in the amount of data being indexed after the change in the reg expression.

Thanks,
Phil

[anshu]

Okay, just as an fyi, there's no need to reboot the server, just the service. How is the cisco ASA data being sent to Splunk? over syslog?

[pnazario]

Anshu,

Yes over syslog udp/514. When I look at source types I have 2 at the moment

cisco:asa Quick Report 1,536,092,054 6/8/16 3:17:36.000 PM
cisco_syslog Quick Report 70,655,524 6/8/16 3:17:36.000 PM

Thanks,
Phil

[anshu]

The sourcetypes might be set by the Cisco add-on, if you have that installed, while the data is being parsed. You can try applying the transforms referencing the "source" of the data. So if the "source" of the data is "udp:514" you could do the following in props.conf

[udp:514]
TRANSFORMS-null = setnull

I would look at the "source" field when you do your search to see exactly what value is being set for it.

[pnazario]

Anshu,

I just received the violation reset key. I'm still seeing events coming in that should be filtered out. Here is an Example of some of the events I see

6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(57937) -> Outside/X.X.X.X(80) hit-cnt 1 first hit [0x87344829, 0x8bff5156]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(14860) -> Outside/X.X.X.X(80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-DMZ-in permitted tcp DMZ/X.X.X.X(65010) -> Inside/X.X.X.X(80) hit-cnt 1 first hit [0xe6542d3c, 0x664ba493]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted udp Inside/X.X.X.X(53535) -> Outside/4.2.2.2(53) hit-cnt 1 first hit [0x87344829, 0x8bff5156]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted udp Inside/X.X.X.X(18305) -> Outside/X.X.X.X(53) hit-cnt 1 first hit [0x87344829, 0x8bff5156]

Thanks,
Phil

[pnazario]

Anshu,

I figured it out thanks for all your help you definitely guided me down the right pat. Especially with the regular expression.

This is what worked

props.conf

[default]
TRANSFORMS-null = setnull

transforms.conf

[setnull1]
REGEX = (%ASA-\d+-106100.*(Built|Teardown|permitted))
DEST_KEY = queue
FORMAT = nullQueue

Thanks,
Phil

[anshu]

Can you explain a bit more about your deployment? Is this is a standalone install or distributed deployment?

Have you restarted the Splunk service on the instance this configuration resides on?

Ensure the files don't have a .txt extension at the end of them, Splunk will ignore these files.

Is it possible to provide a (scrubbed) sample event?

[pnazario]

Anshu,

It is a standalone install. The files do not have a .txt extension.

Sample Event:
This is from the ASA itself. Unfortunately, I've already violated my license, so I cannot search anymore.

Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X.(52455) -> Outside/X.X.X.X(443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(52456) -> Outside/X.X.X.X(443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(52457) -> Outside/ X.X.X.X (443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (52458) -> Outside/ X.X.X.X (443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (2443) -> Outside/ X.X.X.X (80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (4178) -> Outside/ X.X.X.X (80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in denied udp Inside/ X.X.X.X (42030) -> Outside/ X.X.X.X (53) hit-cnt 1 first hit [0x36396194, 0x0]

So I'm trying to filter out anything with the event id matching 106100 and containing Built or Teardown or permitted.

Thanks,
Phil

[anshu]

Thanks for the info, I'll post my suggestion as an answer.