topic Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working? in Getting Data In

Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Tue, 07 Jun 2016 18:28:27 GMT

I've created this filter and placed them in the config files mentioned below in the following directory:

D:\Program Files (x86)\Splunk\etc\system\local

props.conf

[cisco:asa]
TRANSFORMS-null = setnull

transforms.conf

[setnull]
REGEX = (?=.*ASA-4-106100)(?=.\b(Built|Teardown|permitted)\b)
DEST_KEY = queue
FORMAT = nullQueue

The filter doesn't seem to work. Anyone have any suggestions?

Thanks

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Tue, 07 Jun 2016 20:17:32 GMT

Can you explain a bit more about your deployment? Is this is a standalone install or distributed deployment?

Have you restarted the Splunk service on the instance this configuration resides on?

Ensure the files don't have a .txt extension at the end of them, Splunk will ignore these files.

Is it possible to provide a (scrubbed) sample event?

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Tue, 07 Jun 2016 20:39:04 GMT

Anshu,

It is a standalone install. The files do not have a .txt extension.

Sample Event:
This is from the ASA itself. Unfortunately, I've already violated my license, so I cannot search anymore.

Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X.(52455) -> Outside/X.X.X.X(443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(52456) -> Outside/X.X.X.X(443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(52457) -> Outside/ X.X.X.X (443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (52458) -> Outside/ X.X.X.X (443) hit-cnt 1 first hit [0x8741ea3f, 0x4eba6142]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (2443) -> Outside/ X.X.X.X (80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/ X.X.X.X (4178) -> Outside/ X.X.X.X (80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
Jun 07 2016 16:27:30: %ASA-6-106100: access-list access-Inside-in denied udp Inside/ X.X.X.X (42030) -> Outside/ X.X.X.X (53) hit-cnt 1 first hit [0x36396194, 0x0]

So I'm trying to filter out anything with the event id matching 106100 and containing Built or Teardown or permitted.

Thanks,
Phil

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Wed, 08 Jun 2016 04:03:20 GMT

Thanks for the info, I'll post my suggestion as an answer.

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Wed, 08 Jun 2016 04:50:15 GMT

I was not able to get your expression to work with the data that was provided. Please try the following value for the "REGEX" attribute.

REGEX = %ASA-\d+-106100.*(Built|Teardown|permitted)

If needed, you should be able to make the expression case-insensitive. Please refer to the following post: https://answers.splunk.com/answers/25305/case-insensitive-transforms-conf.html

After modifying the configuration files please make sure to restart the Splunk service before testing.

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Wed, 08 Jun 2016 17:41:34 GMT

Anshu,

Thanks the regular expression you provided does work when I test it but my filter still doesn't seem to work this is what I did

props.conf

[cisco:asa]
TRANSFORMS-null = setnull

transforms.conf

[setnull]
REGEX = (%ASA-\d+-106100.*(Built|Teardown|permitted))
DEST_KEY = queue
FORMAT = nullQueue

In the following directory D:\Program Files (x86)\Splunk\etc\system\local

Does this look right?

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Wed, 08 Jun 2016 18:32:43 GMT

That looks good. Did you restart the Splunk service? What does the configuration look like for the cisco:asa data input, this can be found in inputs.conf. What search are you running to verify the results?

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Wed, 08 Jun 2016 19:12:04 GMT

Anshu,

I rebooted the server which should of restarted the service.

inputs.conf in the local directory only contains
[default]
host = SPLUNK01

in the default inputs.conf I don't see anything for ASA either. Unfortunately I've violoated my license and am waiting on a license violation reset key but most of the data being indexed is from my ASA's so I would of expected a significant drop in the amount of data being indexed after the change in the reg expression.

Thanks,
Phil

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Wed, 08 Jun 2016 19:14:40 GMT

Okay, just as an fyi, there's no need to reboot the server, just the service. How is the cisco ASA data being sent to Splunk? over syslog?

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Wed, 08 Jun 2016 19:18:58 GMT

Anshu,

Yes over syslog udp/514. When I look at source types I have 2 at the moment

cisco:asa Quick Report 1,536,092,054 6/8/16 3:17:36.000 PM
cisco_syslog Quick Report 70,655,524 6/8/16 3:17:36.000 PM

Thanks,
Phil

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

anshu — Wed, 08 Jun 2016 19:26:17 GMT

The sourcetypes might be set by the Cisco add-on, if you have that installed, while the data is being parsed. You can try applying the transforms referencing the "source" of the data. So if the "source" of the data is "udp:514" you could do the following in props.conf

[udp:514]
TRANSFORMS-null = setnull

I would look at the "source" field when you do your search to see exactly what value is being set for it.

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Thu, 09 Jun 2016 18:55:12 GMT

Anshu,

I just received the violation reset key. I'm still seeing events coming in that should be filtered out. Here is an Example of some of the events I see

6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(57937) -> Outside/X.X.X.X(80) hit-cnt 1 first hit [0x87344829, 0x8bff5156]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted tcp Inside/X.X.X.X(14860) -> Outside/X.X.X.X(80) hit-cnt 1 first hit [0x8741ea3f, 0x44de932b]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-DMZ-in permitted tcp DMZ/X.X.X.X(65010) -> Inside/X.X.X.X(80) hit-cnt 1 first hit [0xe6542d3c, 0x664ba493]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted udp Inside/X.X.X.X(53535) -> Outside/4.2.2.2(53) hit-cnt 1 first hit [0x87344829, 0x8bff5156]
host = X.X.X.X source = udp:514 sourcetype = cisco:asa
6/9/16
2:42:56.000 PM  
Jun  9 14:42:56 X.X.X.X Jun 09 2016 14:42:56: %ASA-6-106100: access-list access-Inside-in permitted udp Inside/X.X.X.X(18305) -> Outside/X.X.X.X(53) hit-cnt 1 first hit [0x87344829, 0x8bff5156]

Thanks,
Phil

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

pnazario — Fri, 10 Jun 2016 16:35:02 GMT

Anshu,

I figured it out thanks for all your help you definitely guided me down the right pat. Especially with the regular expression.

This is what worked

props.conf

[default]
TRANSFORMS-null = setnull

transforms.conf

[setnull1]
REGEX = (%ASA-\d+-106100.*(Built|Teardown|permitted))
DEST_KEY = queue
FORMAT = nullQueue

Thanks,
Phil

Re: Trying to filter ASA syslogs before indexing to avoid license violations, why are our props and transforms configurations not working?

duckcluck — Thu, 05 Mar 2020 19:48:38 GMT

Thanks for that. Switching my stanza to default worked for me too.