All Apps and Add-ons

Why are the logs not properly parsing for the Barracuda WAF/ADC Add-on for Splunk?

faisal_saifi
New Member

Hi,
I have added following configuration, but logs are not parsing properly. Do I need to change sourcetype?

[udp://X.X.X.X:514]
index = network
sourcetype= barracuda:waf 
0 Karma
1 Solution

ekost
Splunk Employee
Splunk Employee

Yes, you need to change the source type on the UDP input to sourcetype = barracuda:log per the readme.
What they've done is assume that all of the WAF data is being collected and aggregated into one log file/source.
Based upon that assumption, the Add-on splits that data back out into unique source types based upon regular expression matches.
Sample from the props.conf file in the Add-on:
[barracuda:log] TRANSFORMS-force_sourcetype_barracuda =force_sourcetype_barracuda_web,force_sourcetype_barracuda_waf,force_sourcetype_barracuda_system,
force_sourcetype_barracuda_firewall,force_sourcetype_barracuda_audit,force_sourcetype_barracuda_system_bis, force_sourcetype_barracuda_system_mongotosqlite,force_sourcetype_barracuda_system_servermon

View solution in original post

0 Karma

gaurav_maniar
Builder

Hi @ekost and @alhod2si ,

I'm also facing similar issues with this add-on. After changing the sourcetype to "barracuda:log" I'm not able to find any logs on "barracuda:*" sourcetype.

When I search the internal index, it shows the license_usage and metrics logs for firewall/web sourcetypes.

The only thing I suspect is negative avg_age and max_age in metrics logs.

Can you please help me to identify what I'm doing wrong here.

 

Event samples,

Sep 10 18:22:16 xxx.xxx.xxx.xxx Sep 11 01:22:16 ABCDXYZ 2020-09-10 22:22:16.494 +0000 ABCDXYZ TR xxx.xxx.xxx.xxx 443 xxx.xxx.xxx.xxx 2202 "-" "-" GET TLSv1.2 www.abcd.com HTTP/1.1 200 58111 943 SERVER DEFAULT PASSIVE VALID /ar/Home-standard/Kitchens-Secret xxx.xxx.xxx.xxx 2202 271

Sep 10 18:22:14 xxx.xxx.xxx.xxx Sep 11 01:22:14 ABCDXYZ 2020-09-10 22:22:14.029 +0000 ABCDXYZ WF ALER HEADER_VALUE_LENGTH_EXCEEDED xxx.xxx.xxx.xxx 63998 xxx.xxx.xxx.xxx 443 LOG NONE [length\="2691 bytes" header\="Referer" value\=" https\\://abcd.com/Serving/aasd.bs?cn\=brd&PluID\=0&Pos\=2342342344&EyeblasterID\=1080963447&clk\=1&sct\=1&dg\=1075209614&dgo\=1075209614&pc\=&sessionid\=19283719823&"] GET www.abbcd.com/ TLSv1.2 xxx.xxx.xxx.xxx 63998

 

[udp://<ip>:514]
connection_host = ip
sourcetype = barracuda:log
index = barracuda

 

 

Screen Shot 2020-09-11 at 12.57.06 AM.png

0 Karma

ekost
Splunk Employee
Splunk Employee

At first glance:

Begin by checking what you can see in the barracuda index. The way you've defined the UDP input, you should find data tagged with the correct host=(ip address) and a source=udp::514, even if sourcetype=barracuda* is failing. 

If you do an 'All time, Real time' search on the barracuda index and see the logs, it's likely the date being parsed is skewed outside of your typical search reporting window (most people use last 24 hours.)

 

If there's log data but the date and time is skewed,  the REGEX extractions for the Add-on aren't a match for your log events. Possibly, the date/time on the Barracuda device itself is an issue. There's notes in the post about using https://regex101.com and a data sample to see what's matching. If the timezone of the barracuda device itself seems to be an issue, you can define a TZ setting in props.conf using source::udp514 as noted in the docs here:  Specify time zones in props.conf.

 

 

0 Karma

ekost
Splunk Employee
Splunk Employee

Yes, you need to change the source type on the UDP input to sourcetype = barracuda:log per the readme.
What they've done is assume that all of the WAF data is being collected and aggregated into one log file/source.
Based upon that assumption, the Add-on splits that data back out into unique source types based upon regular expression matches.
Sample from the props.conf file in the Add-on:
[barracuda:log] TRANSFORMS-force_sourcetype_barracuda =force_sourcetype_barracuda_web,force_sourcetype_barracuda_waf,force_sourcetype_barracuda_system,
force_sourcetype_barracuda_firewall,force_sourcetype_barracuda_audit,force_sourcetype_barracuda_system_bis, force_sourcetype_barracuda_system_mongotosqlite,force_sourcetype_barracuda_system_servermon

View solution in original post

0 Karma

faisal_saifi
New Member

Hi ekost,
Thanks for the information. I have added input sourcetype = barracuda:log and getting three different log source barracuda:system,barracuda:web and barracuda:waf but logs are not parsing properly like souce,destination and domain etc. fields are not populating.

0 Karma

ekost
Splunk Employee
Splunk Employee

If I understand you, the data is being ingested and is going to the index (network?) you've defined in the input stanza. The data is also being source typed: the index contains source types: barracuda:system, barracuda:web, and barracuda:waf. Is that correct?
If the above is correct, it means that the regex matches are successful at a base level.

Looking a bit deeper at the Add-on's props.conf for the "barracuda:waf" source type, there are a number of additional fields:

[barracuda_waf_extractions_1]
REGEX = (WF)\s([A-Z]{4})\s(.+?)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(\d+)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(\d+)\s(DENY|LOG|WARNING)\s(NONE|LOCKED)\s\[.+?\]\s([A-Z]+)\s(.+?)\s(.+?)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(\d+)
FORMAT = log_category::$1 vendor_severity::$2 signature::$3 src_ip::$4 src_port::$5 dest_ip::$6 dest_port::$7 vendor_action::$8 vendor_follow_up_action::$9 http_method::$10 url::$11 vendor_protocol::$12 proxy_ip::$13 proxy_port::$14

They were also kind enough to provide a data sample as a reference. You can find the example in: /Splunk_TA_barracuda_waf_adc/samples/barracuda_waf.sample

  1. If you search for the source type "barracuda:waf" in your index over the recent time period when data has been ingested, and the data populates on the Search view, do you see any of those fields on the left side list of fields? The ones I expect to be very common are "dest_ip", "dest_port", "src_ip", and "src_port". I cannot say how often to expect any other fields.
  2. Take a look at the sample data for that source type. Do there appear to be significant differences if you compare it to one of your events that's being ingested "live"?

When I've seen mismatched Add-on parsing for a given data source in the past, it is usually down to product version differences. The Add-on might have been written for an earlier or later version of the barracuda product than the one you're using. As the Add-on doesn't contain any version info, you can always contact the writer for clarification as well. The last line of the readme file has an address to contact.

If those fields are extracting properly, then there might not be any problem. You'd have to find a field that should've extracted from the source data that failed to, and review the regexes designed to capture that field. Grab an event from your index, clean it up a bit, grab the regex strings from the Add-ons, find an on-line regex tester, and do some quick checks! Note: I did this using the regex above with the sample data on regex101, and the resulting groupings were a match for the listed field values.

0 Karma

alhod2si
Engager

Hello Faisal,

I have developed this add-on. As pointed out by ekost, it might not be suitable for every product/version.

But feel free to post a sample event not correctly parsed, we'll check if there is an issue or a slight change to make.

Best regards,

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!