All Apps and Add-ons

Why is the Splunk Add-on for F5 BIG-IP not separating sourcetypes as expected?

jonesnadiam
Path Finder

Has anyone had issues with the Splunk Add-on for F5 BIG-IP setting/separating the sourcetypes?
According to the documentation, if the sourcetype is set to f5:bigip:syslog, the data should be separated into its specific sourcetypes (f5:bigip:apm:syslog, f5:bigip:asm:syslog, f5:bigip:icontrol, etc), but all of our sourcetypes are still coming in as f5:bigip:syslog.

Is there anything specific that I need to change in the configuration files so that these sourcetypes are automatically updated?

Thanks.

esmat777
Explorer

@jonesnadiam wrote:

Has anyone had issues with the Splunk Add-on for F5 BIG-IP setting/separating the sourcetypes?
According to the documentation, if the sourcetype is set to f5:bigip:syslog, the data should be separated into its specific sourcetypes (f5:bigip:apm:syslog, f5:bigip:asm:syslog, f5:bigip:icontrol, etc), but all of our sourcetypes are still coming in as f5:bigip:syslog.

Is there anything specific that I need to change in the configuration files so that these sourcetypes are automatically updated?

Thanks.


yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format

and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs

which will not be presented on Datamodels Or dashboards 

so i make a new files at local folder
props.conf 

Spoiler
### ASM ###
[f5:bigip:asm:syslog]
EVAL-attack_type = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)
EVAL-category = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)

and tags.conf

 

Spoiler

### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled

[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled

 

AND eventtypes.conf

 

Spoiler

[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)

[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)

 

 and now everything is working fine and data are tagged

this should be added to addons in the next release 

0 Karma

esmat777
Explorer

the same issue with F5-addons too

when i change logging profile for F5 v15.1 as below options

1. F5 Logging Profile (Syslog ) ==> addon F5-big IP not working as log come in below format not like
F5-addons format at props/transform files.

130>Sep 30 10:39:44 F5-01.*.com ASM:unit_hostname="F5-01.*.com"

and F5-add on match only below format

<131>Sep 12 23:53:50 F5-01.*.com ASM:f5_asm=Splunk-F5-AS


when i change logging profile from F5 v15.1 to pre-define template format call "Splunk"
2.  F5 Logging Profile (Splunk) ==> logs come in duplicated event parameters

when i change logging profile for F5 v15.1 to custom template at Splunk-Docs 

Configure F5 Logging Profiles for ASM

https://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup


3. 2.  F5 Logging Profile (Custom template) ==> logs come in duplicated event parameters too

0 Karma

youngsuh
Contributor

We're having the same issue with the f5 add-on.  The problem is the add-on because if you don't use the add-on the event braking works with 8.2 just fine.  The sub sorucetype [f5:bigip:apm:syslog] works fine.  The root sourcetype [f5:bigip:syslog] isn't parsing correctly for TCP stream or Syslog format.  

BSD_format.txt
Event 1:
May 17 23:33:58 %masked_host% apmd[13742]: 01490115:5: /Common/fs.training.np2.navy.mil_modern:Common:cbd11dc1: Following rule 'CAC' from item 'Logon Authentication Type Switch' to terminalout 'CAC'

Event 2:
May 17 23:34:01 %masked_host% /Common/fs.training.np2.navy.mil_modern: Common:cbd11dc1: iRule access_policy_default CLIENTSSL_CLIENTCERT | DEBUG | Got 2 certs ||

We had our F5 SME provide two different format that configurable with the F5 appliance. 
Here is the line brake by the TA.
[f5:bigip:syslog]LINE_BREAKER = ([\r\n]+)(.*)(f5_irule|[^"]f5_asm|:\s(?:\d{4})[0-9A-Fa-f]{4}:\d+:\s+(?:[^:\s]{1,100}:[^:\s]{1,100}:\s*)?[0-9a-f]{8}:|( debug | info | warning | err | notice | alert | crit | emerg ))

Fails to parse the above two events.

Splunk_format.txt
Fails to line brake with for all the events.
[f5:bigip:syslog]LINE_BREAKER = ([\r\n]+)(.*)(f5_irule|[^"]f5_asm|:\s(?:\d{4})[0-9A-Fa-f]{4}:\d+:\s+(?:[^:\s]{1,100}:[^:\s]{1,100}:\s*)?[0-9a-f]{8}:|( debug | info | warning | err | notice | alert | crit | emerg ))

Were you able to make progress? 

0 Karma

esmat777
Explorer

yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format

and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs

which will not be presented on Datamodels Or dashboards 

so i make a new files at local folder
props.conf

Spoiler
### ASM ###
[f5:bigip:asm:syslog]
EVAL-attack_type = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)
EVAL-category = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)

and tags.conf

 

Spoiler

### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled

[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled

 

AND eventtypes.conf

 

Spoiler

[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)

[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)

 

 and now every thing is working fine and data are tagged

this should be added to addons in next release 

youngsuh
Contributor

@esmat777 Could you explain in more which sourcetype worked after fix the f5_bigip_asm_syslog?  I don't see any difference between your prop.conf, eventtpe.conf and tags.  Are you saying you had the Splunk system admin to change the regex on f5:bigip:asm:syslog?  If yes, how are you are ingestion the F5 logs?  TCP or syslog?  Are you able to see the following sourcetypes: TRANSFORMS-sourcetype=f5_bigip-irule-default, f5_bigip-irule-http, f5_bigip-irule-dns-request, f5_bigip-irule-dns-response, f5_bigip-irule-lb-failed, f5_bigip-syslog-asm, f5-bigip-apm-syslog, f5_bigip-irule-exclude-audit, f5_bigip-secure, f5_bigip-ltm-ssl-error, f5_bigip-ltm-tcl-error, f5_bigip-ltm-traffic, f5_bigip-ltm-log-error.

Can you create idea of your fix so, that it could be included in the next release of the add-on?

0 Karma

ianbow_concur
New Member

You need to make sure the TA is installed on the indexer or heavy forwarder if you are using them. Due to the line breaks etc a Universal forwarder will not expand the events in the transforms to give you the expanded source types.

0 Karma

youngsuh
Contributor

Yes.  This isn't the issue.  It's different beast entirely. 

0 Karma

esmat777
Explorer

this is not an issue where to install the addon

it is related to a raw data format which is not compatible with addon

0 Karma

esmat777
Explorer

yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format

and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs

which will not be presented on Datamodels Or dashboards 

so i make a new files at local folder
props.conf 

Spoiler
### ASM ###
[f5:bigip:asm:syslog]
EVAL-attack_type = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)
EVAL-category = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)

and tags.conf

 

Spoiler

### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled

[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled

 

AND eventtypes.conf

 

Spoiler

[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)

[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)

 

 and now everything is working fine and data are tagged

this should be added to addons in the next release @jonesnadiam 

wojtek_emca
New Member

This could happened if F5 is not configured well (sending logs format).

Should be like this:
http://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup

Regards,
Wojtek

0 Karma

hungpham
Explorer

Hmm i got same problem, splunk not separating sourcetypes.
I try follow http://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Setup but the splunk only receive syslog audit.

sirajnp
Path Finder

Hi,

Got same problem here. Add-on cannot extract fields for ASM audit events.

Sourcetype: f5:bigip:syslog
Port: udp:9514

Have you guys get through this issue?

pvuong
Explorer

Hello,
Did you get any answer about this question ? I have the same problem for all my F5:big IP logs

esmat777
Explorer

yes , i have fixed it by asking System administrator to change the 1st part of raw data from F5-Logging profile
to the one which match with "f5_asm" format

and it work
=====> but found another issue with Addon (( Addone -F5 BigIp) didnot tag ASM-logs

which will not be presented on Datamodels Or dashboards 

so i make a new files at local folder
props.conf 

Spoiler
### ASM ###
[f5:bigip:asm:syslog]
EVAL-attack_type = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)
EVAL-category = if(isnull(attack_type) or attack_type="" or attack_type="N/A" or attack_type="-", null, attack_type)

and tags.conf

 

Spoiler

### ASM ###
[eventtype=f5_bigip_asm_syslog_attack]
web = enabled
communicate = enabled
network = enabled
attack = enabled
ids = enabled

[eventtype=f5_bigip_asm_syslog]
web = enabled
communicate = enabled
network = enabled

 

AND eventtypes.conf

 

Spoiler

[f5_bigip_asm_syslog]
search = sourcetype="f5:bigip:asm:syslog" (attack_type="N/A" OR NOT attack_type=*)

[f5_bigip_asm_syslog_attack]
search = sourcetype="f5:bigip:asm:syslog" (attack_type!="N/A" AND attack_type=*)

 

 and now everything is working fine and data are tagged

this should be added to addons in the next release  @pvuong  @jonesnadiam  @ianbow_concur  @sirajnp 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...