All Apps and Add-ons
Highlighted

How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Hi All, I need to customize the splunk Add-on for F5 BIG-IP based on the data ingested into splunk.

F5 LTM data are being ingested into splunk Environment from syslogs servers. We have 5 Heavy forwarder instances configured to fetch this syslogs data's and forwards it to the 5 individual indexer instances.
Splunk F5 Add-on is uploaded in search head cluster master and we want to customize the Technology add on, based on the data available in splunk from F5.

Currently we could see the F5 data being ingested into splunk from 17 hosts and could see some 104 fields under interesting field details. Below is the inputs.conf detail.

inputs.conf detail:

F5 LTM

[monitor:///opt/syslogs/webaccess/.../*.log]
index = web
app
sourcetype = f5:network:loadbalancer
host_segment = 4

From the splunk Add-on for F5 BIG-IP, I could see the below details:

Appserver -- Not sure what is the purpose of it and whether we need to use this for our data.
Default folder contains following Technology add on configuration file like
props.conf, transforms.conf,web.conf, eventtypes.conf, tags.conf and these configuration files contains lots of predefined information but not sure how/where to start of with customization.

Kindly guide me on how to customize the Add-on app based on F5 LTM data available in splunk.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Hi All, Can any one guide me to get started with this ...
thanks in advance.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Hi there @Hemnaath

From where it comes from the sourcetype you are using? Because if I'm not mistaken, according to the type of input (syslog) you should use this sourcetype "f5:bigip:syslog". The add-on will recognize certain patterns on your data through regexes and will categorize the events into new sourcetypes, but in order for this "categorization/renaming" to work, the previews sourcetype is needed on your input.

By doing this all the intelligence thats comes from this add-on will occur.

Hope it helps.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Hi Alemarzu, Yes data are being ingested from syslog and we have customized app to monitor the F5 data from syslog. This is the query index= web sourcetype = f5:network:loadbalancer is used to fetch the data in splunk console. Do you want me to change the sourcetype to "f5:bigip:syslog" ?

Not sure what are the stanza in props,transforms,eventtypes,tags should be customized in this app based on the data available in splunk.

Kindly guide me on this please.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Do you want me to change the sourcetype to "f5:bigip:syslog" ?
Yes.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

thanks Alemaruz, after changing the sourcetype, do i need to change anything in props.conf, transforms.conf, eventtype.conf and tags.conf.

In props.conf for the sourcetype=f5:big:syslog I could see below details and along with this stanza there are other stanza related to other sourcetypes. Should we need to remove them as i our case we have only one sourcetype is used to fetch the F5 LTM data into splunk.

  Rename   

General

[f5_bigip:syslog]
rename = f5:bigip:syslog

General

[f5:bigip:syslog]
SHOULDLINEMERGE = false
LINE
BREAKER = ([\r\n]+)
TRANSFORMS-sourcetype=f5bigip-irule-default, f5bigip-irule-http, f5bigip-irule-dns-request, f5bigip-irule-dns-response, f5bigip-irule-lb-failed, f5bigip-syslog-asm, f5-bigip-apm-syslog, f5_bigip-irule-exclude-audit

Kindly guide me on this.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

You do not need to change anything else. Props & Transforms config files holds the parsing/extraction knowledge for your data.

The stanzas that you mention are the ones that recognize certain patterns on your F5 logs and then rename the sourcetypes according to the type of event.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

Hi Alemarzu, thanks for your effort on this. So it mean we need to change only the sourcetype in inputs.conf and upload the splunk Add-on for F5 BIG-IP in the search head cluster master, without altering the other config file present in the add-on.

I have doubt before proceeding to change the sourcetype inputs.conf, since the data which is currently being ingested in splunk is with the sourcetype=f5:network:loadbalancer, so by changing the source type will that have any impact. I mean will data ingest into splunk properly. I am going to push this app in search head cluster member.

Kindly advise me.

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

Motivator

I don´t have a SHC on my arch but it's working okey and no changes were needed on the configuration files besides the sourcetypes in /local/inputs.conf according to the type of input.

Edit: http://docs.splunk.com/Documentation/AddOns/released/F5BIGIP/Install#Distributed_deployment_feature_...

0 Karma
Highlighted

Re: How to customize splunk Add-on for F5 BIG-IP based on the data available in Splunk?

SplunkTrust
SplunkTrust

Hi, folks. I'm trying to follow this train of thought and it sounds like we're at the edge of an answer. If I may throw a few supporting ideas out here to help ...

Also, if we're about at an answer, I would suggest @alemarzu that you convert your answer into an actual answer so that it can be accepted - although, it might be useful to summarize the problem for the answer itself.

Anyway.

Hemnaath, there's a few things you might want to review about how Splunk works with respect to configuration file precedence. The files in the app/default folders shouldn't be changed - you should copy the relevant pieces to your app/local folder to "override" the settings.

It's also a bit more reading, but the getting data in manual has a discussion on sourcetypes and why they matter, too.

The F5 app documents a list of sourcetypes. I know little about this app, but I can see that your sourcetype for incoming syslog data from the F5's should be set to f5:bigip:syslog. I can't be sure about that having little experience with this particular app, but it makes sense; it fits the docs (probably, unless I"m just reading them wrong) and matches what @alemarzu indicates.

So here's the crux of the matter - if you are ingesting the data right now with a sourcetype that isn't right, the data is in Splunk but probably doesn't quite 'work' right. It doesn't populate panels on the F5 dashboards, fields aren't all there and so on. In that case, there's not much harm in changing the sourcetype to something else - it's won't make it much worse, and may fix it all. And if it DOESN'T fix it all, just remove that overridden piece of configuration and it'll all be back to your originals again. This process will only take a few minutes to test, just keep track of what you are doing so you can easily change it back if necessary.

And do let us know if that works out for you!

0 Karma