I have a bit high-level, not too much technical, mostly a "what's your opinion" question for people who use Splunk to collect data from F5. There are two main Splunk add-ons/apps for F5 these days: Splunk Add-on for F5 BIG-IP (built and officially supported by Splunk) F5 Networks - Analytics (built and officially supported by F5) Q1: Which of them do you use in your environment, and why? Q2: Is it possible to use both of them? Do you have such setup in your environment? I have installed F5 Networks - Analytics, which comes with F5 iApp template that collects and uploads data to Splunk via F5 iControl -> Splunk HEC . Now I'm reading about Splunk Add-on for F5 BIG-IP, and it seems that this one collects the data differently. To be honest, I've got lost in its documentation. Ideally, I would love to use both, because Splunk Add-on for F5 BIG-IP is CIM compliant but F5 Networks - Analytics provides a lot of insight info and does additional analysis. However, as much as I can tell at this moment, I would end up in collecting some of the data twice (?). Your opinion is much appreciated. ... View more

Hi there, I have 3 kinds of devices: device1 (IP: 192.168.10.12, 192.168.10.13, 192.168.10.27, 192.168.10.28) device2 (IP: 192.168.20.12, 192.168.20.13, 192.168.20.27, 192.168.20.28) device3 (IP: 192.168.30.12, 192.168.30.13, 192.168.30.27, 192.168.30.28) All of them send their log data via syslog to a Splunk Heavy Forwarder (HF) that acts as a syslog collector for all the devices that cannot run a Universal Forwarder. HF processes the data (sets the sourcetype and index) and forwards it to Splunk Indexers. HF processes and forwards only the data received from device1, device2 or device3 IPs. If anyone else sends anything to HF syslog, that data is dropped (license limits, as well as I would like to have a control over what is being sent to indexers). Currently I'm filtering stuff based on the device IP, but please do not focus on that. The REGEX filtering could (and most likely will) be done based on something else. I have the following configuration: inputs.conf [udp://514] index = nullIndex connection_host = ip disabled = 0 props.conf [source::udp:514] TRANSFORMS-010-device1 = device1_sourcetype, device1_index TRANSFORMS-020-device2 = device2_sourcetype, device2_index TRANSFORMS-030-device3 = device3_sourcetype, device3_index TRANSFORMS-999-drop_everything = drop_null_index transforms.conf [device1_index] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.10\.(1[23]|2[78])$ DEST_KEY = _MetaData:Index FORMAT = device1_i [device1_sourcetype] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.10\.(1[23]|2[78])$ DEST_KEY = MetaData:Sourcetype FORMAT = device1_st [device2_index] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.20\.(1[23]|2[78])$ DEST_KEY = _MetaData:Index FORMAT = device2_i [device2_sourcetype] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.20\.(1[23]|2[78])$ DEST_KEY = MetaData:Sourcetype FORMAT = device2_st [device3_index] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.30\.(1[23]|2[78])$ DEST_KEY = _MetaData:Index FORMAT = device3_i [device3_sourcetype] SOURCE_KEY = MetaData:Host REGEX = ^host::192\.168\.30\.(1[23]|2[78])$ DEST_KEY = MetaData:Sourcetype FORMAT = device3_st [drop_null_index] REGEX = ^nullIndex$ SOURCE_KEY = _MetaData:Index DEST_KEY = queue FORMAT = nullQueue outputs.conf [tcpout] defaultGroup = splunk_indexers disabled = 0 [tcpout:splunk_indexers] server = splunk-indexers.local:9997 maxQueueSize = 500MB useACK = true disabled = 0 Pretty much all input data is marked as nullIndex right away in inputs, and then in props and transforms the desired devices are re-marked with their respective sourcetypes and indexes, and everything else is dropped. I am not sure if this is the BEST configuration (I've spent some time trying to get my head around this), but it certainly WORKS for me quite well. However, the problem is - device1 creates about 50 GB log data / day. That means 50 GB of data goes through seven REGEX filters, even though only the first two apply to device1 data. From the performance perspective, I would like to avoid the other five REGEX filters. So here's my question: is there any way to tell Splunk, right after finishing TRANSFORMS-010-device1 line in props.conf, to pretty much be done with other transformations and send the device1 data immediately to Splunk Indexers? Something similar like "Discard" tilde in RSYSLOG configuration: :fromhost-ip, isequal, "192.168.10.12" @@(o)syslog.local:6514 & ~ :fromhost-ip, isequal, "192.168.10.13" @@(o)syslog.local:6514 & ~ :fromhost-ip, isequal, "192.168.10.27" @@(o)syslog.local:6514 & ~ :fromhost-ip, isequal, "192.168.10.28" @@(o)syslog.local:6514 & ~ however not to discard the data, but remove it from any further processing other than just sending it to Splunk Indexers. So my props.conf would look something like this (?) : [source::udp:514] TRANSFORMS-010-device1 = device1_sourcetype, device1_index, device1_fast_exit TRANSFORMS-020-device2 = device2_sourcetype, device2_index, device2_fast_exit TRANSFORMS-030-device3 = device3_sourcetype, device3_index, device3_fast_exit TRANSFORMS-999-drop_everything = drop_null_index where device1_fast_exit would be (pseudo-code) "send the device1 log data immediately to Splunk Indexers and do not process that data with TRANSFORMS-020-device2 and TRANSFORMS-030-device3 lines." ... View more