All Apps and Add-ons
Highlighted

Matching specific index

New Member

Hi,

I am trying to send events in a specific index, regardless of sourcetype, to the Diode Receiver Add-On but cannot really get it to work.

Setting up the add-on using [default] stanza in props.conf matches events in ALL indexes including _internal and that really makes a mess of the main index in the receiver.

I tried using the CEF Add-On but I'm not sure how to configure the routing for cefout. Can it even be configured to send UDP?

This is all done in a test environment without a hardware diode for easy troubleshooting but the goal is to set up two splunk servers separated by a UDP-only-diode and have the main index in both servers contain the same information.

0 Karma
Highlighted

Re: Matching specific index

New Member

Hi!
The filtering doesn't happen by on the receiver side by default, that's better done on the sending side.
The requirement to send an entire index has come up more often, so that's probably worth to document.
In props.conf it's easy to filter based on sourcetype, host our source, but not really by index, so if you can't limit yourself to host, source or sourcetype, it may be better to go the CEF Add-On route. This also has the advantage you have more control over what is sent and you can send historical data as well.

CEF Add-On can send UDP as well, but if you send large events (like Windows Event Log) you'll run into issues where an event is too big to fit in a single UDP packet.

The problem with the CEF Add-On is the configuration of the syslog destination, do that using the GUI, and once you have your config statement okay, then copy that file to your indexers and run

index=main | eval _raw="t="._time."|h=".host."|st=".sourcetype."|i=".index."|s=".source."|r="._raw| cefout routing=diode

If you run like this you do not need to install the Diode Sender Add-On at all

0 Karma
Highlighted

Re: Matching specific index

New Member

Sorry for being unclear, by sending to the Diode Receiver Add-on I meant through the Diode Sender Add-on.

Perhaps jumbo frames will make sure that any event will fit in an outgoing UDP packet?

I have tried using the CEF Add-on but am a bit unsure of how to configure a data model to match any data in the main index and it seems to be necessary in order to configure a CEF output.
Is it possible to configure an output without the hassle of data models and data sets?

0 Karma
Highlighted

Re: Matching specific index

New Member

Jumbo frames may work, but I have never tried, and it should be supported by all assets the packets go through (the switches, routers, diodes, etc). Some diodes do compliance checks as well on IP packets. It will definitely need testing.

For the CEF Add-On, ignore all data model related items. The only thing you want to use the CEF Add-On for is the sending of the actual syslog packets, anything else like CEF or Data Models can be safely ignored. The only thing that matters is the definition of the output destination.

0 Karma
Highlighted

Re: Matching specific index

New Member

I managed to create an CEF output via the GUI and defining crap for data models, data sets etc but the output can only use TCP. Is it possible to reconfigure the output I created to use UDP instead?
I am using Splunk 8.0.3, Splunk Add-on for CEF output 2.3.0, Splunk App for CEF 2.3.0 and Diode Sender 1.3.5

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.