Getting Data In
Highlighted

How do I Configure the Universal Forwarder to send UnCooked Data?

Communicator

We have a Universal Forwarder on a Domain Controller (DC) that is forwarding all the local logs to a 4.1.7 Forwarder. The 4.1.7 Forwarder is then sending the logs to an Indexer, as well as an IDS via syslog. This 4.1.7 Forwarder is also collecting tons of WMI logs which are being observed on both the Indexer and IDS.

We are seeing the DC logs come across to the indexer however we are not seeing any of the DC logs go to the IDS. As the Universal Forwarder sends cooked data I tried setting cooked data to false:

This is the Universal Forwarder config. /etc/system/local/outputs.conf

[tcpout]
defaultGroup = splunk02..._9998
disabled = false
indexAndForward = 0

[tcpout:splunk02..._9998]
server = splunk02...:9998

[tcpout-server://splunk02...:9998]
sendCookedData=false

After this change I was still able to observe DC logs on the Indexer however none on the IDS. For troubleshooting purposes i installed a LightForwarder on the DC and was able to see DC logs on both the Indexer and the IDS. This leads me to believe that the data is getting cooked by the Universal Forwarder. Anyone have any ideas on how to make the Universal Forwarder send data unCooked or see what im doing wrong here?

Here is the config on the 4.1.7 Forwarder /etc/system/local

outputs.conf

[tcpout]
disabled = false
indexAndForward = false

[syslog:my_syslog_group]
disabled = false
server = 10.x.x.x:514
type = udp
sendCookedData = false

props.conf

[host::*]
DATETIME_CONFIG = NONE
TRANSFORMS-ROUTING = send_to

transforms.conf

[send_to_AG]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = my_syslog_group

Thanks, I-Man

0 Karma
Highlighted

Re: How do I Configure the Universal Forwarder to send UnCooked Data?

Splunk Employee
Splunk Employee

Hi I-Man

Have you tried the following stanza in outputs.conf?

[tcpout-server://splunk02...:9998]
server = 10.255.4.213:514
sendCookedData=false

I am just basing that off the documentation for forwarding data found here. I am not entirely certain how you want to send data but if its a subset of syslog data then you may be interested in the section near the end (found here)

Highlighted

Re: How do I Configure the Universal Forwarder to send UnCooked Data?

Communicator

No luck but thanks for the suggestion. I spoke with Splunk support regarding this and their best guess was that there is an issue with the 4.2 UF sending to the 4.1.7 forwarder. They suggested that I upgrade the forwarders when 4.2.1 is available in a week or two and see if that works.

0 Karma
Highlighted

Re: How do I Configure the Universal Forwarder to send UnCooked Data?

Communicator

An upgrade resolved the issue.

View solution in original post

0 Karma
Highlighted

Re: How do I Configure the Universal Forwarder to send UnCooked Data?

Splunk Employee
Splunk Employee

UDP forwarding is not supported on the Universal Forwarder. It's allowed only on the Heavy Forwarder.

From the documentation: You can configure a heavy forwarder to send data in standard syslog format. The forwarder sends the data through a separate output processor. You can also filter the data with props.conf and transforms.conf. You'll need to specify SYSLOGROUTING as the DEST_KEY.

Note: The syslog output processor is not available for universal or light forwarders.

The syslog output processor sends RFC 3164 compliant events to a TCP/UDP-based server and port, making the payload of any non-compliant data RFC 3164 compliant. Yes, that means Windows event logs!

To forward syslog data, identify the third-party receiving server and specify it in a syslog target group in the forwarder's outputs.conf file.

Note: If you have defined multiple event types for syslog data, the event type names must all include the string "syslog".

Forward syslog data
In outputs.conf, specify the syslog target group:

[syslog:]
=
=
...

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Forwarddatatothird-partysystemsd

0 Karma