Splunk Search

Data forwarded to third party system missing metadata



I have an all in one enterprise splunk install (indexer, search head, file monitoring) with a number of universal forwarders forwarding data to it.

I need to forward everything I receive on the central splunk server to a third party system. I tried both the:

[syslog] (using TCP)
and the

approaches described in the doc.

The log data is forwarding fine.

The problem is that the metadata is missing. The source,sourcetype and host are missing when using tcpout and the source and sourcetype are missing when using syslog. Without them, the data will make no sense to the receiving system.

Is it possible to use a transform to put the metadata back in the data? If so, How?



0 Karma


Yes its missing unfortunately.


If you need to move data to another system with included metadata you might want to have a look at the dump command or the Hadoop Connect app. You can export to a mounted filesystem if you don't actually have an HDFS cluster running:


0 Karma


That's interesting, Thanks Jeremiah. I did not know about the dump command, but I don't think it helps here because I need to stream data, not move data.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!