Sending data out and keeping metadata

[winter4]

Hi Team, 

I am looking for a way to forward data from my heavy forwarders to a different source while maintaining the metadata like (host, source, sourcetype) 

I have tried using the tcpout config in outputs.conf but I do not see the metadata being transferred. 
syslog config in outputs.conf does not work for me either. 

[winter4]

Hi @gcusello, 

I am trying to forward the logs to both splunk and an external system via syslog. 

Correct, I want to forward the logs coming into my HF to the external 3rd party syslog and maintain the metadata associated with the logs. 

[gcusello]

Hi @winter4 ,

metadata are associated to Splunk, so you can maintain them only in Splunk, you cannot maintain them in a syslog to an external third party.

So, your Indexer will receive logs with metadata, instead the third party will receive logs without metadata.

About metadata:

• sourcetype is a metadata of Splunk so it isn't relevant for a third party.
• host is usually present at the beginning of the syslog and the third party should only extract it.
• source is a metadata that you lose sending syslogs to a third party.

Ciao.

Giuseppe

[PickleRick]

You can't easily do that. I'm not even sure you can to that at all.

The problem is that the data being sent over the syslog output is simply the raw event, optionally(?) prepended by the syslog header. So if you wanted to include the metadata you'd have to include it in the raw event. But even if you managed to do this on a global level (like some catch-all sourcetype definition and a transform adding the metadata to the event), the same event would be sent to your splunktcp output as well which would most probably mean that the event is unusable in this format.

[winter4]

Hi @PickleRick ,

Do you know if there is any possible method of sending data from a splunk HF to a 3rd party endpoint that also maintains the metadata

[PickleRick]

Using just Splunk, you could do an ugly hack and send to another HF instance on which you'd force input data to go through typing queue again, not skip straight to indexing queue. But this is a very unusual and unintuitive design. You might be able to use Cribl but I'm not sure about that.

[winter4]

Hi @PickleRick 

Do you have any documentation detailing the hack that you are thinking of. 

Or do you have a sample of the configs I can input on the HF to get splunk to send that data. Any help will be greatly appreciated and serve as a good starting point. 
Thanks!

[PickleRick]

I don't think it's actually docummented anywhere since it's not normally meant for users to fiddle with.

And I would strongly advise against trying to do that. I'd probably not want to do such thing myself in production environment. In a lab setup just for fun and to see how stuff works - sure, why not. But in prod? Hell, no.

It's not about HF _sending_ data. It's about re-parsing incoming already parsed data (and additionally, this particular HF would need to actually _not_ send data anywhere else, just export it to syslog; it's actually a waste of resources I think).

[winter4]

Is there any other way outside of using syslog? either way thanks for all of the inputs @PickleRick @gcusello. Just an exercise on my part. 

[gcusello]

Hi @winter4 ,

a question: do you want to forward data to an Indexer ot to an external system via syslog?

I suppose that you are meaning that you want to forward logs, that you are receiving from UFs or syslogs or HEC, using a HF, maintaining the original host source and sourcetype.

What's your issue?

if you're sending to an Indexer, you have to use outputs.conf and source, host and sourcetype, by default aren't overwritten and usually remain the original ones, unless you configure overwritting.

If instead your have to send to a third party it's different.

Ciao.

Giuseppe