All Apps and Add-ons

allowing TCP input to define the sourcetype implicitly

acidkewpie
Path Finder

Hi, I'm looking to replicate the output from streamfwd with a separate script, and from what I understand of the Stream solution, it's entirely on the client to send fully formed messages to the server. But obviously streamfwd provides all the Proprietary metadata compared to just spitting a JSON structure into a TCP port which is what my script will need to do (I presume). A key thing to allow the data to be used as if it were actually from streamfwd is the setting of the sourcetype. How can I best allow these messages coming through a single TCP input to have a proper sourcetype as defined by the client? I believe it's possible for props.conf to mangle sourcetypes upon indexing, but I'd like to stay away from that if possible and let the client define it just like I believe streamfwd does. I noticed I could just put "sourcetype" as a field in the JSON payload, but this left the logs having two sourcetypes, the one set there AND "syslog" as defined on the TCP input itself.

0 Karma

ndoshi
Splunk Employee
Splunk Employee

You can use props.conf

[source::stream]
TRANSFORM-mytype=set_sourcetype

transforms.conf

[set_sourcetype]
REGEX=< insert your regex>
FORMAT= sourcetype::<name of your sourcetype>
DEST_KEY = MetaData:Sourcetype

See docs:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

ndoshi
Splunk Employee
Splunk Employee

If you use a heavy forwarder, yes you can apply transforms. I would not recommend a heavy forwarder unless you have a good reason as the Universal Forwarder scales better. The Heavy Forwarder sends events while the UF sends blocks of data at a time making it perform better..

0 Karma

acidkewpie
Path Finder

Well as above, this is a script sending data to a raw tcp input.

0 Karma

acidkewpie
Path Finder

Thanks, is there a way to do this on the client side though?

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...