allowing TCP input to define the sourcetype implicitly

[acidkewpie]

Hi, I'm looking to replicate the output from streamfwd with a separate script, and from what I understand of the Stream solution, it's entirely on the client to send fully formed messages to the server. But obviously streamfwd provides all the Proprietary metadata compared to just spitting a JSON structure into a TCP port which is what my script will need to do (I presume). A key thing to allow the data to be used as if it were actually from streamfwd is the setting of the sourcetype. How can I best allow these messages coming through a single TCP input to have a proper sourcetype as defined by the client? I believe it's possible for props.conf to mangle sourcetypes upon indexing, but I'd like to stay away from that if possible and let the client define it just like I believe streamfwd does. I noticed I could just put "sourcetype" as a field in the JSON payload, but this left the logs having two sourcetypes, the one set there AND "syslog" as defined on the TCP input itself.

[ndoshi]

You can use props.conf

[source::stream]
TRANSFORM-mytype=set_sourcetype

transforms.conf

[set_sourcetype]
REGEX=< insert your regex>
FORMAT= sourcetype::<name of your sourcetype>
DEST_KEY = MetaData:Sourcetype

See docs:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

[ndoshi]

If you use a heavy forwarder, yes you can apply transforms. I would not recommend a heavy forwarder unless you have a good reason as the Universal Forwarder scales better. The Heavy Forwarder sends events while the UF sends blocks of data at a time making it perform better..

[acidkewpie]

Well as above, this is a script sending data to a raw tcp input.

[acidkewpie]

Thanks, is there a way to do this on the client side though?