How do I configure custom sourcetypes on Universal Forwarders and Indexers?

[rob_lamb]

I have two Linux VMs set up, one with a Universal Forwarder and one with an Indexer. I have a script that generates dummy data (on the forwarder) that needs a custom sourcetype set up in order to parse the events correctly.

On the Universal Forwarder props.conf is currently empty, and inputs.conf contains:

[monitor:///home/splunk/data/data1*.soap]
_TCP_ROUTING = SOAP
disabled = false
sourcetype = soaptype

On the Indexer, props.conf contains:

[soaptype]
BREAK_ONLY_BEFORE = <SOAP-ENV:Envelope
TIME_PREFIX = <ns1:dateRequested>

As of right now my events aren't making it into the indexer at all. If I remove the sourcetype from inputs.conf and props.conf, data appears, but it is splitting the events incorrectly.

Any suggestions? Many thanks!

[skoelpin · Accepted solution]

Sounds like you have a few problems here.. First let's address the forwarder not sending data to the indexer.. Do you have the hostname specified in the outputs.conf on the forwarder? If not then this will solve your problem, don't forget to restart the Splunk service on the forwarder

Your next issue is defining the sourcetype so it remains constant. Go into the props.conf on the indexer and add this stanza then restart your Splunk service on the indexer. If you specify if your sourcetype in the props.conf then it will automatically default to what you specified when the data is being indexed

[soaptype]
 BREAK_ONLY_BEFORE = <SOAP-ENV:Envelope
 TIME_PREFIX = <ns1:dateRequested>
 sourcetype = soaptype

[rob_lamb]

I've added "sourcetype = soaptype" to my [soaptype] stanza on the indexer's props.conf per your note above, and restarted the indexer. I also cleaned up my outputs.conf file on my Universal Forwarder and restarted that.
When I dropped in a new file, the indexer picked it up and parsed it correctly. Thanks very much for the assistance!