topic Re: Why am I unable to route syslog data to an index other than main? in Getting Data In

Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 15:41:27 GMT

I almost hesitate to ask this because I know the answer must be simple.

I have a small indexer clustering environment with a cluster master and two indexers. I am successfully receiving UDP:514 data, but it is being placed into the main index.

I have created an app, $splunkhome/etc/master_apps/syslogapp

Inside that, in the local directory, I have created the following inputs.conf:

[udp://514]
connection_host = ip
sourcetype = syslog
disabled = 0
index = poc

I pushed the configuration bundle successfully, however, syslog data is still being sent to the main index, not poc.

What am I missing?

Re: Why am I unable to route syslog data to an index other than main?

craigv_splunk — Tue, 28 Jun 2016 15:49:29 GMT

First I would try to ensure that the configuration is recognized on the indexer correctly. On the indexer. Run:
$SPLUNK_HOME/bin/splunk cmd btool inputs list.

In the resulting printout, do all of those configurations parameters show up in the result

If you have a ton of configuration you may want to run:
$SPLUNK_HOME/bin/splunk cmd btool --app=[your-app-name] inputs list.

Re: Why am I unable to route syslog data to an index other than main?

sjohnson_splunk — Tue, 28 Jun 2016 15:58:25 GMT

Does the data in the main index have the sourcetype = syslog?

Did you create the index = poc on all your indexers?

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 16:10:46 GMT

The index does exist on all indexers and is receiving data from UF's. Syslog data in main index does have the sourcetype of syslog.

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 29 Sep 2020 10:03:57 GMT

Looks good to me... [udp://514]
_rcvbuf = 1572864
allowSslCompression = true
allowSslRenegotiation = true
connection_host = dns
dedicatedIoThreads = 2
disabled = 0
enableSSL = 1
host = ln-mcl-vm-000-02
index = poc
maxSockets = 0
maxThreads = 0
port = 8088
sourcetype = syslog
sslVersions = *,-ssl2
useDeploymentServer = 0

Do you see anything wrong?

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 29 Sep 2020 10:04:03 GMT

Looks good to me...

[udp://514]
_rcvbuf = 1572864
allowSslCompression = true
allowSslRenegotiation = true
connection_host = dns
dedicatedIoThreads = 2
disabled = 0
enableSSL = 1
host = ln-mcl-vm-000-02
index = poc
maxSockets = 0
maxThreads = 0
port = 8088
sourcetype = syslog
sslVersions = *,-ssl2
useDeploymentServer = 0

See anything I'm missing?

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 29 Sep 2020 10:04:06 GMT

Looks ok to me, see anything wrong?

Re: Why am I unable to route syslog data to an index other than main?

craigv_splunk — Tue, 28 Jun 2016 17:12:32 GMT

Hmm. There is one thing I'm not sure about. It might be benign but it definitely strikes me as strange port and dedicatedIothread are both confiugrations associated with the http input not, to my knowledge, the udp port listener. This doesn't explain why its not indexing properly but it might lead to something.
Try running:
$SPLUNK_HOME/bin/splunk cmd btool inputs list --debug

This will show which file is creating that configuration entry. It might be helpful to see where that config is coming from

Re: Why am I unable to route syslog data to an index other than main?

sjohnson_splunk — Tue, 28 Jun 2016 17:59:49 GMT

Your inputs looks fine.

The other possibility is that there is a transform somewhere that is over-riding the index setting for either the source (udp://514) or the sourcetype (syslog). You should run the btool command on the props and look for a TRANSFORMS- statement associated with either source or sourcetype.

If you one, you will need to locate the app and edit the transforms.conf to fix

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 18:10:49 GMT

I just discovered something. Now, since I added this app and inputs, the UDP 514 data is not getting indexed into any index. Neither main nor poc.

Didn't find anything else with an associated transforms.

Re: Why am I unable to route syslog data to an index other than main?

sjohnson_splunk — Tue, 28 Jun 2016 18:15:36 GMT

If this is running on Linux, check your iptables. You could be blocking incoming traffic on that port.

On windows it could be windows firewall or your endpoint protection blocking.

Finally could also be some kind of network firewall rule.

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 18:16:24 GMT

I see nothing in the btool for those entries besides splunk_httpinput

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 18:17:21 GMT

Firewalls have all been disabled

Re: Why am I unable to route syslog data to an index other than main?

timmy13 — Tue, 28 Jun 2016 18:22:24 GMT

Firewalls are off

Re: Why am I unable to route syslog data to an index other than main?

sjohnson_splunk — Tue, 28 Jun 2016 18:27:10 GMT

Last guess is file permissions in the app dir or inputs.conf file.

Re: Why am I unable to route syslog data to an index other than main?

woodcock — Tue, 28 Jun 2016 20:39:10 GMT

Your directory path is wrong.

This is wrong:

$splunkhome/etc/master_apps/syslogapp

It should be this:

$SPLUNK_HOME/etc/syslogapp/default/

The put your inputs.conf, etc. there.
There is probably another problem, too. There is likely some other input already listening on that port. You need to find that first and disable that input.