Can you help me with an inputs.conf wildcard issue?

patouellet — Thu, 20 Sep 2018 17:05:38 GMT

Hi,

I have a forwarder setup with this inputs.conf:

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*]
disabled = false
index = mq
sourcetype = qstats
crcSalt = <SOURCE>

[monitor:///home/mqm/mqstatistics/splunk/*_QM_CHL_*]
disabled = false
index = mq
sourcetype = chlstats
crcSalt = <SOURCE>

The location /home/mqm/mqstatistics/splunk/ has many files, here is a sample directory listing:

-rw-r--r---    1 mqm      mqm           30335 Sep 19 12:24 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-06.splunk
-rw-r--r---    1 mqm      mqm           29468 Sep 19 12:25 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-07.splunk
-rw-r--r---    1 mqm      mqm            5325 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-08.splunk
-rw-r--r---    1 mqm      mqm           10626 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-09.splunk
-rw-r--r---    1 mqm      mqm               0 Sep 19 13:18 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-10.splunk
-rw-r--r---    1 mqm      mqm           32233 Sep 19 13:19 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-11.splunk
-rw-r--r---    1 mqm      mqm           39100 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-12.splunk
-rw-r--r---    1 mqm      mqm           32861 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-13.splunk
-rw-r--r---    1 mqm      mqm           32758 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-14.splunk
-rw-r--r---    1 mqm      mqm            9269 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-15.splunk
-rw-r--r---    1 mqm      mqm           11222 Sep 19 13:22 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-16.splunk
-rw-r--r---    1 mqm      mqm           31818 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-17.splunk
-rw-r--r---    1 mqm      mqm           32847 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-18.splunk
-rw-r--r---    1 mqm      mqm          178561 Sep 19 12:24 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-06.splunk
-rw-r--r---    1 mqm      mqm          177300 Sep 19 12:25 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-07.splunk
-rw-r--r---    1 mqm      mqm          128417 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-08.splunk
-rw-r--r---    1 mqm      mqm          140852 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-09.splunk
-rw-r--r---    1 mqm      mqm               0 Sep 19 13:18 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-10.splunk
-rw-r--r---    1 mqm      mqm          181606 Sep 19 13:19 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-11.splunk
-rw-r--r---    1 mqm      mqm          195047 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-12.splunk
-rw-r--r---    1 mqm      mqm          183082 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-13.splunk
-rw-r--r---    1 mqm      mqm          181658 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-14.splunk
-rw-r--r---    1 mqm      mqm          136505 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-15.splunk
-rw-r--r---    1 mqm      mqm          140286 Sep 19 13:22 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-16.splunk
-rw-r--r---    1 mqm      mqm          181603 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-17.splunk
-rw-r--r---    1 mqm      mqm          181470 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-18.splunk

I confirm that I can read those files as the splunk ID. I also manually loaded a couple of those files in Splunk Enterprise and they look good.

Issue is: I'm not receiving any data. Everywhere I'm looking tells me I should be receiving data. The MQ index exists. There are no warning or errors in the logs. The forwarder reports this:

09-20-2018 12:46:49.014 -0400 INFO  TailingProcessor - Adding watch on path: /home/mqm/mqstatistics/splunk.
09-20-2018 12:46:49.014 -0400 INFO  TailingProcessor - Adding watch on path: /home/mqm/mqstatistics/splunk.
09-20-2018 12:46:49.013 -0400 INFO  TailingProcessor - Parsing configuration stanza: monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*.
09-20-2018 12:46:49.013 -0400 INFO  TailingProcessor - Parsing configuration stanza: monitor:///home/mqm/mqstatistics/splunk/*_QM_CHL_*.

I am receiving data from other sources for this Forwarder, just not this one. Why doesn't this inputs.conf work?

Thanks.

Re: Can you help me with an inputs.conf wildcard issue?

stcrispan — Thu, 20 Sep 2018 17:14:45 GMT

Do all your files end in .splunk?

My advice is to add that to your wildcard pattern. so instead of

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*]

Make it

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*.splunk]

Re: Can you help me with an inputs.conf wildcard issue?

patouellet — Tue, 29 Sep 2020 21:20:31 GMT

They all end with .splunk. Changed the input stanzas to QM_Q.splunk and QM_CHL.splunk. No dice.

Re: Can you help me with an inputs.conf wildcard issue?

nick405060 — Thu, 20 Sep 2018 17:22:30 GMT

I'm pretty sure inputs.conf treats * weird. Try using [\s\S] instead, or %

I had a problem with * in inputs.conf with a different issue:

https://answers.splunk.com/answers/671735/why-is-blacklisting-windows-event-logs-on-a-deploy-1.html

Re: Can you help me with an inputs.conf wildcard issue?

stcrispan — Thu, 20 Sep 2018 19:59:01 GMT

Have you tried it without CRC? or, in your CRSalt line, try putting quotes around the entire thing.

https://answers.splunk.com/answers/35210/crcsalt-issue.html

Re: Can you help me with an inputs.conf wildcard issue?

patouellet — Thu, 20 Sep 2018 20:27:18 GMT

Tried both. Still no dice. Thanks for the suggestion though.

Re: Can you help me with an inputs.conf wildcard issue?

patouellet — Thu, 20 Sep 2018 20:38:52 GMT

Have to thank Splunk Support for this one: the files were being ignored because they had a .splunk extension. Splunk ignores those as it thinks they are metadata files.

topic Can you help me with an inputs.conf wildcard issue? in Getting Data In

Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?

Re: Can you help me with an inputs.conf wildcard issue?