topic Re: Question about Inputs.conf in Getting Data In

Question about Inputs.conf

zekiramhi — Wed, 25 Nov 2020 13:01:08 GMT

Hello,

I have made a new app under deployment apps with the following inputs.conf

[monitor:///root/something/something/something/something/] index = test whitelist=console-202[\S\s]+\.log$

whitelist is written to input filenames such as console-2020-06-02.log etc

I have not created any sourcetype for the index, so I do not have a props.conf file on the deployment app, neither on the searchheads. I have reloaded the server class that is linked to the host and app but I do not see any attempts to monitor the path I have given on the following spl query:

"index=_internal sourcetype=splunkd *something*"

Am I missing something on the inputs.conf? Am I forced to put a sourcetype? Cant I create my own custom sourcetpe via the gui or do I have to create a props and transforms conf for a sourcetype that does not exist?

Any help is appreciated,

Regards,

Re: Question about Inputs.conf

rnowitzki — Wed, 25 Nov 2020 13:14:42 GMT

Hi @zekiramhi,

Is the user that runs Splunk (I guess "splunk") able to read the files in the monitoring stanza?

Sourcetype is not mandatory (but recommended). Per Documentation:
"If not set, the indexer analyzes the data and chooses a source type."

BR
Ralph

Re: Question about Inputs.conf

zekiramhi — Wed, 25 Nov 2020 13:26:53 GMT

Hello,

Yes, I have given specific rights for the responsible user just as I have with my previous deployment app which is working.

Thanks,

Re: Question about Inputs.conf

rnowitzki — Wed, 25 Nov 2020 13:35:32 GMT

Is the app in the serverclass configured to restart the forwarder?

(just checking the easy/obvious stuff 🙂 )

Re: Question about Inputs.conf

richgalloway — Wed, 25 Nov 2020 13:48:44 GMT

Every input should have a sourcetype associated with it and every sourcetype should have a props.conf stanza. This keeps Splunk from having to guess about how to ingest your data and possibly getting it wrong. You can create a sourcetype in the UI at Settings->Source types->New Source Type.

When you created the new app did you specify the Restart Splunkd option? If not, then the inputs.conf has not taken effect.

Re: Question about Inputs.conf

zekiramhi — Wed, 25 Nov 2020 14:18:50 GMT

Maybe because I have never created a sourcetype for this index, is the reason it is not accepting to monitor this path. My main goal was to have Splunk ingest the data into Splunk and then create a Sourcetype for the incoming log on the index via the gui.

How I go about doing my sourcetype is the following:

1. Place the log sample on a test server via Add Data > Upload

2. Check if any of the pretrained sourcetypes produce a healthy result, which in this case they havent. So I proceeded to write my own regex for the key value pairs

3. Here lies the main question, now should I just copy the Avanced Settings "Copy to Clippboard" which I am pleased with how it has extracted the time and split the events as I want but the thing is it has set the sourcetype as [ __auto__learned__ ] which I dont think I should change for the events to extract the time automatically.

So now do I create a props.conf with [ __auto__learned__ ] and then reload the serverclass for the logs to flow? (if I go this path do I name the sourcetype to : [ __auto__learned__ ] in inputs.conf?) or can I just set the sourcetype to some dummy name in inputs.conf that does not exist in which I create via gui after the log arrives?

Apologies for the long explanation, but hopefully I have made myself clear

Regards,

Re: Question about Inputs.conf

zekiramhi — Wed, 25 Nov 2020 14:22:02 GMT

Yes, I have forgotten to do so but I have applied and reloaded the serverclass with no changes unfortunately 😕

Thank you for the suggestion 😁

Re: Question about Inputs.conf

richgalloway — Wed, 25 Nov 2020 14:27:09 GMT

Once you have used the Add Data wizard to process a sample data file and are happy with the results, click the Save As button to save your settings as a new sourcetype with a name you specify. Put that same name in the inputs.conf file.

Re: Question about Inputs.conf

zekiramhi — Wed, 25 Nov 2020 17:03:39 GMT

Hello,

I have done as you said, and do see the logs that I want being ingested via

INFO LicenseUsage - type=Usage.. Logs,

But I do not see the logs when I try to search for the index or sourcetype, is there anything I am supposed to check?

Re: Question about Inputs.conf

richgalloway — Wed, 25 Nov 2020 20:25:33 GMT

Re-run your INFO LicenseUsage - type=Usage.. Logs search in Verbose Mode. Check the index and sourcetype fields in the events returned. Use those values when you search by index or sourcetype.

Re: Question about Inputs.conf

zekiramhi — Thu, 26 Nov 2020 06:09:35 GMT

It was still in the process of injesting, after checking for all time I was able to see my logs. Many Thanks 😁