Getting Data In

Question about Inputs.conf

zekiramhi
Path Finder

Hello,

I have made a new app under deployment apps with the following inputs.conf

 

 

[monitor:///root/something/something/something/something/]
index = test
whitelist=console-202[\S\s]+\.log$

 

 

whitelist is written to input filenames such as console-2020-06-02.log etc

 I have not created any sourcetype for the index, so I do not have a props.conf file on the deployment app, neither on the searchheads. I have reloaded the server class that is linked to the host and app but I do not see any attempts to monitor the path I have given on the following spl query:

"index=_internal sourcetype=splunkd *something*"

Am I missing something on the inputs.conf? Am I forced to put a sourcetype? Cant I create my own custom sourcetpe via the gui or do I have to create a props and transforms conf for a sourcetype that does not exist?

Any help is appreciated,

Regards,

Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Once you have used the Add Data wizard to process a sample data file and are happy with the results, click the Save As button to save your settings as a new sourcetype with a name you specify.  Put that same name in the inputs.conf file.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Every input should have a sourcetype associated with it and every sourcetype should have a props.conf stanza.  This keeps Splunk from having to guess about how to ingest your data and possibly getting it wrong.  You can create a sourcetype in the UI at Settings->Source types->New Source Type.

When you created the new app did you specify the Restart Splunkd option?  If not, then the inputs.conf has not taken effect.

---
If this reply helps you, Karma would be appreciated.
0 Karma

zekiramhi
Path Finder

Maybe because I have never created a sourcetype for this index, is the reason it is not accepting to monitor this path. My main goal was to have Splunk ingest the data into Splunk and then create a Sourcetype for the incoming log on the index via the gui.

How I go about doing my sourcetype is the following:

1. Place the log sample on a test server via Add Data > Upload 

2. Check if any of the pretrained sourcetypes produce a healthy result, which in this case they havent. So I proceeded to write my own regex for the key value pairs

3. Here lies the main question, now should I just copy the Avanced Settings "Copy to Clippboard" which I am pleased with how it has extracted the time and split the events as I want but the thing is it has set the sourcetype as [ __auto__learned__ ]  which I dont think I should change for the events to extract the time automatically.

So now do I create a props.conf with  [ __auto__learned__ ] and then reload the serverclass for the logs to flow? (if I go this path do I name the sourcetype to : [ __auto__learned__ ] in inputs.conf?) or can I just set the sourcetype to some dummy name in inputs.conf that does not exist in which I create via gui after the log arrives?

Apologies for the long explanation, but hopefully I have made myself clear

Regards,

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Once you have used the Add Data wizard to process a sample data file and are happy with the results, click the Save As button to save your settings as a new sourcetype with a name you specify.  Put that same name in the inputs.conf file.

---
If this reply helps you, Karma would be appreciated.
0 Karma

zekiramhi
Path Finder

Hello,

I have done as you said, and do see the logs that I want being ingested via

INFO LicenseUsage - type=Usage.. Logs,

But I do not see the logs when I try to search for the index or sourcetype, is there anything I am supposed to check?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Re-run your INFO LicenseUsage - type=Usage.. Logs search in Verbose Mode.  Check the index and sourcetype fields in the events returned.  Use those values when you search by index or sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma

zekiramhi
Path Finder

It was still in the process of injesting, after checking for all time I was able to see my logs. Many Thanks 😁

0 Karma

rnowitzki
Builder

Hi @zekiramhi,

Is the user that runs Splunk (I guess "splunk") able to read the files in the monitoring stanza?

Sourcetype is not mandatory (but recommended). Per Documentation: 
"If not set, the indexer analyzes the data and chooses a source type."

BR
Ralph

--
Karma and/or Solution tagging appreciated.
0 Karma

zekiramhi
Path Finder

Hello,

Yes, I have given specific rights for the responsible user just as I have with my previous deployment app which is working.

Thanks,

0 Karma

rnowitzki
Builder

Is the app in the serverclass configured to restart the forwarder?

(just checking the easy/obvious stuff 🙂 )

--
Karma and/or Solution tagging appreciated.
0 Karma

zekiramhi
Path Finder

Yes, I have forgotten to do so but I have applied and reloaded the serverclass with no changes unfortunately 😕

Thank you for the suggestion 😁

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...