topic Re: Old log files are not getting ingested into Splunk Cloud in Getting Data In

Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Mon, 09 Sep 2019 13:41:21 GMT

Hi Team,

We got an requirement to ingest the xyz.log from a client machine.

So i have created an app in the deployment master and deployed the same. The app has been successfully reached the client machine as well.

I have created an app and deployed the same on 8th Sep 2019 and the log file (xyz.log) has been lastly updated on 5th Sep 2019 in the client machine. Actually i believe the log file should be ingested into Splunk Cloud but here in this case its not getting ingested into Splunk Cloud.

So can i know what is the reason behind it and have enclosed my inputs.conf for reference. So kindly check and help on this.

[monitor:///abc/def/ijk/lmn/xyz.log]
sourcetype = pgr:stv
index = 123
disabled = 0

Kindly note the file has the splunk read permission and also in the internal logs it states that the configuration stanza as been parsed. The internal logs are reaching Splunk Cloud without any issues there is no connectivity issues as well.

But still i couldn't able to see the logs in Splunk Cloud.

Re: Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Mon, 09 Sep 2019 13:53:47 GMT

Kindly help on my request

Re: Old log files are not getting ingested into Splunk Cloud

tkomatsubara_sp — Mon, 09 Sep 2019 13:59:20 GMT

At least, you should check the message in the splunkd.log. What can you find?

Re: Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Mon, 09 Sep 2019 14:15:26 GMT

@tkomatsubara,

In splunkd.log the file is getting parsed refer below:

09-09-2019 05:20:30.415 -0500 INFO TailingProcessor - Parsing configuration stanza: monitor:///abc/def/ijk/lmn/xyz.log

But still the logs are not getting indexed. So can i know how Splunk works? Will it ingest old data as well.

Re: Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Mon, 09 Sep 2019 14:21:21 GMT

should i need to modify the inputs.conf stanza to ingest the old date logs. And the log date is on 5th Sep only. All seems to be fine but something it happens at the background and hence we couldn't able to ingest those logs.

Is this how Splunk works? Is it wont be able to ingest the old data logs kindly confirm please.

Re: Old log files are not getting ingested into Splunk Cloud

tkomatsubara_sp — Mon, 09 Sep 2019 14:24:31 GMT

There must be some errors. Can you find?

Re: Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Tue, 10 Sep 2019 07:31:39 GMT

There are no errors at all. Am i missing anything in the stanza. And one thing can you confirm is splunk can index the old date data as well.

Re: Old log files are not getting ingested into Splunk Cloud

anandhalagarasa — Tue, 10 Sep 2019 15:13:49 GMT

can anyone kindly help on my query.