- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Showing warn message in index=_internal logs . Missiing some logs from forwarders
In the internal logs showing a warn message as below for particular servers. Is below are the reasons for missing some logs. If so how to troubleshoot this ?
- WARN TailingProcessor Insufficient Permissions to read . file /apps/logs/application.log.2017-2-2-44.log (hint:No such file or directory ).
2 . Info Thruputsprocessor the current maxkbps reached to max. Try to increase the maxkbps in limits.conf.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You need to do a chmod g+r
on those files and make sure that the user that is running the splunk process is a member of the group that the user is that owns the files.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
All the permissions are there for the file and directory. But still missing few logs .
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The text "Insufficient Permissions to read" is quite conclusive.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Where can we the see whether the logs are missing or not in splunk ?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
index=* source="*application.log.2017-2-2-44.log" | stats count by source
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe sharing your inputs.conf for this monitor input and the output of ls -alrt /apps/logs
would be helpful in making progress towards a resolution.
The error message contains "(hint:No such file or directory )" and there has to be a reason for that.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Actually on the server there are 4 logs but splunk is showing only 2 logs. where can we confirm in splunk about the missing logs ?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't understand why you need confirmation; the log is crystal clear. Splunk can see the log (enter the directory) but it cannot read it. If it cannot read it, of course the source's data will not be in Splunk.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even after giving complete permissions also seeing the same warn message. What could be the problem ?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe you are looking at old errors?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI arunsony,
first message is related to grants to access files in the target server, to monitor your file, you have to give to splunk additional grants.
the second one is related to a parameter to limit the thruput of Universal Forwarders and Heavy Forwarders towards the indexer.
from https://docs.splunk.com/Documentation/Splunk/6.6.2/Admin/Limitsconf
[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor in the ingestion pipeline to the specified rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer processes to the rate (in KBps) you specify.
* Note that this limit will be applied per ingestion pipeline. For more information about multiple ingestion pipelines see
parallelIngestionPipelines in the server.conf.spec file.
* With N parallel ingestion pipelines the thruput limit across all of the ingestion pipelines will be N * maxKBps.
* Default: 0 (unlimited)
Anyway, you don't lose logs for the maxKBps issue, you only receive logs slower, instead you don't ingest logs for the first problem.
Bye.
Giuseppe