- Find Answers
- :
- Splunk Administration
- :
- Monitoring Splunk
- :
- How to search splunkd.log to find tailing status o...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to search splunkd.log to find tailing status of a file?
How can we search splunkd.log effectively to know tailing status of a file and when is the last read?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think this is not the right answer. Almost nothing I can see after some noises filtered.
This is more close.
index=_internal component=WatchedFile
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you are using a heavy forwarder or indexing locally, you can query the _thefishbucket index to get run time information on the tailing of logs: “index=_thefishbucket"
Unfortunately, this isn't available on the univ/light forwarder.
It won't give you the exact timestamp of the last event exactly, but you can see the seek pointer and the last modified date and time. Based on this, you can see what it is doing (waiting for more log lines to be written), or processing a backlog, or stuck due to indexing being blocked and so forth.
More information:
http://blogs.splunk.com/2008/08/14/what-is-this-fishbucket-thing/
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No this information is not logged in splunkd.log
Usually only the event of the first discovery of a file is listed, or when a batch processor finished to read a large file.
If you want the status of a file, use the REST-endpoint on the forwarder :
https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus
or use the file monitoring view in the SOS app on the local instance (if it has an UI)
Also you will never get the latest timestamp from the internal logs
For that you do a real time search on the indexed events, sort by _indextime and look at the most recent _time per source/host.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, I just realized SoS's "File Monitor Inputs" page is doing that. I think it's prefect for if using indexer to read logs. But badly, it doesn't work with forwarders. I'm not sure yet.
Behind the scenes an customized command bundled with SoS doing good job to parse TailingProcessor REST response
| tpstatusquery targetserver=
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can use metadata command as well for getting the most recent event timestamp by host/source/sourcetype
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is a good start but not enough.
I wish to list all proceeded/processing logs by indexer/forwarder and the best also have the latest tailing timestamp
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For starters, below can give all TailingProcessor activities including errors.
index=_internal component=TailingProcessor
Enterprise Security Content Update (ESCU) | New Releases
Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?
Index This | What are the 12 Days of Splunk-mas?
