How to search splunkd.log to find tailing status of a file?

[philip_wong]

How can we search splunkd.log effectively to know tailing status of a file and when is the last read?

[philip_wong]

I think this is not the right answer. Almost nothing I can see after some noises filtered.

This is more close.

index=_internal component=WatchedFile

[jnicholsenernoc]

If you are using a heavy forwarder or indexing locally, you can query the _thefishbucket index to get run time information on the tailing of logs: “index=_thefishbucket"

Unfortunately, this isn't available on the univ/light forwarder.

It won't give you the exact timestamp of the last event exactly, but you can see the seek pointer and the last modified date and time. Based on this, you can see what it is doing (waiting for more log lines to be written), or processing a backlog, or stuck due to indexing being blocked and so forth.

More information:
http://blogs.splunk.com/2008/08/14/what-is-this-fishbucket-thing/

[yannK]

No this information is not logged in splunkd.log
Usually only the event of the first discovery of a file is listed, or when a batch processor finished to read a large file.
If you want the status of a file, use the REST-endpoint on the forwarder :
https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus

or use the file monitoring view in the SOS app on the local instance (if it has an UI)

Also you will never get the latest timestamp from the internal logs
For that you do a real time search on the indexed events, sort by _indextime and look at the most recent _time per source/host.

[philip_wong]

Yes, I just realized SoS's "File Monitor Inputs" page is doing that. I think it's prefect for if using indexer to read logs. But badly, it doesn't work with forwarders. I'm not sure yet.

Behind the scenes an customized command bundled with SoS doing good job to parse TailingProcessor REST response

| tpstatusquery targetserver= namespace="system" owner="nobody" admin/inputstatus/TailingProcessor:FileStatus | table *

[somesoni2]

You can use metadata command as well for getting the most recent event timestamp by host/source/sourcetype

[philip_wong]

This is a good start but not enough.
I wish to list all proceeded/processing logs by indexer/forwarder and the best also have the latest tailing timestamp

[somesoni2]

For starters, below can give all TailingProcessor activities including errors.

index=_internal component=TailingProcessor