Splunk Search

TailingProcessor File Status without port 8089?

vcarbona
Path Finder

We would like to have forwarders run as root in order to overcome file permissions. However, we also will be security hardening it as much as possible. One of these measures is to stop port 8089 on the forwarder. I assume this will not give us the ability to read the REST endpoint https://hostname:8089/services/admin/inputstatus/TailingProcessor%3AFileStatus. Are there any other ways to gather this data without the REST endpoint being available?

Tags (2)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You can call the endpoint directly from the CLI:

./splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus

View solution in original post

vcarbona
Path Finder

Actually "ignored file (crc conflict, needs crcSalt)" does appear, it just has a different error. I run it as "index=_internal component=TailingProcessor ERROR salt". It shows up as an ERROR alert level. The "File did not match whitelist ..." appears when I set TailingProcessor to DEBUG level.

0 Karma

vcarbona
Path Finder

Thanks! This actually helped us to identify some problem areas. But it appears conditions like "ignored file (crc conflict, needs crcSalt)" or "File did not match whitelist ..." do not show up.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You can call the endpoint directly from the CLI:

./splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus

vcarbona
Path Finder

I noticed the btprobe command shows some interesting data about the file status. It appears it is able to retrieve the modtime and seek pointer. Is it correct to assume that sptr (or seek pointer) is where the forwarder left off reading the file?

splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/messages

Using logging configuration at /opt/splunkforwarder/6.0.3-204106/etc/log-cmdline.cfg.
key=0xf4e82f9f021c429d scrc=0xc6e25d94afc02135 sptr=871 fcrc=0x452905a167cf4509 flen=0 mdtm=1404740503 wrtm=1404740504

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

So you've made port 8089 unavailable even from localhost? Then it might indeed be tough to call the REST API.

0 Karma

vcarbona
Path Finder

That would have been cool, but I get this:

ps -ef|grep splunk

splunk 5604 1 53 11:39 ? 00:00:13 splunkd -p 8089 restart
splunk 5605 5604 0 11:39 ? 00:00:00 [splunkd pid=5604] splunkd -p 8089 restart [process-runner]

./splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus
QUERYING: 'https://127.0.0.1:8089/services/admin/inputstatus/TailingProcessor:FileStatus'
This command [GET /services/admin/inputstatus/TailingProcessor:FileStatus] needs splunkd to be up, and splunkd is down.

0 Karma

somesoni2
Revered Legend

How about using search?
index=_internal component=TailingProcessor

0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...