topic Log source with 2 deployment app send log from only 1 in Splunk Enterprise

Log source with 2 deployment app send log from only 1

SplunkExplorer — Wed, 17 Apr 2024 10:23:48 GMT

Hi Splunkers,

we have a Windows log source with a UF installed on it. We have no access to this log source: we only know that we collect Windows logs via UF and it works properly. Collected logs are the usual one: Security, Applications, and so on.
Starting from today, we need to add a monitor input: some files are stored in a folder and we need to collect them. So, on our DS, we created another app, inside deployment-app folder, with a proper inputs.conf and props.conf and then we deployed it.
Why we created another app and does not simply added a monitor stanza in inputs.conf for Windows addon? Simply because Windows addon is deployed on many host; on the other side, we need to monitor the path only on 1 specific host, so we preferred to deploy another dedicated app, with its server class and so on.

DS give no error; app is shown as deployed with no issues. At the same time, we got no error looking on splunkd.log and/or _internal index. By the way, logs are not collected.
For sure, we are going to reach Host owner and perform basic checks, like:

Is provided path the right one?
User in charge of execute UF has read permission on that folder?
In UF app folder, is the one deployed by us viewable?

But before this, there is a doubt I have: above point 2, in case of permission denied, I should see in _internal logs some error message, right? Because currently I don't see any error message related to this issue. The behavior is like the inputs.conf we set in deployment app is totally ignored: searching on _internl and/or splunkd.log, I cannot see anything related to path we have to monitor.

Re: Log source with 2 deployment app send log from only 1

scelikok — Wed, 17 Apr 2024 10:48:18 GMT

Hi @SplunkExplorer,

Did you check "Restart Splunkd" option for your new input app on app settings? Splunk Forwarder needs to be restarted for the new inputs.

Re: Log source with 2 deployment app send log from only 1

SplunkExplorer — Wed, 17 Apr 2024 10:53:03 GMT

Hi @scelikok yes, it's first check I performed; restart splunkd is correctly flagged

Re: Log source with 2 deployment app send log from only 1

scelikok — Wed, 17 Apr 2024 11:02:43 GMT

If there is a file read permission error you should have seen in _internal logs. You can check if app is installed on your host using below query;

index=_internal component=PackageDownloadRestHandler host=YourHost app=YourAppName

On my experience, most of the problems on this kind of blind configurations is given pathname or filename is wrong. And please remember file inputs are case-sensitive.

Re: Log source with 2 deployment app send log from only 1

SplunkExplorer — Wed, 17 Apr 2024 12:12:01 GMT

A very useful suggestion @scelikok, it is something new I learned. Thanks.

Executing this query I got some result; message says that "app bundle download has started and completed". The only think I don't know if it's right, is that host field is populated with DS hostname and not the log source one.

By the way, this lead me to agree with you about your last consideration: there must be some error in path/filename provided. We are going to check those parameter.

Re: Log source with 2 deployment app send log from only 1

scelikok — Wed, 17 Apr 2024 21:43:16 GMT

Hi @SplunkExplorer,

You are right that is Deployment Server log but it should show client ip address too. You can use below search to check deployment steps on client;

index=_internal host=YourClientHost sourcetype=splunkd (DeployedApplication OR ApplicationManager OR "Restarting Splunkd")

You should see similar events on regarding host logs;

INFO DeployedApplication - Checksum mismatch 0 <> 18281318892102154454 for app=your_app_name. Will reload from='x.x.x.x:8089/services/streams/deployment?name=default:your_serverclass_name:your_app_name' INFO DeployedApplication - Downloaded url=x.x.x.x:8089/services/streams/deployment?name=default:your_serverclass_name:your_app_name to file='C:\Program Files\SplunkUniversalForwarder\var\run\your_serverclass_name\your_app_name-1711990721.bundle' sizeKB=xx INFO DeployedApplication - Installing app=your_app_name to='C:\Program Files\SplunkUniversalForwarder\etc\apps\your_app_name' INFO ApplicationManager - Detected app creation:your_app_name WARN DC:DeploymentClient - Restarting Splunkd...

If everything seems ok on these log, we can think the problem is on provided path/filename.