auto discover web log directories

[Andre_]

Hello,

Is it possible to configure a Universal Forwarder to automatically discover the location of weblogs for IIS or Apache? I can programmatically get the locations and have a script for Windows and Linux that returns a list of locations. 

Kind Regards

Andre

[PickleRick]

I have a deja vu. I think I answered the same question recently.

But to the point.

1) There is no way to create an inputs with a dynamic definition using just Splunk built-in mechanisms.

2) It's hard to believe that you have a decently sized environment without any standarization. If you do, I strongly advise to get it cleaned up because otherwise it will bite you in the most inconvenient place at the most inconvenient time.

3) A very ugly way to try to go around it could be to define an "input" running your script which would generate inputs.conf dynamically but this would require bending over backwards to handle forwarder restarts. I would very strongly (as opposed to just "strongly" from previous point) advise against it.

[Andre_]

Hi @PickleRick ,

totally agree with you, some business models provide interesting challenges let's say.....

Can you use OS environment variables in the inputs.conf? If so, would they only be read on UF start up?

Cheers

Andre

[PickleRick]

Unfortunately, no.

[Andre_]

The only option would be an external mechanism to update the inputs.conf and reload the UF?
For example, have a scheduled task every hour that compares the inputs.conf and IIS configuration - if different, update inputs.conf and reload UF?

Kind Regards

Andre

[gcusello]

Hi @Andre_ ,

this is the mechanism used by the Deployment Server, so you can apply it, but it requires a restart of the local Splunk every time.

Are you sure that my hint isn't applicable?

Ciao.

Giuseppe

[Andre_]

Hi @gcusello ,

Unfortunately, I am not in control of the application layer. Web logs could be in any directory on any drive.

With a script to check and overwrite the inputs.conf, that will only require a local splunk restart if the location changes so. That usually happens rarely, but we need to capture it.

1. check log location

2. compare with current setting in inputs.conf

3. if different only - update & restart

Kind Regards

Andre

[gcusello]

Hi @Andre_ ,

ok, use your script, the locis seems to be correct.

Are you sure that it isn't possible to define a rule for IIS logs?

it seems very strange that your IIS are distributed without rules in all the filesystem, I suppose that they are in a predefined location and you could start from that location for your ingestion.

Ciao.

Giuseppe

[gcusello]

Hi @Andre_ ,

I don't think that's possible, also because Universal Forwarder's configurations are usually managed using a Deployment Server.

But, you could have very large inputs and take all the weblogs or Apache logs, e.g. if your Apache logs are in the folder /opt/apache/<app>/data/<other_folders>/apache.log

you could usi in your input:

[monitor:///opt/apache/.../*.log]

Ciao.

Giuseppe