Getting Data In
Highlighted

How to use a portion of the hostname in your inputs.conf monitor path?

Communicator

Got a bunch of logs to pickup from different machines. Evidently each machine has a share to the other machines, so I need to only pickup the log in the directory matching with the actual host name. How can I get the last 4 digits from the host name, and use that in my inputs.conf to pickup only the log files in that subdirectory? Tried different combinations of host_segment but not getting anything that works.

Server Name (Log Path)
abcserver1001 (/opt/log/AppName1001/server.log)
abcserver1002 (/opt/log/AppName
1002/server.log)
abcserver1003 (/opt/log/AppName1003/server.log)
abcserver1004 (/opt/log/AppName
1004/server.log)
abcserver1005 (/opt/log/AppName_1005/server.log)

Thanks in advance!!! Joe

0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Esteemed Legend

The easiest will be to use host_segment=3 in inputs.conf and then use a traditional method of host override (google it) or possibly even INGEST_EVAL-host=replace(host, "^[^_]+_", "abcserver") on the Indexers. The first part in inputs.conf (for server abcserver1001) is like this:

[monitor:///opt/log/AppName_*/server.log]
index = middleware
host_segment = 3
sourcetype = jboss:server:log
disabled = 0
0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Communicator

Checking this method out now (I googled it!).

0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Communicator

Still not able to figure out these method(s).

INPUTS.CONF (for server abcserver1001)
[monitor:///opt/log/AppName_(how to dynamically set here based on last 4 of server name)/server.log]
index = middleware
sourcetype = jboss:server:log
disabled = 0

Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Esteemed Legend

See my updated answer.

0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Explorer

Have you tried the host_segment = 3?
If you only want the last four digits you will need to use transforms and props in indexer or HF so you can reassign and extract the host field.

Are you doing this by Splunk UF -> Splunk Indexer ?

sample tranforms.conf
[overridinghost]
REGEX = AppName_(?<host>[0-9]+)
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf
[sourcetype or the source]
TRANSFORMS-serverhost=overridinghost

0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Communicator

Not using heavy forwarders for this input (just sending from UF to index cluster) so can implement on the indexer cluster. Still not understanding how changes on that tier would effect the inputs.conf of the monitor.

0 Karma
Highlighted

Re: How to use a portion of the hostname in your inputs.conf monitor path?

Communicator

Doing this transform on the Indexer tier would be AFTER the monitor has collected the data. If the path to the correct log is wrong in the monitor, i don't see how that could work.

I need to set the monitor line of the inputs.conf with some intelligence, to only get the last 4 digits of the host, then use that value in the monitor line.

Or are you saying to include the props.conf and transforms.conf with the app I push to the forwarder? My understanding is that the props/transforms portion comes in AFTER the monitor of the inputs.conf... that is too late for what I'm trying to do.

0 Karma