Hi I have log files like this:
log.machine03.20200310.bz2
log.machine04.20200310.bz2
log.machine05.20200310.bz2
these files copy each day to /opt , and splunk continuously monitor /opt.
now how can I use log filename to set "host" "date" of logs in splunk?
FYI: in every line of log file only store time NOT date! that's why I need to use date that exist in file name.
e.g.
file name = log.machine05.20200310.bz2
01:00:00 info logmessage
02:00:00 info logmessage
03:00:00 info logmessage
...
Thanks,
You can set host from file name in inputs.conf on forwarder.
inputs.conf
[monitor://<path>]
sourcetype = sourcetype_name
host_regex = [a-zA-Z]+\d+(?!$)
For setting _time from file name use INGET-EVAL. Add below configurations on indexer server.
props.conf
[sourcetype_name]
TRANSFORMS = timestamp_eval
trsansforms.conf
[timestamp_eval]
INGEST_EVAL = _time=strptime(replace(source, "/opt/log\.[\w]+\.", ""), "%Y%m%d.bz2")
Note: INGEST_EVAL is supported in splunk version >= 7.2.0.
Unfortunately I can’t use forwarders on Operation and logs copy manually to logserver.
You wrote splunk continuously monitor /opt. Which server is this? You need to put inputs.conf here.
On Splunk server I add /opt to index continuously.
Logs coming from other machines without Forwarders.