Getting Data In

how can I use log filename to set "host" "date"

indeed_2000
Motivator

Hi I have log files like this:

log.machine03.20200310.bz2
log.machine04.20200310.bz2
log.machine05.20200310.bz2

these files copy each day to /opt , and splunk continuously monitor /opt.
now how can I use log filename to set "host" "date" of logs in splunk?

FYI: in every line of log file only store time NOT date! that's why I need to use date that exist in file name.
e.g.
file name = log.machine05.20200310.bz2
01:00:00 info logmessage
02:00:00 info logmessage
03:00:00 info logmessage
...

Thanks,

0 Karma

manjunathmeti
Champion

You can set host from file name in inputs.conf on forwarder.
inputs.conf

[monitor://<path>]
sourcetype = sourcetype_name
host_regex = [a-zA-Z]+\d+(?!$)

For setting _time from file name use INGET-EVAL. Add below configurations on indexer server.

props.conf

[sourcetype_name]
TRANSFORMS = timestamp_eval

trsansforms.conf

[timestamp_eval]
INGEST_EVAL = _time=strptime(replace(source, "/opt/log\.[\w]+\.", ""), "%Y%m%d.bz2")

Note: INGEST_EVAL is supported in splunk version >= 7.2.0.

0 Karma

indeed_2000
Motivator

Unfortunately I can’t use forwarders on Operation and logs copy manually to logserver.

0 Karma

manjunathmeti
Champion

You wrote splunk continuously monitor /opt. Which server is this? You need to put inputs.conf here.

0 Karma

indeed_2000
Motivator

On Splunk server I add /opt to index continuously.
Logs coming from other machines without Forwarders.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...