Splunk Search

input.conf path has numbers how do i capture this?

zergid
New Member

our log path looks like this

/var/www/webapp/application/logs/2014/09/13/03.log

where 2014 is the year, 09 is the month, 13 is the day, and 03 is the hour.

How can i capture this path pattern in input.conf so all auto generated starting with the year, month, day, hour are captured and the logs are sent to splunkstorm index?

0 Karma

kml_uvce
Builder

Use regex under stanza

[monitor:///var/www/webapp/logs]
whitelist=\/var\/www\/webapp\/application\/logs\/\d{4}\/\d{2}\/\d{2}.log

Please change regex if it does not work 🙂

jrodman
Splunk Employee
Splunk Employee

This will definitely limit the stanza to only match filenames like that (though I recommend anchoring the regex with ^ and $, but it won't make the numbers available elsewhere.

0 Karma

jrodman
Splunk Employee
Splunk Employee

EDIT: I may have misunderstood your goal, and perahaps the other answer is the one you want.
If you just want to index those files, a wildcards or regex whitelist will do the job.

If you want to find out the times from the path, the rest of my answer is relevant.


Splunk will attempt to guess the date from the filename first by TIME_FORMAT and then falling back to regexes as an initial seed/guess value before running the time extraction per-event logic. In other words the filename can influence timestamping.

However, I'm unclear whether the full path is passed into this logic. I think it is not.

The remaining options are:

  • ensure the modtime is accurate. Splunk will use the modtime as a guide for the data, so the pathname may be unnecessary.
  • Put the date into the filename so that the filename logic can work
  • Put timestamps in the file

Timestamps in the file is definitely the best outcome, but it might not be an availble choice to you.

kristian_kolb
Ultra Champion

have you looked at the wildcard characters? Either of the following should work - take a look at the docs for inputs.conf in the Search Reference manual.

[monitor:///var/www/webapp/logs/.../*.log]
[monitor:///var/www/webapp/logs/2014/*/*/*.log]

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...