topic Montoring apache logs using splunk in Getting Data In

Montoring apache logs using splunk

splunker_123 — Fri, 10 Aug 2012 15:00:21 GMT

My requiremenent is to monitor day to day apache access logs and error logs through splunk
But the access logs are written as eg:ccess.123.10-08-2012 ,this will be gunzipped in the same location by log rotation script.I dont want to index the gunzip logs ,just I want the current logs
The challenge here is - the second numeric in the access log name will keep on changing and obviousuly the date as well.I meant this would be access.xxx.date

Is there a way I can give the above file name as input in splunk to monitor it on a daily basis?
I know if it had been access.log,then I can pass on the name in input file,but the file name change is dynamic.Is there a way to sort it out please?

Thanks

Re: Montoring apache logs using splunk

kristian_kolb — Mon, 13 Aug 2012 06:24:52 GMT

Yes, if you look at the documentation for inputs.conf you'll see that you can;

Specify a directory to monitor instead of a specific file -

[monitor:///var/log/httpd]

Set the sourcetype -

sourcetype=access_combined

here you can also limit what files to monitor through a blacklist -

blacklist = .gz

and if splunk should ignore older files

ignoreOlderThan = 7d

When searching, you can find all your logs through the sourcetype, regardless what the filename was.

Hope this helps,

Kristian

Re: Montoring apache logs using splunk

splunker_123 — Wed, 15 Aug 2012 12:38:47 GMT

Thank you so much ,it is working.
But I need to monitor both apache and plugin logs which is under same location.
At the moment my inputs.conf file looks like below

[monitor:///var/log/httpd]
sourcetype=access_combined
blacklist = .gz

[monitor:///var/log/httpd/http_plugin.log]

The issue is http_plugin.log is not getting indexed ,all the apache logs are indexed.Do I have to add anything else in inputs.conf please?

Re: Montoring apache logs using splunk

kristian_kolb — Mon, 28 Sep 2020 12:16:15 GMT

You should be aware that your [monitor:///var/log/httpd] will match the http_plugin.log as well and have the same sourcetype, i.e. access_combined.

Perhaps something like the following would work better.

[monitor:///var/log/httpd/access*] sourcetype=access_combined blacklist = .gz

[monitor:///var/log/httpd/http*]
sourcetype=http_plugin
blacklist = .gz

Re: Montoring apache logs using splunk

splunker_123 — Wed, 15 Aug 2012 16:40:04 GMT

Awesome.thankyou ..that worked like a charm
One last question...
when I try to view the logs through splunk web ,it reads, each line by line with space inbetween with numbers attached to each line.Can I make it to view as a single file for eg:assume I'm opening the same log file in textpad it will not have any space in between lines or numbers to it? Is it possible to display the log files in that fashion?

Re: Montoring apache logs using splunk

kristian_kolb — Wed, 15 Aug 2012 16:48:51 GMT

well, that's not really the point of indexing events, but you can at least have a partial likeness to the original file by clicking the little blue down-arrow next to an event and choose 'show source'.