Getting Data In

Montoring apache logs using splunk

splunker_123
Path Finder

Hi

My requiremenent is to monitor day to day apache access logs and error logs through splunk
But the access logs are written as eg:ccess.123.10-08-2012 ,this will be gunzipped in the same location by log rotation script.I dont want to index the gunzip logs ,just I want the current logs
The challenge here is - the second numeric in the access log name will keep on changing and obviousuly the date as well.I meant this would be access.xxx.date

Is there a way I can give the above file name as input in splunk to monitor it on a daily basis?
I know if it had been access.log,then I can pass on the name in input file,but the file name change is dynamic.Is there a way to sort it out please?

Thanks

1 Solution

kristian_kolb
Ultra Champion

Yes, if you look at the documentation for inputs.conf you'll see that you can;

Specify a directory to monitor instead of a specific file -

[monitor:///var/log/httpd]

Set the sourcetype -

sourcetype=access_combined

here you can also limit what files to monitor through a blacklist -

blacklist = .gz

and if splunk should ignore older files

ignoreOlderThan = 7d

When searching, you can find all your logs through the sourcetype, regardless what the filename was.

Hope this helps,

Kristian

View solution in original post

kristian_kolb
Ultra Champion

Yes, if you look at the documentation for inputs.conf you'll see that you can;

Specify a directory to monitor instead of a specific file -

[monitor:///var/log/httpd]

Set the sourcetype -

sourcetype=access_combined

here you can also limit what files to monitor through a blacklist -

blacklist = .gz

and if splunk should ignore older files

ignoreOlderThan = 7d

When searching, you can find all your logs through the sourcetype, regardless what the filename was.

Hope this helps,

Kristian

kristian_kolb
Ultra Champion

well, that's not really the point of indexing events, but you can at least have a partial likeness to the original file by clicking the little blue down-arrow next to an event and choose 'show source'.

/k

0 Karma

splunker_123
Path Finder

Awesome.thankyou ..that worked like a charm
One last question...
when I try to view the logs through splunk web ,it reads, each line by line with space inbetween with numbers attached to each line.Can I make it to view as a single file for eg:assume I'm opening the same log file in textpad it will not have any space in between lines or numbers to it? Is it possible to display the log files in that fashion?

0 Karma

kristian_kolb
Ultra Champion

You should be aware that your [monitor:///var/log/httpd] will match the http_plugin.log as well and have the same sourcetype, i.e. access_combined.

Perhaps something like the following would work better.


[monitor:///var/log/httpd/access*]
sourcetype=access_combined
blacklist = .gz

[monitor:///var/log/httpd/http*]
sourcetype=http_plugin
blacklist = .gz

0 Karma

splunker_123
Path Finder

Thank you so much ,it is working.
But I need to monitor both apache and plugin logs which is under same location.
At the moment my inputs.conf file looks like below

[monitor:///var/log/httpd]
sourcetype=access_combined
blacklist = .gz

[monitor:///var/log/httpd/http_plugin.log]

The issue is http_plugin.log is not getting indexed ,all the apache logs are indexed.Do I have to add anything else in inputs.conf please?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...