Getting Data In
Highlighted

How can I import Apache log files?

New Member

A certain web host stopped offering AWStats with its hosting. Instead, they point you to Google Webmaster Tools. I used that for a while, until it stopped reporting the basic stats - unique visitors and hits. Google Analytics, as well as most other stats programs I've looked at are very complex and more than I need. I simply want to do the following, please provide directions. I installed Splunk, started it via the command line and logged in.

  1. Download the Apache log files from the web host
  2. Import log files into Splunk and view the reports.

And that's it. I chose Splunk because I wanted something I could run locally and not have to upload or install anything to the website. Thank you.

Tags (1)
0 Karma
Highlighted

Re: How can I import Apache log files?

Splunk Employee
Splunk Employee

If you can access the server, install the forwarder on the server where the logs are.
Then setup the input to monitor the folders of the log file
and setup the forwarder to send the logs to the indexer (configured to accept remote data)

Otherwise, you will have to create a script to download the logs to the indexer and monitor them on the indexer.

Hint, for the apache logs, you probably want to specify the sourcetype=access_combined at the input level.

Highlighted

Re: How can I import Apache log files?

New Member

First of all, I really really appreciate the fast and detailed responses. I did go through some of the documentation, but was hoping to avoid some of the configuration steps. I will go through the tutorial though.

I can download the Apache logs from the server, they are separate files for each day. All I really want to do is import these files into Splunk and view simple reports similar to AW stats. I would be OK if I could only view a day at a time, but it would be nice to look at the data in a monthly view.

0 Karma
Highlighted

Re: How can I import Apache log files?

Splunk Employee
Splunk Employee

Look at @grijhwani 's answer.
Retrieve the log files, and put them in a folder that the indexer is monitoring.
Use the manager UI to create this monitor on the folder and specify the appropriate sourcetype (access_combined)

Then search for the events in Splunk, look for existing apps on splunkbase, or build your own dashboards.
You probably want to a high level report per day, and summarize or accelerate it.

0 Karma
Highlighted

Re: How can I import Apache log files?

Legend

You should go through the Splunk tutorial and also follow @yannK's advice

Highlighted

Re: How can I import Apache log files?

Motivator

First you need some means of exporting the log files from the host to your local drive, then when you have sample files on your drive you need to go to Manager » Data inputs, click "Add data", select "From files and directories", then follow the guided dialogue.