Getting Data In

How can I import Apache log files?

lonwinters
New Member

A certain web host stopped offering AWStats with its hosting. Instead, they point you to Google Webmaster Tools. I used that for a while, until it stopped reporting the basic stats - unique visitors and hits. Google Analytics, as well as most other stats programs I've looked at are very complex and more than I need. I simply want to do the following, please provide directions. I installed Splunk, started it via the command line and logged in.

  1. Download the Apache log files from the web host
  2. Import log files into Splunk and view the reports.

And that's it. I chose Splunk because I wanted something I could run locally and not have to upload or install anything to the website. Thank you.

Tags (1)
0 Karma

grijhwani
Motivator

First you need some means of exporting the log files from the host to your local drive, then when you have sample files on your drive you need to go to Manager » Data inputs, click "Add data", select "From files and directories", then follow the guided dialogue.

lguinn2
Legend

You should go through the Splunk tutorial and also follow @yannK's advice

yannK
Splunk Employee
Splunk Employee

If you can access the server, install the forwarder on the server where the logs are.
Then setup the input to monitor the folders of the log file
and setup the forwarder to send the logs to the indexer (configured to accept remote data)

Otherwise, you will have to create a script to download the logs to the indexer and monitor them on the indexer.

Hint, for the apache logs, you probably want to specify the sourcetype=access_combined at the input level.

yannK
Splunk Employee
Splunk Employee

Look at @grijhwani 's answer.
Retrieve the log files, and put them in a folder that the indexer is monitoring.
Use the manager UI to create this monitor on the folder and specify the appropriate sourcetype (access_combined)

Then search for the events in Splunk, look for existing apps on splunkbase, or build your own dashboards.
You probably want to a high level report per day, and summarize or accelerate it.

0 Karma

lonwinters
New Member

First of all, I really really appreciate the fast and detailed responses. I did go through some of the documentation, but was hoping to avoid some of the configuration steps. I will go through the tutorial though.

I can download the Apache logs from the server, they are separate files for each day. All I really want to do is import these files into Splunk and view simple reports similar to AW stats. I would be OK if I could only view a day at a time, but it would be nice to look at the data in a monthly view.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...