Getting Data In

How to collect Apache logs in Splunk without a forwarder?

Ultra Champion

Someone just asked me an interesting question that I don't have the answer to...but I bet this community does 😉

Has anyone ever created an interesting way of getting apache logs off a server without actually installing a forwarder?

If it requires installing or running any script, then we might as well use a forwarder. But you never know if someone out there found some native way to send this log data into splunk using some other means than the forwarder.

Thanks for any ideas!

Engager

I have this issue too, but I am working on a python script that will send events to Splunk using the HTTP Event Collector. It's not finished yet but, I should get an initial version done this week. When it's done you can call the script via a cron job and it will read the latest logs and send them over to Splunk.

I have the first few commits here: https://github.com/alecdhuse/VPS-Log-Watch

0 Karma

Ultra Champion

I like that you're using the HTTP Event Collector but I've been burned before with custom scripts. If I was able to have anything run on the endpoint apache server, I would go with a forwarder (proven, small, "real-time"). Are you able to share the circumstances that motivated this effort?

0 Karma

Engager

I have a website on a shared VPS server. I do not manage the server and I do not have rights to install any software including the Splunk forwarder. However, I am allowed to run scripts and schedule cron jobs.

This seemed like an acceptable way to fill this gap. I am open to other methods of moving the data though.

0 Karma

Ultra Champion

Oh ok. I have a similar situation and so I used the tar (not rpm) version to place the splunk forwarder. Then I have a nightly cronjob that simply runs ./splunk start to makes sure it didn't get killed. So far so good and more stable than my poor programming.

I figured I'd share in case my similar scenario inspires something.

0 Karma

Engager

Thanks, that is probably a better solution.

0 Karma

Esteemed Legend
0 Karma

Contributor

NFS (Network File System)

0 Karma

Splunk Employee
Splunk Employee

or you can figure out how to use the fancy new HTTP Event Collector: http://blogs.splunk.com/2015/10/06/http-event-collector-your-direct-event-pipe-to-splunk-6-3/

0 Karma

Contributor

You can syslog the data out
or
You can have some kind of scp/scripted pull action from some centralized entity
or
you can write them to a local SQL/DB instance and use dbconnect to pull it
or
you can install the forwarder
:)