How to collect Apache logs in Splunk without a forwarder?

[sloshburch]

Someone just asked me an interesting question that I don't have the answer to...but I bet this community does 😉

Has anyone ever created an interesting way of getting apache logs off a server without actually installing a forwarder?

If it requires installing or running any script, then we might as well use a forwarder. But you never know if someone out there found some native way to send this log data into splunk using some other means than the forwarder.

Thanks for any ideas!

[dhuseau]

I have this issue too, but I am working on a python script that will send events to Splunk using the HTTP Event Collector. It's not finished yet but, I should get an initial version done this week. When it's done you can call the script via a cron job and it will read the latest logs and send them over to Splunk.

I have the first few commits here: https://github.com/alecdhuse/VPS-Log-Watch

[sloshburch]

I like that you're using the HTTP Event Collector but I've been burned before with custom scripts. If I was able to have anything run on the endpoint apache server, I would go with a forwarder (proven, small, "real-time"). Are you able to share the circumstances that motivated this effort?

[dhuseau]

I have a website on a shared VPS server. I do not manage the server and I do not have rights to install any software including the Splunk forwarder. However, I am allowed to run scripts and schedule cron jobs.

This seemed like an acceptable way to fill this gap. I am open to other methods of moving the data though.

[sloshburch]

Oh ok. I have a similar situation and so I used the tar (not rpm) version to place the splunk forwarder. Then I have a nightly cronjob that simply runs ./splunk start to makes sure it didn't get killed. So far so good and more stable than my poor programming.

I figured I'd share in case my similar scenario inspires something.

[VinayakJamadar]

Can you please explain a little bit more about this approach?

[PickleRick]

You're trying do dig out a thread from some 8 years ago. Most probably most of the participants are no longer actively following Answers.

Your best bet would be to create a new thread and desciribe your problem there (possibly providing a link to this one if your case is similar.

[dhuseau]

Thanks, that is probably a better solution.

[woodcock]

You can use Stream and snoop the wire:

https://www.splunk.com/content/dam/splunk2/pdfs/fact-sheets/splunk-for-stream.pdf

[hylam]

NFS (Network File System)

[mreynov_splunk]

or you can figure out how to use the fancy new HTTP Event Collector: http://blogs.splunk.com/2015/10/06/http-event-collector-your-direct-event-pipe-to-splunk-6-3/

[ltrand]

You can syslog the data out
or
You can have some kind of scp/scripted pull action from some centralized entity
or
you can write them to a local SQL/DB instance and use dbconnect to pull it
or
you can install the forwarder
:)