Getting Data In

How to collect Apache logs in Splunk without a forwarder?

sloshburch
Splunk Employee
Splunk Employee

Someone just asked me an interesting question that I don't have the answer to...but I bet this community does 😉

Has anyone ever created an interesting way of getting apache logs off a server without actually installing a forwarder?

If it requires installing or running any script, then we might as well use a forwarder. But you never know if someone out there found some native way to send this log data into splunk using some other means than the forwarder.

Thanks for any ideas!

dhuseau
Engager

I have this issue too, but I am working on a python script that will send events to Splunk using the HTTP Event Collector. It's not finished yet but, I should get an initial version done this week. When it's done you can call the script via a cron job and it will read the latest logs and send them over to Splunk.

I have the first few commits here: https://github.com/alecdhuse/VPS-Log-Watch

0 Karma

sloshburch
Splunk Employee
Splunk Employee

I like that you're using the HTTP Event Collector but I've been burned before with custom scripts. If I was able to have anything run on the endpoint apache server, I would go with a forwarder (proven, small, "real-time"). Are you able to share the circumstances that motivated this effort?

0 Karma

dhuseau
Engager

I have a website on a shared VPS server. I do not manage the server and I do not have rights to install any software including the Splunk forwarder. However, I am allowed to run scripts and schedule cron jobs.

This seemed like an acceptable way to fill this gap. I am open to other methods of moving the data though.

0 Karma

sloshburch
Splunk Employee
Splunk Employee

Oh ok. I have a similar situation and so I used the tar (not rpm) version to place the splunk forwarder. Then I have a nightly cronjob that simply runs ./splunk start to makes sure it didn't get killed. So far so good and more stable than my poor programming.

I figured I'd share in case my similar scenario inspires something.

0 Karma

VinayakJamadar
Loves-to-Learn Lots

Can you please explain a little bit more about this approach?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You're trying do dig out a thread from some 8 years ago. Most probably most of the participants are no longer actively following Answers.

Your best bet would be to create a new thread and desciribe your problem there (possibly providing a link to this one if your case is similar.

0 Karma

dhuseau
Engager

Thanks, that is probably a better solution.

0 Karma

woodcock
Esteemed Legend
0 Karma

hylam
Contributor

NFS (Network File System)

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

or you can figure out how to use the fancy new HTTP Event Collector: http://blogs.splunk.com/2015/10/06/http-event-collector-your-direct-event-pipe-to-splunk-6-3/

0 Karma

ltrand
Contributor

You can syslog the data out
or
You can have some kind of scp/scripted pull action from some centralized entity
or
you can write them to a local SQL/DB instance and use dbconnect to pull it
or
you can install the forwarder
:)

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...