Getting Data In

How can I pull logs from a shared hosting account and get it into Splunk to index?

walderbachj1
Engager

The hosting provider is Rackspace Cloud Sites. In the root of each site is a logs dir, ex. somesite.com/logs. There are two different logs I want to grab each day:

1) ourlog_posts_YESTERDAYSDATE.csv
2) access_log_YESTERDAYSDATE.zip

Note I am unable to install a Universal Forwarder as this is essentially a shared hosting site. So I want to grab each log, each morning, and get it into Splunk to index.

  • I can set up cron jobs on the shared hosting site.
    • We have Linux and Windows servers which currently send logs to Splunk via forwarders, but I want these 2 logs to have a unique hostname/source so they don't get thrown in with any other servers logs. So any solution that pulls these logs via FTP to a location already being monitored, needs to have that addressed.

I'd think someone has come up against this problem before and I'd like to hear your solution.

0 Karma
1 Solution

jterry
Splunk Employee
Splunk Employee

Assuming your cron scripts/actions can get the files off the hosting system & on to a system that splunk has access to, you could configure a local dir input on the splunk server (or forwarder, etc) & associate the desired metadata to any data that's indexed from that source. The cron actions could then just put the data there.

View solution in original post

jterry
Splunk Employee
Splunk Employee

Assuming your cron scripts/actions can get the files off the hosting system & on to a system that splunk has access to, you could configure a local dir input on the splunk server (or forwarder, etc) & associate the desired metadata to any data that's indexed from that source. The cron actions could then just put the data there.

walderbachj1
Engager

That's my initial thought. But thought I'd solicit other ideas too.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...