data from remote hosts

[pezcrap]

I wish to ingest events from a large number of remote hosts. I cannot install any Splunk infrastructure on these hosts.

I have looked into the various remote interfaces for splunk and none seem appropriate for my needs. It seems likely that I will need to build my own service to collect events from these hosts.

My question is: what is the best way to get data from my service into Splunk? I would like to be able to guarantee that once I have sent an 'ACK' to the remote host, that the data will make its way into splunk. I would also like to be able to scale the infrastructure horizontally.

I could have the server write to a monitored file, but I don't really want to create huge log files just to get data into Splunk.

I could use a FIFO queue, but that would not provide the guarantee I was talking about.

Perhaps I should use a Splunk SDK from within my service?

Can a splunk forwarder help here?

[yannK]

I cannot install any Splunk infrastructure on these hosts.

So you have to bring the logs to splunk somehow, to the indexer or to a forwarder.

• for log file, use a shared folder monitored remotely by forwarder
• a script to copy the files up to a forwarder
• use a syslog server to send the logs ( to another syslog that will write to disk, then monitor with splunk), but avoid UDP of course.

[pezcrap]

This doesn't really answer my question. I already noted I will need to implement my own service to collect events. I was asking how to go about implementing that service.

edit: Thanks a lot for responding though, I sounded a bit ungrateful there 🙂