Getting Data In

Forward data from logstash-forwarder to Splunk Indexer

yAlff
Path Finder

Hi all,

we have an ELK-cluster in our company and now we want to have the data, we have in ELK, as well in Splunk. Installing splunk Forwarders on the servers to monitor is not an option unfortunately.
We don't want to query Kibana from Splunk and import those results into Splunk; we want the raw logs.
Has anyboy of you experiences with configuring logstash-forwarders, to send the data as well to the Splunk Indexers?
We are thinking as well of shipping the logs to a file on a syslog-server where a UF can pick them up and forward them to the indexers - but maybe there is something more elegant?

Thanks,
Yannic

1 Solution

jtrucks
Splunk Employee
Splunk Employee

It is fairly common to use syslog to land on a syslog aggregation machine that runs a Splunk Heavy Forwarder setup. The Splunk instance on the machine reads all the syslog written files and forwards the data into the Splunk indexer tier. I suggest using this method as it is a well-known solution in the Splunk community and easier to troubleshoot and/or maintain by Splunk support and the Splunk user community.

--
Jesse Trucks
Minister of Magic

View solution in original post

jtrucks
Splunk Employee
Splunk Employee

It is fairly common to use syslog to land on a syslog aggregation machine that runs a Splunk Heavy Forwarder setup. The Splunk instance on the machine reads all the syslog written files and forwards the data into the Splunk indexer tier. I suggest using this method as it is a well-known solution in the Splunk community and easier to troubleshoot and/or maintain by Splunk support and the Splunk user community.

--
Jesse Trucks
Minister of Magic

martin_mueller
SplunkTrust
SplunkTrust

Indeed. Mandatory read for Splunk and syslog: http://www.georgestarcher.com/splunk-success-with-syslog/

0 Karma

davidleecrites
New Member

While this is a quick-and-dirty answer, it did not answer his need, nor would it answer mine.

Here is my situation:

I have a client who already has logstash everywhere on their system. Every box has a logstash forwarder, and they all point to the single logstash server. I want to provide a serious, and real-time "proof" that Splunk can do the job better. I don't have the luxury of installing splunk universal forwarder on every single production system in order to do this, and while a subset of them might show a bit of the power, getting everything will come much closer to being a slam dunk for the deal.

I cannot imagine that I am the first person who would like to siphon off the logstash forwarder data for use in Splunk, at least temporarily. So I'm thinking that SOMEONE "out there" has figured out how to pull it off. I'm not asking for a permanent solution, just a way to pull it off for a quick demo.

Once everyone is on-board, then the systematic replacement of the logstash forwarders will commence.

Thanks muchly,

DL

0 Karma

hcannon
Path Finder

It's been a while, but curious if you ever found a solution to this David, as its very similar to a use case I am currently looking into.

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...