Getting Data In

automatically forward splunk data to database

nielsenr
New Member

Ok so I am new to splunk and have an instance set up with logs from several servers feeding into it.
My question is can i get the data from splunk into a database(probably mysql) automatically.
There seems to be no clear explanation to say whether this is possible or not.

I have read a bit about the splunk forwarder would this be able to do it? once more there is no real explanation I can find.

Any suggestions would be helpful
thanks

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

A couple of approaches :

1) The Splunk MYSQL connector includes a search command, mysqloutput, that you can use to insert or update records in a table in a MySQL database based on fields resulting from your Splunk search.

2) you could use one of the Developer SDKs, write a custom program to execute a Splunk search , process the XML/JSON/CSV result and roll this up into a SQL statement to insert/update tables in your database.

3) you could write your own custom search command , and insert this at the end of you search pipeline to insert/update your DB tables with Splunk search fields ie: index=foo sourcetype=goo | stats count by host | myCustomOutputToDBCommand

View solution in original post

0 Karma

Damien_Dallimor
Ultra Champion

A couple of approaches :

1) The Splunk MYSQL connector includes a search command, mysqloutput, that you can use to insert or update records in a table in a MySQL database based on fields resulting from your Splunk search.

2) you could use one of the Developer SDKs, write a custom program to execute a Splunk search , process the XML/JSON/CSV result and roll this up into a SQL statement to insert/update tables in your database.

3) you could write your own custom search command , and insert this at the end of you search pipeline to insert/update your DB tables with Splunk search fields ie: index=foo sourcetype=goo | stats count by host | myCustomOutputToDBCommand

0 Karma

RohiniJindam
Path Finder

@nielsenr I am trying to achieve something similar to what you stated. Could you find a solution?

0 Karma

nielsenr
New Member

Thanks for the feedback
I have already looked at MYSQL connector and unless I'm mistaken it seems to only to be able to insert into a table data that has been searched form the database(please correct me if I'm wrong)

I am currently using the SDK's provided but was hoping for an easier solution (just being lazy)

umm, a custom search command never crossed my mind I'll have a look at that and see if it's useful.

Ill keep this updated with what I do. I have a feeling I'm not the first person to wonder about this, of course anymore suggestions would be welcome

0 Karma

Takajian
Builder

Splunk forwarder can forward data to splunk other instance or third party software like syslogd with text format. I do not think splunk forwarder can foward data to mysql. you will need to make script to geta data from splunk and put into the mysql.

0 Karma

nielsenr
New Member

Thanks for clearing that up

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...