Getting Data In

automatically forward splunk data to database

nielsenr
New Member

Ok so I am new to splunk and have an instance set up with logs from several servers feeding into it.
My question is can i get the data from splunk into a database(probably mysql) automatically.
There seems to be no clear explanation to say whether this is possible or not.

I have read a bit about the splunk forwarder would this be able to do it? once more there is no real explanation I can find.

Any suggestions would be helpful
thanks

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

A couple of approaches :

1) The Splunk MYSQL connector includes a search command, mysqloutput, that you can use to insert or update records in a table in a MySQL database based on fields resulting from your Splunk search.

2) you could use one of the Developer SDKs, write a custom program to execute a Splunk search , process the XML/JSON/CSV result and roll this up into a SQL statement to insert/update tables in your database.

3) you could write your own custom search command , and insert this at the end of you search pipeline to insert/update your DB tables with Splunk search fields ie: index=foo sourcetype=goo | stats count by host | myCustomOutputToDBCommand

View solution in original post

0 Karma

Damien_Dallimor
Ultra Champion

A couple of approaches :

1) The Splunk MYSQL connector includes a search command, mysqloutput, that you can use to insert or update records in a table in a MySQL database based on fields resulting from your Splunk search.

2) you could use one of the Developer SDKs, write a custom program to execute a Splunk search , process the XML/JSON/CSV result and roll this up into a SQL statement to insert/update tables in your database.

3) you could write your own custom search command , and insert this at the end of you search pipeline to insert/update your DB tables with Splunk search fields ie: index=foo sourcetype=goo | stats count by host | myCustomOutputToDBCommand

0 Karma

RohiniJindam
Path Finder

@nielsenr I am trying to achieve something similar to what you stated. Could you find a solution?

0 Karma

nielsenr
New Member

Thanks for the feedback
I have already looked at MYSQL connector and unless I'm mistaken it seems to only to be able to insert into a table data that has been searched form the database(please correct me if I'm wrong)

I am currently using the SDK's provided but was hoping for an easier solution (just being lazy)

umm, a custom search command never crossed my mind I'll have a look at that and see if it's useful.

Ill keep this updated with what I do. I have a feeling I'm not the first person to wonder about this, of course anymore suggestions would be welcome

0 Karma

Takajian
Builder

Splunk forwarder can forward data to splunk other instance or third party software like syslogd with text format. I do not think splunk forwarder can foward data to mysql. you will need to make script to geta data from splunk and put into the mysql.

0 Karma

nielsenr
New Member

Thanks for clearing that up

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...