Getting Data In

How do I setup an input for SQL Data

newkbi
Engager

I would like to create an input to ingest SQL data. I would also like a Dashboard to analyze the data I take into Splunk by extracting SQL information I find to be useful.

I'm a newbie to Splunk. I would like to solve the above problems. please help.

Tags (4)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

You've got several things to address with regard to this subject. The proper way to get sql data into splunk is via a scripted input. You'd set up a script to pull data from the database, and then have splunk eat that data. Information on that can be found here:

http://www.splunk.com/base/Documentation/latest/Developer/ScriptedInputsIntro

There are some sample scripts in splunk you can take a look at which are referenced in that document. Additionally, this may be useful to you:

http://www.splunk.com/wiki/Apps:DatabaseCollection

With regard to setting up a dashboard, the first thing you need to do is ensure that the fields are being extracted in the manner you expect. You can do that via index time field extractions:

http://www.splunk.com/base/Documentation/latest/Data/Configureindex-timefieldextraction

Once you have the fields extracted in the manner you desire, you'll need to develop a search with the data that you'd like to analyze over the dashboard. Once you have that search, you can finally move on to creating a dashboard.

http://www.splunk.com/base/Documentation/latest/Developer/DashboardIntro

What you are looking to do is completely feasible, though not a trivial task. Following the steps outlined above you should be able to achieve this goal.

View solution in original post

netgeek1983
Engager

SplunkMSE- Splunk searching with mysql - Step By Step Guide

http://wesecure.wordpress.com/2011/05/06/splunkmse/

jbsplunk
Splunk Employee
Splunk Employee

You've got several things to address with regard to this subject. The proper way to get sql data into splunk is via a scripted input. You'd set up a script to pull data from the database, and then have splunk eat that data. Information on that can be found here:

http://www.splunk.com/base/Documentation/latest/Developer/ScriptedInputsIntro

There are some sample scripts in splunk you can take a look at which are referenced in that document. Additionally, this may be useful to you:

http://www.splunk.com/wiki/Apps:DatabaseCollection

With regard to setting up a dashboard, the first thing you need to do is ensure that the fields are being extracted in the manner you expect. You can do that via index time field extractions:

http://www.splunk.com/base/Documentation/latest/Data/Configureindex-timefieldextraction

Once you have the fields extracted in the manner you desire, you'll need to develop a search with the data that you'd like to analyze over the dashboard. Once you have that search, you can finally move on to creating a dashboard.

http://www.splunk.com/base/Documentation/latest/Developer/DashboardIntro

What you are looking to do is completely feasible, though not a trivial task. Following the steps outlined above you should be able to achieve this goal.

xorred
Engager

Isn't DB Connect and Saved Searches replacing that? I already got saved sql searches (a query) to work at a specified time... now the question is how to set it as data input (specific columns of it, rows, etc or the whole data)

0 Karma

chris
Motivator

SPP (www.spp.at) is working on an database connector app. So there will be an easy way to connect to databases in the future

0 Karma

dps
Engager

To start I'd check out scripted inputs. You can probably use some sort of hard-coded service account with a read-only password to your tables and execute a query from a command line in some short perl or BASH shell script (i.e. mysql). Splunk will basically index the anything from the stdout stream. That will at least get your data in the index so that you can perform query operations against it.

http://www.cyberciti.biz/faq/run-sql-query-directly-on-the-command-line/

http://www.splunk.com/base/Documentation/latest/Developer/ScriptedInputsIntro

Dan

Brian_Osburn
Builder

Sorry, your question really doesn't make any sense..can you please rephrase this in a way that's understandable?

MuS
SplunkTrust
SplunkTrust

Yoda, is this you?

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...