Splunk DB connect to send data to sql database from Splunk Cloud

[santosh11]

Dear All,

I am trying to push data from splunk cloud to HF(heavy forwarder) and from there to sql DB. I am not able to send data from Splunk cloud to HF.

Do i need to raise splunk ticket to install splunk DB connect in search head?
Or is there any way we can send data from Splunk cloud to HF as splunk DB connect is already available there and from there we can push to SQL?

Please let me know how to achieve this. I have succeeded in pushing data from sql to splunk cloud but the other way is what is pending.

Regards,
Santosh

[santosh11]

Dear Giuseppe,

Thanks for the reply.

I want to send Splunk cloud data to HF where Splunk DB connect is installed and from there i was to push it to Oracle DB.

Please help me on this.

Regards,
Santosh

[gcusello · Accepted solution]

Hi santosh11,
let me understand: do you want to write results of a search on Splunk Cloud using DB-Connect?

To have the results of a search on Splunk Cloud you have to use a Search Head and then install on it DB-Connect.
This means that the SH must be local in your organization, I don't know if it's possible to use DB-Connect on Splunk Cloud but I don't do it, especially when you want to give write grants to it!
You could also use one of yours Heavy Forwarders (use always at least two HFs for High Availability!) but you have to analyze the load of this server (using e.g. the Monitoring Console) to be sure not overloading it.

Bye.
Giuseppe

[gcusello]

Hi santosh11,
As I said in my comment, to have the results of a search on Splunk Cloud you need to use a Search Head and then install on it DB-Connect.
So follow the following steps:

install in your network a Splunk Enterprise Server and configure it as a Search Head connected to Splunk Cloud,

on this SH, install DB-Connect,

configure it to access yout DB, giving the correct grants (I don't like to give to DB-Connect write grants on a DB!),

Create a Search on your SH to extract the data to insert in DB,

then in the same search insert at the end the dbxquery command with the db query to insert data in the database.

The SH must be local in your organization, I don't know if it's possible to use DB-Connect on Splunk Cloud but I don't do it, especially when you want to give write grants to it!

You could also use one of yours Heavy Forwarders (use always at least two HFs for High Availability!) but you have to analyze the load of this server (using e.g. the Monitoring Console) to be sure not overloading it.

About the problem of grants: I don't like to give to DB-Connect write grants on a DB, so I'd prefer to use a script or a program (e.g. in php) to extract data using REST API and then write them in the DB, but probably this is a workaround that I'd prefer!

Bye.
Giuseppe

[santosh11]

Thanks Giuseppe for the detailed explaination.

I understood now about the search head part and how can we proceed about it.

I have one doubt, You mentioned: "You could also use one of yours Heavy Forwarders (use always at least two HFs for High Availability!) but you have to analyze the load of this server (using e.g. the Monitoring Console) to be sure not overloading it."

Can you please explain it that how can i use HF?

Like do i need to do a API call from HF to splunk cloud for getting the data and send it to DB via DB connect?

Regards,
Santosh

[gcusello]

Hi santosh11,,
When you use Splunk Cloud, you can open the firewall routes between Splunk Cloud and all the target servers but it isn't a good idea for your security!
To avoid this, you can install on your network one (or more) Heavy Forwarder that concentrates the logs from all the Universal Forwarders and sends them to Splunk Cloud.
In this way you have to open firewall routes only between Splunk Cloud and Heavy Forwarders.
To do this, you have only to take in mind the following issues:

• you have to use al least two (or more) HFs to avoid Single Point of Failure;
• you have to properly add resources to HFs to manage the load;
• you have to correctly configure your HFs to avoid that they are the bottle neck of you network (e.g. the max log volume per second).

So when you have these HFs, you can use them also to take syslogs from appliances and install DB-Connect.
For DB-Connect, after you installed and configured HFs you can use the Monitoring Console to check if DB-Connect can run on the same servers or not.

If you don't use the approach of Search Head and prefer a program using REST API, you have to see in Splunk documentation how to do this, I'm not an expert.

Bye.
Giuseppe

[santosh11]

Dear Giuseppe,

Thanks for your reply. Sorry if my question was not clear.

We can do two things by DB Connect.

1) From Oracle DB --> HF( I have installed Splunk DB Connect here)--> Splunk Cloud (Indexer) (It is done).

2) From Splunk Cloud --> HF( I have installed Splunk DB Connect here)--> OracleDB (It is not working. and i need help in this)

How can i push a splunk query data to Oracle DB via splunk DB Connect?

Please guide me in this.

Regards,
Santosh