Splunk Search

Data virtualization with Splunk?

barriersbill
Explorer

hi,
what are your thoughts on data virtualization and how does it apply to Splunk?
I ave been researching data virtualization solutions like presto, gemini, etc.. and so far I am not sure whether this is something that is recommended to have or use for data centralization and unification.
the main purpose of what I am trying to make is to combine data from all my silos whilst avoiding data copy and movement, making my Splunk deployments talk together with my other data lakes without any copy, or movement.
could this help be done, if so please how ?
-Bill

1 Solution

DavidHourani
Super Champion

Hi @barriersbill,

I agree with @niketnilay, it's possible to use db connect to add access to all your DBs via Splunk. That covers connections to relational databases. You can find the app here :
https://splunkbase.splunk.com/app/2686/

It does not however cover access to your data lakes, you will need other connectors for Hadoop for example :
https://splunkbase.splunk.com/app/3311/

If you're looking for a data unification solution that's outside to Splunk then yeah Gemini Data, Presto and others are doing Data virtualization and allow you to unify access to your data. You can use such solutions to centralize access to the data and have a view that allows you query data from multiple sources seamlessly without having to worry about where it resides.

To answer your question, I would say yes, you can use data virtualization to combine your Splunk silos together and even to connect them with other solutions such as ELK, Hadoop, etc... Absolutely recommended if you don't want to copy all your data to a data lake in order to avoid data movement and conserve a single source of truth.

Let me know if that helps.

Cheers,
David

View solution in original post

DavidHourani
Super Champion

Hi @barriersbill,

I agree with @niketnilay, it's possible to use db connect to add access to all your DBs via Splunk. That covers connections to relational databases. You can find the app here :
https://splunkbase.splunk.com/app/2686/

It does not however cover access to your data lakes, you will need other connectors for Hadoop for example :
https://splunkbase.splunk.com/app/3311/

If you're looking for a data unification solution that's outside to Splunk then yeah Gemini Data, Presto and others are doing Data virtualization and allow you to unify access to your data. You can use such solutions to centralize access to the data and have a view that allows you query data from multiple sources seamlessly without having to worry about where it resides.

To answer your question, I would say yes, you can use data virtualization to combine your Splunk silos together and even to connect them with other solutions such as ELK, Hadoop, etc... Absolutely recommended if you don't want to copy all your data to a data lake in order to avoid data movement and conserve a single source of truth.

Let me know if that helps.

Cheers,
David

niketn
Legend

@barriersbill one option would be to use Splunk DB Connect app with dbxlookup and/or dbxquery to only read the data from supported databases (https://docs.splunk.com/Documentation/DBX/latest/DeployDBX/Installdatabasedrivers)

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Get Updates on the Splunk Community!

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...