Data virtualization with Splunk?

[barriersbill]

hi,
what are your thoughts on data virtualization and how does it apply to Splunk?
I ave been researching data virtualization solutions like presto, gemini, etc.. and so far I am not sure whether this is something that is recommended to have or use for data centralization and unification.
the main purpose of what I am trying to make is to combine data from all my silos whilst avoiding data copy and movement, making my Splunk deployments talk together with my other data lakes without any copy, or movement.
could this help be done, if so please how ?
-Bill

[DavidHourani · Accepted solution]

Hi @barriersbill,

I agree with @niketnilay, it's possible to use db connect to add access to all your DBs via Splunk. That covers connections to relational databases. You can find the app here :
https://splunkbase.splunk.com/app/2686/

It does not however cover access to your data lakes, you will need other connectors for Hadoop for example :
https://splunkbase.splunk.com/app/3311/

If you're looking for a data unification solution that's outside to Splunk then yeah Gemini Data, Presto and others are doing Data virtualization and allow you to unify access to your data. You can use such solutions to centralize access to the data and have a view that allows you query data from multiple sources seamlessly without having to worry about where it resides.

To answer your question, I would say yes, you can use data virtualization to combine your Splunk silos together and even to connect them with other solutions such as ELK, Hadoop, etc... Absolutely recommended if you don't want to copy all your data to a data lake in order to avoid data movement and conserve a single source of truth.

Let me know if that helps.

Cheers,
David

[niketn]

@barriersbill one option would be to use Splunk DB Connect app with dbxlookup and/or dbxquery to only read the data from supported databases (https://docs.splunk.com/Documentation/DBX/latest/DeployDBX/Installdatabasedrivers)