Splunk Search

Data virtualization with Splunk?

barriersbill
Explorer

hi,
what are your thoughts on data virtualization and how does it apply to Splunk?
I ave been researching data virtualization solutions like presto, gemini, etc.. and so far I am not sure whether this is something that is recommended to have or use for data centralization and unification.
the main purpose of what I am trying to make is to combine data from all my silos whilst avoiding data copy and movement, making my Splunk deployments talk together with my other data lakes without any copy, or movement.
could this help be done, if so please how ?
-Bill

1 Solution

DavidHourani
Super Champion

Hi @barriersbill,

I agree with @niketnilay, it's possible to use db connect to add access to all your DBs via Splunk. That covers connections to relational databases. You can find the app here :
https://splunkbase.splunk.com/app/2686/

It does not however cover access to your data lakes, you will need other connectors for Hadoop for example :
https://splunkbase.splunk.com/app/3311/

If you're looking for a data unification solution that's outside to Splunk then yeah Gemini Data, Presto and others are doing Data virtualization and allow you to unify access to your data. You can use such solutions to centralize access to the data and have a view that allows you query data from multiple sources seamlessly without having to worry about where it resides.

To answer your question, I would say yes, you can use data virtualization to combine your Splunk silos together and even to connect them with other solutions such as ELK, Hadoop, etc... Absolutely recommended if you don't want to copy all your data to a data lake in order to avoid data movement and conserve a single source of truth.

Let me know if that helps.

Cheers,
David

View solution in original post

DavidHourani
Super Champion

Hi @barriersbill,

I agree with @niketnilay, it's possible to use db connect to add access to all your DBs via Splunk. That covers connections to relational databases. You can find the app here :
https://splunkbase.splunk.com/app/2686/

It does not however cover access to your data lakes, you will need other connectors for Hadoop for example :
https://splunkbase.splunk.com/app/3311/

If you're looking for a data unification solution that's outside to Splunk then yeah Gemini Data, Presto and others are doing Data virtualization and allow you to unify access to your data. You can use such solutions to centralize access to the data and have a view that allows you query data from multiple sources seamlessly without having to worry about where it resides.

To answer your question, I would say yes, you can use data virtualization to combine your Splunk silos together and even to connect them with other solutions such as ELK, Hadoop, etc... Absolutely recommended if you don't want to copy all your data to a data lake in order to avoid data movement and conserve a single source of truth.

Let me know if that helps.

Cheers,
David

niketn
Legend

@barriersbill one option would be to use Splunk DB Connect app with dbxlookup and/or dbxquery to only read the data from supported databases (https://docs.splunk.com/Documentation/DBX/latest/DeployDBX/Installdatabasedrivers)

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...