All Apps and Add-ons

MongoDb data indexing

sssid
New Member

I am planning to use splunk for visualizing my data and could use some help modelling it .

  • i have some json data stored in mongodb that i need to access in splunk .
  • Data reaches upto 1 TB per day .
  • I need to fetch, process and index this data so it can be viewed in splunk.

My question is with the fetching the data . What is the best way to fetch this data periodically ? I need to fetch data from mongodb for a particular time frame . Ie this is not real time streaming data .

After I specify data sources using hunk , how do i index the data for specific time ranges ?

Should i create an intermediary application (node.js or java) just for fetching and indexing the data periodically ?

0 Karma

puneethgowda
Communicator

But what are the configuration needed to bring the data into Splunk from mangodb

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

You can either use the Hunk App for MongoDB or DB Connect with JDBC to MongoDB.
With Hunk App for MongoDB:
in the search use: index=mongoVIX | collect my-local-splunk-index
or
set a schedule search and send the data to my-local-splunk-index

With DB Connect:
Setup Database Input based on rising column or batch. In both cases the data will be sent to a local Splunk index.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Build the Future of Agentic AI: Join the Splunk Agentic Ops Hackathon

AI is changing how teams investigate incidents, detect threats, automate workflows, and build intelligent ...

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...