I am planning to use splunk for visualizing my data and could use some help modelling it .
i have some json data stored in mongodb that i need to access in splunk .
Data reaches upto 1 TB per day .
I need to fetch, process and index this data so it can be viewed in splunk.
My question is with the fetching the data . What is the best way to fetch this data periodically ? I need to fetch data from mongodb for a particular time frame . Ie this is not real time streaming data .
After I specify data sources using hunk , how do i index the data for specific time ranges ?
Should i create an intermediary application (node.js or java) just for fetching and indexing the data periodically ?
... View more