Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Admin Other
- :
- Knowledge Management
- :
- Re: Best ways to report on large volumes of data?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
For Service-Now dashboards we need to report on large volumes of data (one year of data to be precise).
Is summary index the best way to prevent slowness of dashboards?
To get all panels on the dashboards "Active tickets", "Active Tickets aged more than 60 days", "Not Updated tickets", etc - we would need to search on all available data for an year.
Thank you in advance
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Datamodel/tstats would be my first choice as they can be updated/rebuilt when you have maintenance or changes to the fields you are reporting on. Data models can only retain data as long as the source index.
Summary indexes can live indefinitely however may need manual intervention to backfill data gaps during maintenance windows or failures.
Metrics/mcollect/mstats is the newest and fastest method for historical metrics but lacks some of the automation of a data model.
Lastly there are lookup tables/CSVs and the kvstore that sometimes work well for edge use cases where you have a frequently changing datcache however won't work with the default time picker
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A summary index is perfect for this use case, but so is an accelerated data model. You should be able to prototype both in the same day if you know your data and outcomes well, so play around and enjoy the exercise.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A summary index is perfect for this use case, but so is an accelerated data model. You should be able to prototype both in the same day if you know your data and outcomes well, so play around and enjoy the exercise.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@woodcock a delayed thank you!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Which way did you go and why?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Datamodel/tstats would be my first choice as they can be updated/rebuilt when you have maintenance or changes to the fields you are reporting on. Data models can only retain data as long as the source index.
Summary indexes can live indefinitely however may need manual intervention to backfill data gaps during maintenance windows or failures.
Metrics/mcollect/mstats is the newest and fastest method for historical metrics but lacks some of the automation of a data model.
Lastly there are lookup tables/CSVs and the kvstore that sometimes work well for edge use cases where you have a frequently changing datcache however won't work with the default time picker
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rob_jordan , delayed thank you!
.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...
A Season of Skills: New Splunk Courses to Light Up Your Learning Journey
Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...
