dashboard data retention

[splunk_novice99]

Hello Splunk community,

I have a dashboard whereby I can search on data going back for a maximum of 30 days.   I'm looking for a way whereby I can achieve long term trending.  What would be the best approach for comparing data on a month-by-month basis for example?  After 30 days I want to save that data, recall that data at a later date and do a comparison.  Is this even possible?  

Thanks in advance.

[bowesmana]

The data will be retained in Splunk for as long as it's been configured to stay, so although your dashboard may be searching data for the last 30 days, it may be the data is there for longer.

Generally the approach to your problem is to look at summary indexing. What people often do is to ingest data from their sources and then do aggregations on those source and save aggregations to a summary index. The main index with all the data is then retained for a short period, whereas the smaller data volume is configured to be retained for a longer period so it can be used for long term analysis.

Look at reports/summary indexing which can do summary indexing automatically and also the collect SPL command allows you to do it manually.

When people ask the question about whether something is possible, the answer and almost always yes and often there is more than one way.

As for dashboarding, that's the easy part - if you have prepared your data, then you can do what you like on that data, as long as you have it.

[splunk_novice99]

Thanks for your reply.

It seems that the approach that I need to utilise for this is to use a savedsearch to periodically populate a csv lookup table and then have a dashboard to search against the table which contains the historic data.

Now sure exactly how to achieve this at a this stage.

[bowesmana]

Unless you need a CSV, I would suggest using Splunk's indexes to summarise data. It is more flexible to get data out of the index than a CSV, but you are on the right track.

Write yourself a search that collects data for an interval that summarises it in a way you would want to save. Typically this may run daily or hourly and the saved search has a 'summary indexing' option, so you can tell Splunk to write it to a summary index.

You will need the index to exist, but it's a simple option to enable.  Searches (Reports) can be scheduled, so if you want to run it daily, you could schedule it to run after midnight each day and then use a time range of 'yesterday' for its search.