I have about 1TB of machine data as CSV files primarily feeding health/metric data about the different-sub systems in machines ( cpu stats, io stats, san stats etc..)
I wanted to build some summaries ( daily metric aggregations using stat commands, across about 100 servers ) for about 6 months, that perform really well, to get some top-level insights across the fleet ( raw data size ~ 1TB for all servers ). I plan to maintain data for rolling 6 months, where historic data beyond 6 months would be deleted periodically.
This link (http://docs.splunk.com/Documentation/Splunk/7.0.2/Knowledge/Aboutsummaryindexing) has some great suggestions on when to use report acceleration vs data model acceleration vs summary indexing. But is there a documentation that adds the new metrics store to the performance/aggregation comparison or a similar guide with points to consider?
The CSV I have is formatted in the following fashion
Timestamp | ServerName | DBStatValue | AppStatValue | IOStatValue | CacheStatValue ...( few other metrics )
Now, to comply with the "Metrics" store standard, I would have messaged the CSV's I receive from my data stores to the following format, if I am not mistaken, which adds a static "metric name" column, for each metric being collected.
Timestamp | ServerName | DBMetricName | DBMetricValue | AppMetricName | AppMetricValue | IOMetricName | IOMetricValue |CacheMetricName| CacheStatValue ...( few other metrics )
Few questions in my mind, where I need help.
I am just getting started with the implementation and have the luxury and flexibility to choose the path that would best support the described use case. Any tips, best practices or suggestions from community experts are greatly appreciated.