Splunk Dev

How splunk internally stores events and Matrix data

santosh121
Explorer

I wanted to understand how actually splunk internally stores the events and Matrix data and what is the benefits of Matrix way of storing over events ?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Yes, storing the values as metrics will be orders of magnitude faster than storing them as events.  You will need to redesign how you store them, however, because metrics can contain a single value along with several "dimensions" that enhance the metric.  See https://docs.splunk.com/Documentation/Splunk/8.2.0/Metrics/GetStarted for more information.

There are other ways to speed up processing.  Consider an accelerated datamodel or a summary index.

Also, make sure your queries are as efficient as possible.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

santosh121
Explorer

Can anyone suggest on the above usecase please.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Perhaps you mean "metrics" rather than "matrix"?

Events and metrics are stored very differently.  Events must be general in nature so they can store any type of data whereas metrics are rather structured and intended to store numeric data.  Metrics indexes perform on the order of 2,000 times better than event indexes.

---
If this reply helps you, Karma would be appreciated.
0 Karma

santosh121
Explorer

Thanks richgalloway.

Yeah i meant that 🙂

 

I have a data set like

We get events every sec multiple entries with the columns Env, BG, APP, Interface,BusinessField1, Valuefield1,techSuccessCount, TechnicalInprogressCnt, TechnicalFailureCnt, BusinessSuccessCount,BusinessInprogressCount,BusinessfailureCount,Valuefield2Count 

Env, BG, APP, Interface,BusinessField1, Valuefield1 (These columns are string and can be  unique row when we do stats with these.

alongwith the above fields we have techSuccessCount, TechnicalInprogressCnt, TechnicalFailureCnt, BusinessSuccessCount,BusinessInprogressCount,BusinessfailureCount,Valuefield2Count (These columns are int)

Now when we do stats everytime for last 6-8 months it is too much of processing so we are performing bin span 15m _time and storing them uniquely.

will it be to store them as metrics? 

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, storing the values as metrics will be orders of magnitude faster than storing them as events.  You will need to redesign how you store them, however, because metrics can contain a single value along with several "dimensions" that enhance the metric.  See https://docs.splunk.com/Documentation/Splunk/8.2.0/Metrics/GetStarted for more information.

There are other ways to speed up processing.  Consider an accelerated datamodel or a summary index.

Also, make sure your queries are as efficient as possible.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | What did the zero say to the eight?

June 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

Splunk Observability Cloud's AI Assistant in Action Series: Onboarding New Hires & ...

This is the fifth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...