Splunk Enterprise

Is it possible to calculate storage and remove part of log?

SplunkySplunk
Explorer

Hello

Is it possible to calculate the storage that part of log is taking ? 
I have a log file that contains a message that i want to calculate the storage it takes

after getting the numbers, is it possible to exclude it from index ?

 

Thanks

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Depends on what you need it for and when.

One thing is length of the raw data itself. And that was already shown. Another thing is how much space in splunk indexes is consuned by storing this particular event. And for this there is no clear single answer as it depends on how good the event (and whole bucket data) compresses, if you extract any additional indexed files and so on.

About excluding from indexes - again depends on what you really want. If you want to really prevent long events from getting indexed (why not just truncate them?), you'd have to do some check using ingest-time eval and then route to nullqueue based on the result of that check. If, however, you just want to not show long events in the results just filter your events with

| where len(_raw)<120

or whatever limit you want.

Don't use delete command. It has its very limited uses but generally shouldn't be used.

0 Karma

jotne
Builder

You can do some like this:

 

<your search> | eval len=len(_raw) | stats sum(len)

 

It will give you size in bytes.  This may also take some time depending of how much data you have how long back you do search.

To remove the same data, you can delete them.  You need to give admin the delete role first:

 

<your search> | delete

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...