Splunk Enterprise

Is it possible to calculate storage and remove part of log?

SplunkySplunk
Explorer

Hello

Is it possible to calculate the storage that part of log is taking ? 
I have a log file that contains a message that i want to calculate the storage it takes

after getting the numbers, is it possible to exclude it from index ?

 

Thanks

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Depends on what you need it for and when.

One thing is length of the raw data itself. And that was already shown. Another thing is how much space in splunk indexes is consuned by storing this particular event. And for this there is no clear single answer as it depends on how good the event (and whole bucket data) compresses, if you extract any additional indexed files and so on.

About excluding from indexes - again depends on what you really want. If you want to really prevent long events from getting indexed (why not just truncate them?), you'd have to do some check using ingest-time eval and then route to nullqueue based on the result of that check. If, however, you just want to not show long events in the results just filter your events with

| where len(_raw)<120

or whatever limit you want.

Don't use delete command. It has its very limited uses but generally shouldn't be used.

0 Karma

jotne
Builder

You can do some like this:

 

<your search> | eval len=len(_raw) | stats sum(len)

 

It will give you size in bytes.  This may also take some time depending of how much data you have how long back you do search.

To remove the same data, you can delete them.  You need to give admin the delete role first:

 

<your search> | delete

 

Tags (1)
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...