Splunk Enterprise

Is it possible to calculate storage and remove part of log?

SplunkySplunk
Explorer

Hello

Is it possible to calculate the storage that part of log is taking ? 
I have a log file that contains a message that i want to calculate the storage it takes

after getting the numbers, is it possible to exclude it from index ?

 

Thanks

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Depends on what you need it for and when.

One thing is length of the raw data itself. And that was already shown. Another thing is how much space in splunk indexes is consuned by storing this particular event. And for this there is no clear single answer as it depends on how good the event (and whole bucket data) compresses, if you extract any additional indexed files and so on.

About excluding from indexes - again depends on what you really want. If you want to really prevent long events from getting indexed (why not just truncate them?), you'd have to do some check using ingest-time eval and then route to nullqueue based on the result of that check. If, however, you just want to not show long events in the results just filter your events with

| where len(_raw)<120

or whatever limit you want.

Don't use delete command. It has its very limited uses but generally shouldn't be used.

0 Karma

jotne
Builder

You can do some like this:

 

<your search> | eval len=len(_raw) | stats sum(len)

 

It will give you size in bytes.  This may also take some time depending of how much data you have how long back you do search.

To remove the same data, you can delete them.  You need to give admin the delete role first:

 

<your search> | delete

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Developer Program!

Hey Splunk community!  We are excited to announce that Splunk is launching the Splunk Developer Program in ...

Splunkbase Year in Review 2024

Reflecting on 2024, it’s clear that innovation and collaboration have defined the journey for Splunk ...

Developer Spotlight with Brett Adams

In our third Spotlight feature, we're excited to shine a light on Brett—a Splunk consultant, innovative ...