Hello
Is it possible to calculate the storage that part of log is taking ?
I have a log file that contains a message that i want to calculate the storage it takes
after getting the numbers, is it possible to exclude it from index ?
Thanks
Depends on what you need it for and when.
One thing is length of the raw data itself. And that was already shown. Another thing is how much space in splunk indexes is consuned by storing this particular event. And for this there is no clear single answer as it depends on how good the event (and whole bucket data) compresses, if you extract any additional indexed files and so on.
About excluding from indexes - again depends on what you really want. If you want to really prevent long events from getting indexed (why not just truncate them?), you'd have to do some check using ingest-time eval and then route to nullqueue based on the result of that check. If, however, you just want to not show long events in the results just filter your events with
| where len(_raw)<120
or whatever limit you want.
Don't use delete command. It has its very limited uses but generally shouldn't be used.
You can do some like this:
<your search> | eval len=len(_raw) | stats sum(len)
It will give you size in bytes. This may also take some time depending of how much data you have how long back you do search.
To remove the same data, you can delete them. You need to give admin the delete role first:
<your search> | delete