Does splunk store data in ansi.? I.E. is len(_raw) an indicator of bytes stored?

[wkupersa]

Hi!

I am trying to determine how much certain events affect our license and storage. I am a user of my system and not an admin. Therefore, I am more limited in what I can see and do in order to answer my question.

I am wondering how Splunk licenses/stores incoming events. Does splunk store data in ansi? i.e. is len(_raw) an indicator of bytes stored?

Thanks!

[richgalloway · Accepted solution]

The len function returns the "character length" of a string, which does not necessarily map to byte count since strings are UTF-8, not ANSI. To further complicate the matter, Splunk compresses data in indexes so having a byte count would not reflect the actual storage space used, but would indicate license use.

License and storage use are really admin matters. What problems are you trying to solve that requires users to know about them?

[wkupersa]

Thanks! I am looking at Windows event logs. As a consumer of the information, I want as many event types. Mgmt wants cost vs benefit. I can't approach cost without knowing impact to license and storage. Even if I send some hosts to a server where I have admin, is there a good way to determine impact of certain eventtypes w/o sending them to their own index? A command to see size of a field (including _raw) would be useful.

[richgalloway]

If you have admin access to a server, send the new data to a separate index on that server. Then you can see exactly how much storage and license is used by that index. Using a separate index makes it simpler to see the effect of the new data and makes it easy to delete the data when you're done. It also avoids affecting any apps that read existing indexes.