topic Does the length of metadata fields and its value, host, source, and sourcetype count against license consumption? in Getting Data In

Does the length of metadata fields and its value, host, source, and sourcetype count against license consumption?

johnhuang — Thu, 13 Apr 2023 16:33:39 GMT

Does the length of metadata fields and its value such as time, host, source and sourcetype count against license consumption?

For example, the following HEC JSON has a length of 212 characters but the event (_raw) is only 20 characters, is license calculated against the total json length or _raw length?

{ "time":1437522387, "host":"dataserver01.applicationmonitoring.com", "source":"/var/logs/application_monitoring.log", "sourcetype":"application_status", "event":{ "message":"Seems OK" } }

Re: Metadata fields license consumption

gcusello — Thu, 13 Apr 2023 06:26:36 GMT

Hi @johnhuang,

it seems the also the raw log is 212 chars not 20.

Anyway, License consuption is calculated only on the volume of _raw logs indexed:

if your event has 212 chars, it causes a License consuption of 212 bytes, even if you extract from it many fields.

Ciao.

Giuseppe

Re: Metadata fields license consumption

mattymo — Thu, 13 Apr 2023 14:46:48 GMT

When using the HEC event format and sending to the event endpoint (not raw endpoint) only the "event" field should be rated. The metadata fields are not.

Re: Metadata fields license consumption

johnhuang — Thu, 13 Apr 2023 15:08:17 GMT

Thank you @mattymo, I was able to test and confirm your answer.

The following event sent to HEC would incur license usage of 22 bytes.

{ "index":"summary_is_temp", "host":"testing_hec_license_consumption.acme.org", "source":"/var/logs/application_monitoring.log", "sourcetype":"application_status", "event":{ "message":"Seems OK" } }

Re: Metadata fields license consumption

mattymo — Thu, 13 Apr 2023 15:12:17 GMT

kudos to you to mistrusting and verifying!!! 🙂