topic Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold in Getting Data In

How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

MP5591 — Tue, 13 Aug 2019 15:19:02 GMT

How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

Sukisen1981 — Tue, 13 Aug 2019 16:58:10 GMT

are you asking size and retention of sourcetype or index?
One index has data size / time to retention bucket roll over policies. It can have multiple source / sourcetypes, that does not matter.
But I am thinking that perhaps you are asking for something else?

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

MP5591 — Wed, 14 Aug 2019 10:20:12 GMT

@Sukisen1981 -Thanks for responding.

I was asking about sourcetype. How to know the size allocated to sourcetype. Also what is the limit on the size data we pass in a transaction? we see that transaction with 1mB data is not getting logged in

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

jawaharas — Thu, 15 Aug 2019 09:46:35 GMT

Little more detail will be helpful to assist you.

What you mean by transaction with 1 MB data? Is it log file size or single log event size?

Can you post your inputs.conf file config from Universal Forwarder for the sourcetype?

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

kmorris_splunk — Thu, 15 Aug 2019 13:56:23 GMT

To @Sukisen1981's point, sourcetype has no size associated with it. You define the size of an index. Sourcetype is simply a way of categorizing your data.

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

MP5591 — Fri, 16 Aug 2019 10:20:39 GMT

@jawaharas-its single log event of size 1 MB. is there any limit on this .

Re: How to check size allocated to source-type and what is the maximum size of a transaction that sourcetype can hold

somesoni2 — Fri, 16 Aug 2019 12:56:58 GMT

A real-life transaction may contain multiple Splunk "events" (based on how the sourcetype is breaking events in your log file. The default size of an event is 10,000 bytes after which an event will be truncated (not dropped fully, just first 10,000 bytes are retained). You can change this by setting up TRUNCATE attribute for your sourcetype. You can refer to following Splunk documentation for information on various sourcetype level attributes:
https://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf#Line_breaking