topic How to partition indexed logfiles into multiple sourcetype? in Getting Data In

How to partition indexed logfiles into multiple sourcetype?

edgarrity — Wed, 19 Oct 2022 17:41:42 GMT

We need to index logfiles from our monitored devices which are partitioned into two segments. The first segment is CSV. The last segment are events.

col1,col2,col3,…,colN

col1,col2,col3,… ,colN

event1

event2

event3

…

eventN

This data from the logfile needs to be sent to one index with a two sourcetypes. Sourcetype_csv for the first segment in the logfile and sourcetype_events for the last segment in the logfile. How do we structure the inputs.conf, props.conf, and transforms.conf for this?

We were thinking we could leverage filtering to take advantage that the events are all prefixed and postfixed with “***”. However, there does not seem to be a way to have the one logfile type partitioned into more than one sourcetype.

Re: How to partition indexed logfiles into multiple sourcetype

gcusello — Wed, 19 Oct 2022 17:28:15 GMT

Hi @edgarrity,

if you can find a regex to identify logs of the dirst or of the second type, you have two solutions:

create a rule to assign sourcetype (for more infos see at https://docs.splunk.com/Documentation/Splunk/9.0.1/Data/Configurerule-basedsourcetyperecognition
override the sourcetype, assign the same sourcetype to both and then overriding one using the regex and following the method you can find at https://docs.splunk.com/Documentation/Splunk/9.0.1/Data/Advancedsourcetypeoverrides

Ciao.

Giuseppe

Re: How to partition indexed logfiles into multiple sourcetype

edgarrity — Thu, 20 Oct 2022 13:31:44 GMT

Thanks.

That worked.