topic Re: How do you create a duplicate source type? in Getting Data In

How do you create a duplicate source type?

santosh_hb — Tue, 27 Nov 2018 10:12:39 GMT

Hi All,

Need a quick help on creating duplicate source types in Splunk.

Currently, the data is flowing into index=test1 sourcetype=data1

Now, I would like to send the same data into another source type as well while keeping the original source type also.

So, the final result I am looking for is like below:

On Splunk Web, execute the search as below:

index=test1

Then, I should be able to see 2 source types as sourcetype=data1 and sourcetype=data2 for the same index.

Thanks in advance,
Santosh

Re: How do you create a duplicate source type?

richgalloway — Tue, 27 Nov 2018 12:57:27 GMT

Events can have exactly one sourcetype.

Re: How do you create a duplicate source type?

dkeck — Tue, 27 Nov 2018 13:20:43 GMT

What is it you want to achieve with the separation?

Re: How do you create a duplicate source type?

FrankVl — Tue, 27 Nov 2018 13:53:06 GMT

What would be the purpose of this? And would you intend to duplicate the data then, or do you mean that part of your data should get assigned data1 and part of it should get assigned data2 as sourcetype?

Re: How do you create a duplicate source type?

santosh_hb — Thu, 29 Nov 2018 05:43:44 GMT

Hi All,

Thanks for the response.

Purpose of creating duplicate sourcetype:

The original sourcetype (data1) was created by another team and we don't have any control on this sourcetype. Going forward if they delete this sourcetype without our knowledge then we won't be having any control on the sourcetype. Hence, we are creating a duplicate sourcetype (data2) so that we can have the control of the data flowing into this sourcetype and can parse the data easily.

regards,
Santosh

Re: How do you create a duplicate source type?

FrankVl — Thu, 29 Nov 2018 17:09:02 GMT

Sounds like you’re trying to solve a people/process problem with a technology solution. That isn’t always the best way to go.

But if you really cannot solve this on a people/process level, why not simply take a backup of that sourcetype config so that in the case they remove it, you can simply add it again?

Re: How do you create a duplicate source type?

eagle4splunk — Tue, 29 Sep 2020 23:32:25 GMT

You can use clone_sourcetype to clone your data into another sourcetype.

Your will need to configure your props.conf and transforms.conf to look something like this:

props.conf

[original_sourcetype]
parm1 = xxx
parm2 = 123

[duplicate_sourcetype]
parm1 = xxx
parm2 = 123

[source::]
TRANSFORMS-clone = clone_sourcetype

transforms.conf:

[clone_sourcetype]
CLONE_SOURCETYPE = duplicate_sourcetype
REGEX = .

Re: How do you create a duplicate source type?

Jem_17 — Mon, 24 Apr 2023 13:02:14 GMT

Mine purpose for duplicate sourcetype is - I am having another data inputs with extractions similar to previous sourcetype1 but different sources.

So I need to create a clone of sourcetype1 conf with another name(sourcetype2).

Re: How do you create a duplicate source type?

PickleRick — Mon, 24 Apr 2023 15:42:31 GMT

1. Please use a formatted block or a code sample box to insert code into your text. It makes it more readable.

2. You are aware that CLONE_SOURCETYPE duplicates your events? Which - among other things - results in double license usage for affected events?

Re: How do you create a duplicate source type?

PickleRick — Mon, 24 Apr 2023 15:49:14 GMT

Wait a second. As @FrankVl said - your biggest problem isn't creation or not of a sourcetype. You don't have a well-developed data onboarding process. And you are trying to use technical means to walk around an organizational problem. Even if you manage to make the "source" sourcetype visible under another name (either by applying this data-duplication recipe or by using the rename option, noone can guarantee that your data format won't suddenly change or the "source" sourcetype won't get renamed to something else rendering your walkaround useless.

So try to solve organizational problems with organizational tools.