topic Re: Source Override in Getting Data In

Sourcetype Override

nabeel652 — Tue, 07 Jul 2020 03:02:25 GMT

I have a deployed a scripted input with source=perfmon_script that gets server and workstation data.

in props.conf I have:

[source::perfmon_script] TRANSFORMS-changesourcetype = sourcetype_new

in transforms.conf

[sourcetype_new] REGEX = . FORMAT = sourcetype::somesrctype DEST_KEY = MetaData::Sourcetype

Sourcetype not changing. What am I doing wrong?

Re: Source Override

gcusello — Mon, 06 Jul 2020 06:11:12 GMT

Hi @nabeel652 ,

what's the wrong behavior?

Anyway, to have as sourcetype you have to use a different FORMAT:

Re: Source Override

gcusello — Mon, 06 Jul 2020 06:13:20 GMT

Hi @nabeel652 ,

what's the wrong behavior?

Anyway, to have as sourcetype perfmon:srv or perfmon:ws you have to use a different FORMAT:

FORMAT = perfmon:$1

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 06:15:08 GMT

It's simply not working 😞

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 06:18:04 GMT

Thanks @gcusello

Yes, that's correct. However, my transform is not working at all

Re: Source Override

gcusello — Mon, 06 Jul 2020 06:20:21 GMT

HI @nabeel652 ,

yes, but in which way: remain the original sourcetype? or override both the the sourcetype with the same? or what else?

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 06:21:40 GMT

The original sourcetype remains

Re: Source Override

nabeel652 — Tue, 07 Jul 2020 04:10:37 GMT

@gcusello

Some more details:
I've deployed the scripted input on one of my heavy forwarders. I've tried this transform on the same heavy forwarder as well as the indexer but fails to change the sourcetype to new one.

Re: Source Override

gcusello — Mon, 06 Jul 2020 06:39:47 GMT

Hi @nabeel652 ,

debug the problem:

use a static overriding to understand if the problem is the transformation:

[sourcetype_override] REGEX = . FORMAT = sourcetype::perfmon DEST_KEY = MetaData:Sourcetype

If this transformation runs the problem is in the original transformation itself, if it doesn't run the problem is before.

Obviously you restarted Splunk on the HF that you modified, is it correct?

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Tue, 07 Jul 2020 04:11:24 GMT

Tried that but still not working.

Yes, I'm restarting Splunk everytime I make changes

Re: Source Override

gcusello — Mon, 06 Jul 2020 07:06:33 GMT

Hi @nabeel652 ,

You said that the sourcetype isn't overwritten.

This means that the problem isn't in the transformation, but in the flow.

Only one question: the sourcetype performance that you assign to the script in the inputs.conf, is used only in this case or has another use?

in other words, try to modify your configuration in this way (I used performance_test but you can use the one you like):

inputs.conf:

[script://./bin/serverPerformance.py] disabled=0 sourcetype=performance_test source = perfmon_script interval=30

props.conf:

[performance_test] TRANSFORMS-changesourcetype = sourcetype_override

transforms.conf:

[sourcetype_override] REGEX = src\=(srv|ws)\_ FORMAT = sourcetype::perfmon:$1 DEST_KEY = MetaData::Sourcetype

in few words, use original sourcetype, instead source for the overriding.

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 07:48:57 GMT

Thanks @gcusello

Sourcetype has no other use. I'm in fact trying to create an example to demonstrate sourcetype override. Works fine with monitored inputs but scripted inputs giving problems.

Still no luck. I've used the original sourcetype i. e "performance" but no change at all.

Re: Source Override

gcusello — Mon, 06 Jul 2020 08:32:28 GMT

hi @nabeel652 ,

could you share the inputs,props and transforms you used in the last test?

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Tue, 07 Jul 2020 04:04:51 GMT

props.conf

[performance] TRANSFORMS-changesourcetype = sourcetype_override

transforms.conf

[sourcetype_override] REGEX = . FORMAT = sourcetype::new_srctype DEST_KEY = MetaData::Sourcetype

Re: Source Override

gcusello — Mon, 06 Jul 2020 10:29:59 GMT

Hi @nabeel652 ,

can you confirm that props and transforms are located on Heavy Forwarder?

and that HF was restarted after files updates?

there isn't any addition reasono for the problem.

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 10:37:11 GMT

Yep,

All three files inputs.conf, props.conf and transforms.conf is in

/opt/splunk/etc/apps/mycustomapp/local/

Re: Source Override

gcusello — Mon, 06 Jul 2020 10:43:05 GMT

Hi @nabeel652 ,

The last try I hint is to put props.conf and transforms.con also on Indexers, but it shouldn't be relevant!

After Open a case to Splunk!

Ciao.

Giuseppe

Re: Source Override

nabeel652 — Mon, 06 Jul 2020 11:19:10 GMT

No luck 😞

Will log a case

Re: Source Override

nabeel652 — Tue, 07 Jul 2020 02:56:36 GMT

@gcusello

Well, what a silly mistake that I've made

It is MetaData:Sourcetype NOT MetaData::Sourcetype

Fixed it and all good!

Thanks anyway for your time and sorry once again for the small typo that caused big hassle 😄