topic Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc? in Getting Data In

Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

HeinzWaescher — Fri, 07 Aug 2015 07:44:46 GMT

Hi,

is it possible to define field aliases, calculated fields, or automatic lookups for multiple sourcetypes? It would be great to avoid creating a configuration for every sourcetype itself. Wildcards don't help here.

Best

Heinz

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

woodcock — Fri, 07 Aug 2015 13:53:26 GMT

Actually, there is an undocumented wildcard syntax for props.conf ; it works like this:

[(?:::){0}SourcetypePrefxTextHere*]

So this matches anything that starts with SourcetypePrefixTextHere and matches will be processed in this props.conf stanza.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

alacercogitatus — Fri, 07 Aug 2015 14:10:27 GMT

No, you cannot have wildcarded sourcetype configurations. As @woodcock mentioned, there is a hack, a really messy hack. It is undocumented for many reasons.

You should not use it for these reasons:

This feature is not a feature
It is not supported
It is not QA'ed
It is not guaranteed to exist in next release
Performance impacts at scale
Adds complexity to troubleshooting and future development

Here is the blog post where this stems from http://blogs.splunk.com/2014/07/31/quick-tip-wildcard-sourcetypes-in-props-conf/
Additionally - @jrodman says not to use it, and he would know.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

woodcock — Fri, 07 Aug 2015 14:23:53 GMT

Other users have mentioned that this hack was actually suggested to them by Splunk personnel and I cannot see Splunk ever decommissioning it because of this. It does work and I use it in production but I do keep an eye on it.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

alacercogitatus — Fri, 07 Aug 2015 14:32:07 GMT

It was never actually "commisioned" to begin with. It is fallout from the way Splunk processes the host::, source::, rule::, delayedrule:: directives. There is a reason Splunk is removing this hack from all official TAs and Apps.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

woodcock — Fri, 07 Aug 2015 16:47:37 GMT

Conceded: decommission was the wrong word. I should have said, "make a change that will break it".

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

HeinzWaescher — Tue, 11 Aug 2015 07:39:57 GMT

Thanks for your input!

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

wryanthomas — Wed, 30 Sep 2020 02:03:14 GMT

So... since 2015, has Splunk provided a way to do this? With some add-ons creating a multitude of sub-sourcetypes, this seems like a fairly compelling need. My use case is to create a transaction_id across all proofpoint sub-sourcetypes (to span across the sourcetypes all instances of qid or sendmail_id). I don't want to have to create one for every soucetype. 😕

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

splunkmarc — Thu, 05 Dec 2019 01:36:50 GMT

Works in the web interface too.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

wryanthomas — Thu, 05 Dec 2019 13:30:14 GMT

Thanks. Can you elaborate? What exactly did you enter in the web interface? (Maybe share a screenshot?)

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

woodcock — Thu, 05 Dec 2019 16:20:03 GMT

He means that if you go to Settings -> Fields -> Calculated Fields -> New (or similar) and enter (?:::){0}SourcetypePrefxTextHere* for the named field under Apply to when sourcetype is selected, it will work as a wildcard.

Re: Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

woodcock — Thu, 05 Dec 2019 16:21:44 GMT

No, this is till the only way and it is in many Splunk apps that Splunk themselves created and support. It is rock solid and never going away.