Getting Data In

Can we define multiple sourcetypes for field aliases, calculated fields, automatic lookups etc?

HeinzWaescher
Motivator

Hi,

is it possible to define field aliases, calculated fields, or automatic lookups for multiple sourcetypes? It would be great to avoid creating a configuration for every sourcetype itself. Wildcards don't help here.

Best

Heinz

1 Solution

alacercogitatus
SplunkTrust
SplunkTrust

No, you cannot have wildcarded sourcetype configurations. As @woodcock mentioned, there is a hack, a really messy hack. It is undocumented for many reasons.

You should not use it for these reasons:

  1. This feature is not a feature
  2. It is not supported
  3. It is not QA'ed
  4. It is not guaranteed to exist in next release
  5. Performance impacts at scale
  6. Adds complexity to troubleshooting and future development

Here is the blog post where this stems from http://blogs.splunk.com/2014/07/31/quick-tip-wildcard-sourcetypes-in-props-conf/
Additionally - @jrodman says not to use it, and he would know.

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

No, you cannot have wildcarded sourcetype configurations. As @woodcock mentioned, there is a hack, a really messy hack. It is undocumented for many reasons.

You should not use it for these reasons:

  1. This feature is not a feature
  2. It is not supported
  3. It is not QA'ed
  4. It is not guaranteed to exist in next release
  5. Performance impacts at scale
  6. Adds complexity to troubleshooting and future development

Here is the blog post where this stems from http://blogs.splunk.com/2014/07/31/quick-tip-wildcard-sourcetypes-in-props-conf/
Additionally - @jrodman says not to use it, and he would know.

View solution in original post

woodcock
Esteemed Legend

Other users have mentioned that this hack was actually suggested to them by Splunk personnel and I cannot see Splunk ever decommissioning it because of this. It does work and I use it in production but I do keep an eye on it.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

It was never actually "commisioned" to begin with. It is fallout from the way Splunk processes the host::, source::, rule::, delayedrule:: directives. There is a reason Splunk is removing this hack from all official TAs and Apps.

woodcock
Esteemed Legend

Conceded: decommission was the wrong word. I should have said, "make a change that will break it".

0 Karma

HeinzWaescher
Motivator

Thanks for your input!

0 Karma

wryanthomas
Communicator

So... since 2015, has Splunk provided a way to do this? With some add-ons creating a multitude of sub-sourcetypes, this seems like a fairly compelling need. My use case is to create a transaction_id across all proofpoint sub-sourcetypes (to span across the sourcetypes all instances of qid or sendmail_id). I don't want to have to create one for every soucetype. 😕

0 Karma

woodcock
Esteemed Legend

No, this is till the only way and it is in many Splunk apps that Splunk themselves created and support. It is rock solid and never going away.

0 Karma

woodcock
Esteemed Legend

Actually, there is an undocumented wildcard syntax for props.conf ; it works like this:

[(?:::){0}SourcetypePrefxTextHere*]

So this matches anything that starts with SourcetypePrefixTextHere and matches will be processed in this props.conf stanza.

splunkmarc
Engager

Works in the web interface too.

0 Karma

wryanthomas
Communicator

Thanks. Can you elaborate? What exactly did you enter in the web interface? (Maybe share a screenshot?)

0 Karma

woodcock
Esteemed Legend

He means that if you go to Settings -> Fields -> Calculated Fields -> New (or similar) and enter (?:::){0}SourcetypePrefxTextHere* for the named field under Apply to when sourcetype is selected, it will work as a wildcard.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.