Getting Data In

What is CLONE_SOURCETYPE used for in transforms.conf? Are there examples?


It gets dangerous when I start looking at docs and start seeing features that I hadn't noticed before. So I was looking at the spec for transforms.conf and I noticed this paragraph stuck in here:

* Specifies the sourcetype of a cloned data stream.
* The value string must be nonempty.
* The value string should be different from the sourcetype of the original data stream.
  If it is the same, no cloning happens and a warning will be logged.

I don't see it mentioned on the Indexing wiki and I'm thinking, could this be used somehow for multi-pass parsing? It also raises the question on how to get a cloned data stream, if I could somehow send to multiple queues after the typing pipeline, or do I have to adjust TCPROUTING and forward data back to myself (possibly also twiddling my routes or something), or I'm completely off, and this doesn't have to do with parsing.

Does anyone know more about the intended use of this setting and/or have examples of how they've used it?


So did you figure out a full config example on how this was used?

Can you clone to a different index?

0 Karma

Esteemed Legend

Let's say that you must clone an index (and possibly modify it somehow, maybe to anonymize it) so that some users have access to one version and others to the modified version. This is how that could be done.