In the transforms.conf file, how do I support the alternatives on the REGEX line with the
corresponding FORMAT line group’s numbering? See the example attempt below:
I can have either 4 groups or 6 groups on the REGEX line
(that’s why there is a pipe "|" there;
InComingList SOURCE_KEY could have either 4 groups or 6 groups).
Corresponding FORMAT line then need to be able to: either support 4 groups’ matches
or 6 groups’ matches. How to do that?
How to number the groups - increase the numbers or restart-the-numbering again
for the part after the pipe?
[Unfortunately transforms.conf specification does not support
Any quick insights will be deeply appreciated!
[testTransformPara] REGEX=((\w+?)_(\w+)=(\d+):(\d+)) | ((\w+?)_(\w+)=(\d+):(\d+)@(\w+)=(\d+)) FORMAT = (FIELD1::$1 FIELD2::$2 FIELD3::$3 FIELD4::$4) | (FIELD1::$1 FIELD2::$2 FIELD3::$3 FIELD4::$4 FIELD5::$5 FIELD4::$6) SOURCE_KEY = InComingList MV_ADD = true
Why not just have two separate transforms? Like you say transforms.conf does not support this kind of conditional extraction and I think it's likely it never will either since you can solve the problem by using separate transforms.
Also, "better practices" (I would say "best" but that is subjective) suggest that you build context-specific apps for your various inputs and their processing. If you have a single
props.conf with 'thousands of lines' in it, then you may want to consider breaking those down into different apps and managing your revisions in smaller units.
Ayn: Can't get into more specifics here, but, imagine a scenario, where you already have thousands of lines of sourcetypes defined in props.conf and if any of the existing paras are changed, that then mandates a regression testing cycle for all the related clients. If the props is not touched in this fashion, then this dilemma is circumvented. transforms.conf's existing fields are matched perfectly and remain unperturbed. Some optional new fields are added, that are then tested out for the new client-apps.
I agree with Ayn here. The proper solution is multiple
transforms.conf stanzas with multiple references to them within
props.conf. If your change management / code freeze processes are such that you can change one Splunk config file but not a related Splunk config file, perhaps your processes are not aligned with reality in a live Splunk deployment.
Ayn: If there is a constraint (say, for some reason), that the corresponding props.conf can not be changed. So, the new transforms' para creation is not possible. In that case, we have to design the alternatives of the REGEX and FORMAT in the existing paragraph expressions only. What is the formal guideline from Splunk on this, if it exists. Could not find any direct examples/discussion of this topic yet.