What is the role of props.conf vs. transforms.conf in field extraction? How do they relate to each other in order to make field extraction work?
The high-level answer is that props.conf says what rules are applied to any event and when they are applied, and transforms.conf actually defines those rules.
So in props.conf, you say "events with the sourcetype XXX has the extraction YYY applied to it at parse time" or "events from host HHH has lookup JJJ applied at search time". transforms.conf would specify exactly how extraction XXX worked, or where lookup JJJ comes from.
This is generally true, though it's a little muddied because some of the rules are specified directly in props.conf. Some of these (e.g., rules for parsing timestamps or line breaks) are only specified in props.conf, while others (search time field extractions) can be either directly defined in props.conf, or referenced back to transforms.conf
regex based field extraction can be specified:
(1) inline in props.conf via the EXTRACT- field or
(2) in transforms.conf and referenced from props.conf via REPORT-
If you are reusing the extraction rule on other sources/sourcetypes/hosts it is recommended that you use (2), for extractions that you know are not going to be reused use (1)
The high-level answer is that props.conf says what rules are applied to any event and when they are applied, and transforms.conf actually defines those rules.
So in props.conf, you say "events with the sourcetype XXX has the extraction YYY applied to it at parse time" or "events from host HHH has lookup JJJ applied at search time". transforms.conf would specify exactly how extraction XXX worked, or where lookup JJJ comes from.
This is generally true, though it's a little muddied because some of the rules are specified directly in props.conf. Some of these (e.g., rules for parsing timestamps or line breaks) are only specified in props.conf, while others (search time field extractions) can be either directly defined in props.conf, or referenced back to transforms.conf
when there's an option to put something in props.conf or transforms.conf, could you add more detail about when you'd want to put info in one vs. the other?
As far as i know all probs.conf get merged together - like any other conf as well.
Is it right that each indexed event gets parsed through the whole probs.conf with matching sourcetypes?
My input for example is:
UDP port Source type
514 juniper
And the probs.conf can change the sourcetype?
[source::....juniper_sslvpn]
sourcetype = juniper:sslvpn
I thought indexed data is absolut fix and cannot be changed?
Thank you all
Michele