I wish to use a search-time extracted field as the basis for routing to a specific index.
In my props.conf file I have a search-time extraction of fields. These fields are taken from the monitored log file name, which is a name that follows a known pattern but whose exact name cannot be predicted. Here is a typical log file name:
/var/log/containers/coredns-845cddccfc-prm27_kube-system_coredns-4a676a5795e68f4f2bacaed884bc3811de019083801b803ad67974b4ac3c221d.log
The set of log files are of a common sourcetype=kubernetes, evidenced in the LWF inputs.conf
[monitor:///var/log/containers/*.log]
sourcetype = kubernetes
multiline_event_extra_waittime = true
In props.conf I have
[kubernetes]
EXTRACT-sourcefields = /var/log/containers/(?<pod>.*)_(?<namespace>.*)_(?<containerName>.*)-(?<containerID>.*)\.log in source
To route to an index of interest, I add this to my props.conf
TRANSFORMS-theindex = theindex
In transforms.conf I add the associated stanza for this index, with the intent of routing to myindex when containerName=aContainerName. I note that the index myindex does in fact exist.
[theindex]
SOURCE_KEY = field:containerName
REGEX = aContainerName
DEST_KEY = _MetaData:Index
FORMAT = myindex
However, I find that the myindex never accrues events.
I suspect I am not allowed to use search-time extracted fields in transforms. If that is true, and my desire to route based on an extracted field, what approach should I take?
... View more