Is it possible to set up forwarders to index data on the path of the file and a portion of the file name automatically.
Ie
/home/info/data/logs/"cluster"/host1.type1.log
/home/info/data/logs/"cluster"/host2.type2.log
The setup should automatically forward any files under /home/info/data/logs and add indexes for the "cluster" and the host? Any pointers in the right direction would be helpfull
You'd need to set up props/transforms where the data is being parsed(heavy forwarder/Indexer) to route data to specific indexes based on a regex:
Thanks for that pointer, but I am not sure that is what I was really asking. The logs all go into one index but have the extra tags for cluster and host added by the forwarder.
I could set up forwarders for each "log type". These should be able to add tags for the cluster and host and read all files of that log type under /home/info/data/logs/*/.