I am working on OS log onboarding data under multiple hostname folders and these hostname folders are located at same file path.
My plan is to dynamically onboard these logs to indexes based on relevant hostname with dynamic sourcetype set based on filename text.
My logs directory structure: \opt\myAPP\host1\filename_type1.log \opt\myAPP\host2\filename_type2.log \opt\myAPP\host3\filename_type3.log
Expected index name from foldername: indexname_host1 indexname_host2 indexname_host3
Expected sourcetype name from filename : sourcetype_type1 sourcetype_type2 sourcetype_type3
Following are the configuration am using at inputs.conf , where index=route is just placeholder and no such index is created:
host_segment = 3
index = route
sourcetype = reroute_1
whitelist = (host1|host4|host5)
remember that all the knowledge objects in Splunk are related to sourcetype, so if you have different sourcetypes you cannot use (or it's very difficoult) field extractions, eventtypes, tags, etc...; this means that it isn't a good idea to use different sourcetypes!
At the same time, why do you want to put logs from hosts in different indexes?
Usually indexes are choosen based on retention policies and access right, eventually based on quantity of data (e.g. large data flows aren't stored in indexes together with few data flows), not other.
In other words, Splunk isn't a database and usually logs are stored in indexes which common retention policies and access rights using a limited number of sourcetypes that permits to manage knowledge objects.
Logs are searcheable using all their fields like sourcetype host and others.