This is the inputs.file:: As you can see they all go to the same directory structure, but the last one is supposed to catch all the logs not beginning with the defined *_xxxxx_*.log so that general logs will be stored in Splunk as well. How can I do this?
[monitor:///var/log/containers/*_ctisp1_*.log]
index = ctisp1
sourcetype = dks-ctisp1
followSymlink = true
[monitor:///var/log/containers/*_ocpprd_*.log]
index = ocpprd
sourcetype = dks-ocpprd
followSymlink = true
[monitor:///var/log/containers/*_custconnectp1_*.log]
index = custcontp1
sourcetype = custcontp1
followSymlink = true
[monitor:///var/log/containers/*_ocpnotifp3_*.log]
index = ocpnotifp3
sourcetype = dks-ocpnotifp3
[monitor:///var/log/containers/*_ocpcorep3_*.log]
index = ocpcorep3
sourcetype = ocpcorep3
[monitor:///var/log/containers/*_custcon2p3_*.log]
index = custcon2p3
sourcetype = custcon2p3
[monitor:///var/log/containers/*_custcon1p3_*.log]
index = custcon1p3
sourcetype=custcont1p3
[monitor:///var/log/containers/*_ctisap3_*.log]
index = ctisap3
sourcetype = dks-ctisap3
[monitor:///var/log/containers/*_ctisp1_*.log]
index = ctisp1
sourcetype = dks-ctisp1
[monitor:///var/log/containers/*_ivrp1_*.log]
index = ivrp1
sourcetype = dks-ivrp1
#[monitor:///host/containers/*/[a-f0-9]+-json.log$]
#index=dcp
#sourcetype=dner-logsiamanti-container-logs
#[monitor:///var/lib/docker/containers/*/[a-f0-9]+-json.log$]
#index=dcp
#sourcetype=diamanti-container-logs
[monitor:///var/log/containers/*_ocpnotifp3_*.log]
index = ocpnotifp3
sourcetype = ocpnotifp3
[monitor:///var/log/containers/*_ocpcorep3_*.log]
index = ocpcorep3
sourcetype = ocpcorep3
[monitor:///var/log/containers/*_custcon2p3_*.log]
index = custcon2p3
sourcetype = custcont2p3
[monitor:///var/log/containers/*_igridp2_*.log]
index = igridp2
sourcetype = dks-igridp2
## END of PROD
## Monitor all Diamanti logs
[monitor:///var/log/diamanti/.../*.log]
index=dcp
sourcetype = diamanti-system-logs
# Monitor Container logs
[monitor:///var/log/containers/*.log]
index=dcp
sourcetype = diamanti-container-logs
Hi,
you can add a blacklist. I am not sure if all your files could be easily matched with regex, but it is a possibility.
blacklist = <regular expression> * If set, files from this input are NOT monitored if their path matches the specified regex. * Takes precedence over the deprecated '_blacklist' setting, which functions the same way. * If a file matches the regexes in both the deny list and allow list settings, the file is NOT monitored. Deny lists take precedence over allow lists. * No default.
I will give this a try and see if I can use the blacklist
Hi @NanSplk01,
the problem is probably that there's an overlapping of inputs and usually Splunk index a log only one time.
I hint to blacklist "*_xxxxx_*.log" in the last two stanzas, in other words:
## Monitor all Diamanti logs
[monitor:///var/log/diamanti/.../*.log]
index=dcp
sourcetype = diamanti-system-logs
blacklist = *_xxxxx_*.log
# Monitor Container logs
[monitor:///var/log/containers/*.log]
index=dcp
sourcetype = diamanti-container-logs
blacklist = *_xxxxx_*.log
for more infos, you can see at https://docs.splunk.com/Documentation/Splunk/8.2.6/Admin/Inputsconf
Ciao.
Giuseppe