I am trying to make an area chart which shows the average size of the parsing queue over time. I would like to add a horizontal bar as a threshold. I noticed that some logs have different values for the max_size_kb, so I thought I could use max to get the value and set my threshold to that, but for some reason, my search is returning zero results. I don't know why it's not working. If I hardcode a number for zzz, it'll work, but doesn't seem to work the way it is written now. The value changes between my hosts, so I don't want to hard code it. Current SPL index=_internal host=$hostToken$ group=queue name=parsingqueue | stats max(max_size_kb) AS zzz | timechart avg(current_size_kb) by ingest_pipe | eval threshold = zzz ... View more

I've combed through inputs.conf and the various questions on answers but can't seem to get a definitive example in how to employ a whitelist or modify my monitor stanza to match on specific folders and their sub-directories per my use case. Example: match on /mnt/data/apple/desired_folder/*/* match on /mnt/data/apple/dir_1/*/* match on /mnt/data/apple/folder_two/*/* DONT match /mnt/data/apple/junk/*/*] DONT match on too many others to list Each directory in the whitelist, has one more sub-directory, then the log files themselves, of which I want everything in the folder. Do I have to write 3 monitor stanzas for this? failed attempts - no logs get pulled in [monitor:///mnt/data/apple/(dir_1|folder_two|index_this)/*/*] and [monitor:///mnt/data/apple/*/*/*] whitelist = (dir_1|folder_two|index_this) For now I've resorted to 3 monitor stanza's but I thought there is a cleaner way to do this in Splunk that I've completely forgotten/missed. ... View more

We recently upgraded from 7.2.1 to 7.3.3 and from the _internal logs I can see that these new warnings are showing up across my indexer cluster. What is it saying and how do I go about fixing this? I've noticed that now I have an indexer just randomly lock up once a week. Any insight would be appreciated! 01-14-2020 00:58:27.994 +0000 WARN ProcessTracker - (child_17__Fsck) IndexConfig - idx=_introspection Path coldPath='/opt/splunk/var/lib/splunk/_introspection/colddb' (realpath '/mnt/local/hot/_introspection/colddb') is inside volume=primary (path='/mnt/local/hot', realpath='/mnt/local/hot'), but does not reference that volume. Space used by coldPath will *not* be volume-mananged. Please check indexes.conf for configuration errors. 01-14-2020 00:58:27.995 +0000 WARN ProcessTracker - (child_17__Fsck) IndexConfig - idx=_telemetry Path coldPath='/opt/splunk/var/lib/splunk/_telemetry/colddb' (realpath '/mnt/local/hot/_telemetry/colddb') is inside volume=primary (path='/mnt/local/hot', realpath='/mnt/local/hot'), but does not reference that volume. Space used by coldPath will *not* be volume-mananged. Please check indexes.conf for configuration errors. 01-14-2020 00:58:28.008 +0000 WARN ProcessTracker - (child_17__Fsck) IndexConfig - idx=firedalerts Path coldPath='/opt/splunk/var/lib/splunk/firedalerts/colddb' (realpath '/mnt/local/hot/firedalerts/colddb') is inside volume=primary (path='/mnt/local/hot', realpath='/mnt/local/hot'), but does not reference that volume. Space used by coldPath will *not* be volume-mananged. Please check indexes.conf for configuration errors. 01-14-2020 00:58:28.042 +0000 WARN ProcessTracker - (child_17__Fsck) IndexConfig - idx=wineventlog Path homePath='/opt/splunk/var/lib/splunk/wineventlog/db' (realpath '/mnt/local/hot/wineventlog/db') is inside volume=primary (path='/mnt/local/hot', realpath='/mnt/local/hot'), but does not reference that volume. Space used by homePath will *not* be volume-mananged. Please check indexes.conf for configuration errors. indexes.conf # global settings [default] lastChanceIndex = lastchance malformedEventIndex = malformedevent [volume:primary] path = /mnt/local/hot maxVolumeDataSizeMB = 14000000 [volume:cold] path = /mnt/local/cold maxVolumeDataSizeMB = 58200000 [volume:_splunk_summaries] path = /mnt/local/hot maxVolumeDataSizeMB = 1000000 homePath = volume:primary/$_index_name/db coldPath = volume:cold/$_index_name/colddb thawedPath = /mnt/local/cold/$_index_name/thaweddb homePath.maxDataSizeMB = 2000000 maxWarmDBCount = 250 maxDataSize = auto enableDataIntegrityControl = true frozenTimePeriodInSecs = 188697600 [main] homePath = volume:primary/defaultdb/db coldPath = volume:cold/defaultdb/colddb coldToFrozenDir = /mnt/local/cold/frozen/defaultdb thawedPath = /mnt/local/cold/defaultdb/thaweddb maxDataSize = auto_high_volume frozenTimePeriodInSecs = 31536000 ... ... ... ... View more