Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- Does Splunk recognize when buckets are deleted?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am doing a simple recovery test and deleted some warm buckets, but Splunk doesn't seem to even realize anything is wrong. Is this normal?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From Splunk Support:
"Splunk assumes that you are doing this on-purpose and therefore would not send any WARN/ERROR events.
The only reason you would alert is if a bucket were corrupt or never made it. Once it was there and you deleted it, from Splunk's perspective, everything was functioning."
That doesn't seem like a sound process to me, but that's the explanation thus far.
UPDATE June 14: I've done some more testing and I just can't accept the outcome. Splunk is essentially a database and as an old
Oracle DBA I would expect/assume that it has some self-awareness of its integrity. I'm going to ask for this to be considered a defect.
UPDATE June 30: Support is going to perform some testing and submit an enhancement request.
UPDATE July 06: Support submitted enhancement request SPL-123789
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From Splunk Support:
"Splunk assumes that you are doing this on-purpose and therefore would not send any WARN/ERROR events.
The only reason you would alert is if a bucket were corrupt or never made it. Once it was there and you deleted it, from Splunk's perspective, everything was functioning."
That doesn't seem like a sound process to me, but that's the explanation thus far.
UPDATE June 14: I've done some more testing and I just can't accept the outcome. Splunk is essentially a database and as an old
Oracle DBA I would expect/assume that it has some self-awareness of its integrity. I'm going to ask for this to be considered a defect.
UPDATE June 30: Support is going to perform some testing and submit an enhancement request.
UPDATE July 06: Support submitted enhancement request SPL-123789
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What does |dbinspect index= return for these buckets? Splunk should eventually log an error message since there should be metadata associated with the deleted buckets, but you will have data gaps since the raw data is deleted.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I deleted bucket ids 32-34 and 37-39 and dbinspect only shows results for 35-36, so it is still unaware that anything is missing.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Splunk will perform bucket fixups periodically and find the bucket no longer exists. At which time it will log a message or two or three and then remove the bucket from the manifest.
See index=_internal log_level=warn* OR log_level=err*. The events should occur in less than 24 hours after the manual removal. Searches will just have "holes" in the data if a searchable copy of the bucket doesnt exist.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That search was the first thing I checked, but it had nothing about these buckets.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Probably has to with the log verbosity on BucketMover or something. I'd file a low priority ticket with support if you're THAT interested.
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Announcing Modern Navigation: A New Era of Splunk User Experience
Modernize your Splunk Apps – Introducing Python 3.13 in Splunk
Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...
