What are the .sentinel files in $SPLUNK_DB for?

[mctester]

I was moving some buckets around to make some space on my main storage volume. I know the db_* directories are complete buckets and can be moved around easily, but I noticed there's also a *.sentinel file for every bucket.

1) What does this file do?

2) Do I need to move it around with the bucket?

3) If I drop a bucket into another instance to read the data there, does that instance need the .sentinel file too?

[Lowell]

Just to follow up on jrodmans comment about stale sentinel files. Perhaps some one will find this useful.

Here is a short script I wrote which searches your index folder for .sentinel files without a matching bucket. You can use the output to quickly remove these files (or event have them be removed directly by uncommenting a line in the script).

---

Code listing for lone_sentinel.py:

#!/usr/bin/python
import os

SENTINEL = ".sentinel"

def find_sentinels(path):
    for (root, dirs, files) in os.walk(path):
        if "rawdata" in dirs:
            dirs.remove("rawdata")
        for fn in files:
            if fn.endswith(SENTINEL):
                yield os.path.join(root, fn)

def find_lone_sentinels(path):
    for sentinel in find_sentinels(path):
        bucket = sentinel[:-len(SENTINEL)]
        if not os.path.isdir(bucket):
            print sentinel
            # To remove these files, uncomment the next line
            #os.unlink(sentinel)

if __name__ == '__main__':
    import sys
    if len(sys.argv) > 2:
        path = sys.arvg[1]
    else:
        path = os.environ["SPLUNK_DB"]
    find_lone_sentinels(path)

[jrodman]

Woohoo, generators.

[jrodman · Accepted solution]

The .sentinel files are used on Splunk for win32/win64 as part of the locking strategy, to ensure exclusive access to buckets for certain types of modifications. Windows doesn't allow locking calls on directories with the same semantics as we desired.

They are mostly useless on a unix-based system, but in the interests of the index data format being able to traverse platforms are produced nonetheless.

Best practice is to move the .sentinel with the bucket it goes with.

[jrodman]

Sure, but I don't like leaving stale ones lying around.

[gkanapathy]

Note that if Splunkd is down (which it should be if you're moving buckets around manually), the .sentinel files don't matter at all, even on Windows. You can delete them or not, and they will be recreated as necessary. Note that they are always just empty (zero-byte) files.