[SmartStore] How to check Summary Bucket ( Report acceleration or Data Model ) on Remote Store and also download a file from it?

[rbal_splunk]

I am seeing errors like below

04-19-2019 12:21:42.676 -0400 ERROR CacheManager - action=download, cacheId="ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9", status=failed, elapsed_ms=735
04-19-2019 12:21:41.945 -0400 INFO CacheManager - Cache miss for cache_id=ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9 oldest_evictable_bucket=1555690901.941656

How can I check if these report acceleration searches registered with a remote store?

[rbal_splunk]

1) Here is example to see the content of a bucket.

/bin/splunk cmd splunkd rfs -- ls bucket:_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9

for full paths run: splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json

OR
You could also use the command like below provide you know the path to remote location

bin/splunk cmd  splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json

2)Command to fetch the content of bucket from the remote store

 /bin/splunk cmd splunkd -- getF bucket:_audit~70~D9F2C312-7136-415A-9E5B-579258D7F626/receipt.json <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626>

Where <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626>  is the output directory     

OR you may also use

/bin/splunk cmd splunkd rfs -- getF  volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json  <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9>

Where <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9>  is the directory

3)To list the content from the summary bucket
…/bin/splunk cmd splunkd rfs -- ls --starts-with volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/

size,name
3139,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx
4714,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv
75,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum
831,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json

To get a file for summary bucket from remote location the below is cli example.

4)Here is the command to retrieve the summary bucket.
/bin/splunk cmd splunkd rfs -- getF volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json testrbal

cat ././testrbal/receipt.json

{"objects":[{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx","size":3139},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv","size":4714},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum","size":75},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/done","size":0}],"manifest":{"summary_path":"/home/rbal/742p2/var/lib/splunk/audit/datamodel_summary/79_D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF/DM_search_internal_audit_logs","summary_earliest_time":"1556560545","summary_latest_time":"1556560623","summary_size_on_disk":"16384"},"user_data":{"uploader_guid":"D9F2C312-7136-415A-9E5B-579258D7F626","content_hash":"D4AF95215FF87FD3632AF3B04027504B24A30829CFA545689B0C583C39D5B54A"}}[root@centos65-64sup02 db]#