Knowledge Management
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Ask a Question

[SmartStore] How to check Summary Bucket ( Report acceleration or Data Model ) on Remote Store and also download a file from it?

rbal_splunk
Splunk Employee
Splunk Employee
‎04-29-2019 01:11 PM

I am seeing errors like below

04-19-2019 12:21:42.676 -0400 ERROR CacheManager - action=download, cacheId="ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9", status=failed, elapsed_ms=735
04-19-2019 12:21:41.945 -0400 INFO CacheManager - Cache miss for cache_id=ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9 oldest_evictable_bucket=1555690901.941656

How can I check if these report acceleration searches registered with a remote store?

Tags (1)
0 Karma
Reply

rbal_splunk
Splunk Employee
Splunk Employee
‎04-29-2019 01:17 PM

1) Here is example to see the content of a bucket.

/bin/splunk cmd splunkd rfs -- ls bucket:_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9

for full paths run: splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json

OR
You could also use the command like below provide you know the path to remote location

bin/splunk cmd  splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json

2)Command to fetch the content of bucket from the remote store

 /bin/splunk cmd splunkd -- getF bucket:_audit~70~D9F2C312-7136-415A-9E5B-579258D7F626/receipt.json <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626>

Where <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626>  is the output directory

OR you may also use 

/bin/splunk cmd splunkd rfs -- getF  volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json  <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9>

Where <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9>  is the directory

3)To list the content from the summary bucket
…/bin/splunk cmd splunkd rfs -- ls --starts-with volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/

size,name
3139,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx
4714,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv
75,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum
831,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json

To get a file for summary bucket from remote location the below is cli example.

4)Here is the command to retrieve the summary bucket.
/bin/splunk cmd splunkd rfs -- getF volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json testrbal

cat ././testrbal/receipt.json

{"objects":[{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx","size":3139},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv","size":4714},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum","size":75},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/done","size":0}],"manifest":{"summary_path":"/home/rbal/742p2/var/lib/splunk/audit/datamodel_summary/79_D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF/DM_search_internal_audit_logs","summary_earliest_time":"1556560545","summary_latest_time":"1556560623","summary_size_on_disk":"16384"},"user_data":{"uploader_guid":"D9F2C312-7136-415A-9E5B-579258D7F626","content_hash":"D4AF95215FF87FD3632AF3B04027504B24A30829CFA545689B0C583C39D5B54A"}}[root@centos65-64sup02 db]#
0 Karma
Reply
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...