Thanks Stefano, that is exactly correct! File transfer happens in the following steps: 1) Shuttl will move the file from the colddb dir to a local tmp (defined via localArchiverDir in conf/archiver.xml) 2) Shuttl will then copy the file from localArchiverDir to a tmp area at the destination 3) When the transfer completes, it then will move the file from that area to the final destination 4) Files in the local tmp are deleted when transfers are successful This mechanism gives us the necessary reliability to handle crashes, network interruptions, etc. ... View more

@sbutto here is my inputs in the script CHANGE THIS TO YOUR ACTUAL ARCHIVE DIRECTORY!!! ARCHIVE_DIR = "/splunk/index/splunk/archiveindex" ARCHIVE_DIR = os.path.join(os.getenv('SPLUNK_HOME'), 'frozenarchive') script_path = '/opt/splunk/etc/apps/Encrypt-upload-archived-Splunk-buckets-master/coldToFrozenPlusS3Uplaod.py' log_file_path = '/opt/splunk/etc/apps/Encrypt-upload-archived-Splunk-buckets-master/' gnu_home_dir = '' #where the gpg directory is. For example /home/s3/.gnupg/ gnu_home_dir = ' /home/splunkqa/ ' reciepient_email = '' #the email the gpg uses to encrypt the files reciepient_email = ' xyz@@@domain.com ' Enabling the logging system logger = applyLogging.get_module_logger(app_name='SplunkArchive',file_path=log_file_path) Finding out the epoch value at four month ago so we can copmare the bucket timestamp against it. FYI - my splunk was setup standalone indexr host splunk is installed under /opt/splunk my indexr is configured under /splunk/index ... View more

thanks! that did it. conveniently, it turns out you can store the whole transaction in the summary index without even having to try and squish all of the desired fields onto one line. eventtype=cisco_esa [search | fields mid] | transaction fields=mid mvlist=t | collect index=summary creates the index, and running a search on "index=summary" actually returns the whole transaction for all of the search matches. unfortunately it seems like the field extractions are lost in the process. i need to see about getting those rerun on the index somehow. but at least all of the data is there ... View more

Very helpful post. However, when I run the '_internal call...' command, I do return data, but I find no epoch times listed in the "s:key name=" lines. Can you provide the specific line I'm looking for? Thanks ... View more