Getting Data In

s3 add-on adding data

jonsaenz
Engager

Splunk is installed
s3 add-on is installed.

I have gone to data inputs and added the amazon s3 bucket we wanted.
However, for the life of me I can't figure out how to actually pull in the data from s3 to do anything with.

Can anyone tell me what I need to do?

Cheers.

Tags (3)

sbutto
Explorer

I have, from a Splunk python script, developed coldToFrozenPlusS3Uplaod.py that encrypts and uploads frozen buckets to S3.

It can be found here: https://github.com/marboxvel/Encrypt-upload-archived-Splunk-buckets

0 Karma

Splunk_rocks
Path Finder

Hi Sbuttio,
Im using your script here is sample one getting issues

import sys, os, gzip, shutil, subprocess, random, gnupg
import boto
import datetime
import time
import tarfile

applyLogging is a python script named applyLogging.py that exists at the same level of this script.

If the file applyLogging.py doesn't exist where this file is located, the import statement will fail.

sys.path.append(script_path)
import applyLogging

CHANGE THIS TO YOUR ACTUAL ARCHIVE DIRECTORY!!!

ARCHIVE_DIR = "/splunk/index/splunk/archiveindex"

ARCHIVE_DIR = os.path.join(os.getenv('SPLUNK_HOME'), 'frozenarchive')

script_path = '/opt/splunk/etc/apps/Encrypt-upload-archived-Splunk-buckets-master/coldToFrozenPlusS3Uplaod.py'
log_file_path = '/opt/splunk/var/log/splunk/'

gnu_home_dir = '' #where the gpg directory is. For example /home/s3/.gnupg/

gnu_home_dir = /home/splunkq/.gnupg

reciepient_email = '' #the email the gpg uses to encrypt the files

reciepient_email = xxyxy@gmail.com

Enabling the logging system

logger = applyLogging.get_module_logger(app_name='SplunkArchive',file_path=log_file_path)

all are same

0 Karma

Splunk_rocks
Path Finder

Can you please help me

0 Karma

khourihan_splun
Splunk Employee
Splunk Employee

Here is what you need to do:

$SPLUNK_HOME/etc/apps/s3/README/inputs.conf.spec
inputs.conf.spec

[s3://]

key_id =
* This is Amazon key ID.

secret_key =
* This is the secret key.

http://docs.splunk.com/Documentation/Splunk/6.0.1/AdvancedDev/ModInputsExample

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...