s3 add-on adding data

[jonsaenz]

Splunk is installed
s3 add-on is installed.

I have gone to data inputs and added the amazon s3 bucket we wanted.
However, for the life of me I can't figure out how to actually pull in the data from s3 to do anything with.

Can anyone tell me what I need to do?

Cheers.

[sbutto]

I have, from a Splunk python script, developed coldToFrozenPlusS3Uplaod.py that encrypts and uploads frozen buckets to S3.

It can be found here: https://github.com/marboxvel/Encrypt-upload-archived-Splunk-buckets

[Splunk_rocks]

Hi Sbuttio,
Im using your script here is sample one getting issues

import sys, os, gzip, shutil, subprocess, random, gnupg
import boto
import datetime
import time
import tarfile

applyLogging is a python script named applyLogging.py that exists at the same level of this script.

If the file applyLogging.py doesn't exist where this file is located, the import statement will fail.

sys.path.append(script_path)
import applyLogging

CHANGE THIS TO YOUR ACTUAL ARCHIVE DIRECTORY!!!

ARCHIVE_DIR = "/splunk/index/splunk/archiveindex"

ARCHIVE_DIR = os.path.join(os.getenv('SPLUNK_HOME'), 'frozenarchive')

script_path = '/opt/splunk/etc/apps/Encrypt-upload-archived-Splunk-buckets-master/coldToFrozenPlusS3Uplaod.py'
log_file_path = '/opt/splunk/var/log/splunk/'

gnu_home_dir = '' #where the gpg directory is. For example /home/s3/.gnupg/

gnu_home_dir = /home/splunkq/.gnupg

reciepient_email = '' #the email the gpg uses to encrypt the files

reciepient_email = xxyxy@gmail.com

Enabling the logging system

logger = applyLogging.get_module_logger(app_name='SplunkArchive',file_path=log_file_path)

all are same

[Splunk_rocks]

Can you please help me

[khourihan_splun]

Here is what you need to do:

$SPLUNK_HOME/etc/apps/s3/README/inputs.conf.spec
inputs.conf.spec

[s3://]

key_id =
* This is Amazon key ID.

secret_key =
* This is the secret key.

http://docs.splunk.com/Documentation/Splunk/6.0.1/AdvancedDev/ModInputsExample