Getting Data In

s3 add-on adding data

jonsaenz
Engager

Splunk is installed
s3 add-on is installed.

I have gone to data inputs and added the amazon s3 bucket we wanted.
However, for the life of me I can't figure out how to actually pull in the data from s3 to do anything with.

Can anyone tell me what I need to do?

Cheers.

Tags (3)

sbutto
Explorer

I have, from a Splunk python script, developed coldToFrozenPlusS3Uplaod.py that encrypts and uploads frozen buckets to S3.

It can be found here: https://github.com/marboxvel/Encrypt-upload-archived-Splunk-buckets

0 Karma

Splunk_rocks
Path Finder

Hi Sbuttio,
Im using your script here is sample one getting issues

import sys, os, gzip, shutil, subprocess, random, gnupg
import boto
import datetime
import time
import tarfile

applyLogging is a python script named applyLogging.py that exists at the same level of this script.

If the file applyLogging.py doesn't exist where this file is located, the import statement will fail.

sys.path.append(script_path)
import applyLogging

CHANGE THIS TO YOUR ACTUAL ARCHIVE DIRECTORY!!!

ARCHIVE_DIR = "/splunk/index/splunk/archiveindex"

ARCHIVE_DIR = os.path.join(os.getenv('SPLUNK_HOME'), 'frozenarchive')

script_path = '/opt/splunk/etc/apps/Encrypt-upload-archived-Splunk-buckets-master/coldToFrozenPlusS3Uplaod.py'
log_file_path = '/opt/splunk/var/log/splunk/'

gnu_home_dir = '' #where the gpg directory is. For example /home/s3/.gnupg/

gnu_home_dir = /home/splunkq/.gnupg

reciepient_email = '' #the email the gpg uses to encrypt the files

reciepient_email = xxyxy@gmail.com

Enabling the logging system

logger = applyLogging.get_module_logger(app_name='SplunkArchive',file_path=log_file_path)

all are same

0 Karma

Splunk_rocks
Path Finder

Can you please help me

0 Karma

khourihan_splun
Splunk Employee
Splunk Employee

Here is what you need to do:

$SPLUNK_HOME/etc/apps/s3/README/inputs.conf.spec
inputs.conf.spec

[s3://]

key_id =
* This is Amazon key ID.

secret_key =
* This is the secret key.

http://docs.splunk.com/Documentation/Splunk/6.0.1/AdvancedDev/ModInputsExample

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...