Getting Data In

Archiving and signing at the same time

Mahieu
Communicator

Hey guys,

I'm new on the Splunk planet.
I'm trying to find a script that would :
- Send archive data to a nas location when an index is full (as soon as an index is full, without the need of a human intervention)
- Sign this data automatically before it sends it to the distant location (i've generated keys already for IT data signing which works well, i guess i can use the same keys)

I've had a look at the sample script in the bin directory but i'm not "fluent" in python unfortunately.

If there's another solution than a script to do that, i'd work for me as well.

Thanks a lot in advance for your help.

M

Tags (3)
1 Solution

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

View solution in original post

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

mw
Splunk Employee
Splunk Employee

It sounds like you just want to specify the coldToFrozenDir. If you're signing the data already, that should be preserved during the migration.

http://www.splunk.com/base/Documentation/latest/admin/Automatearchiving#Let_Splunk_archive_the_data_...

If you do need a script, it doesn't have to be python, but the example coldToFrozenExample.py is a good start.

0 Karma

Mahieu
Communicator

I tried the following :

I created a brand new index called test, max size is set to 1 Mo.
In indexes.conf :
[test]
coldtoFrozenScript = "/bin/sh" "/opt/splunk/bin/archive_signing.sh"

In opt/splunk/bin, my archive_signing.sh script is :

!/bin/sh

echo "script execute" > /root/archive.txt
gzip -c $1/*.tsidx > /mnt/tmp/test_archive.gz
signtool -s /mnt/tmp/test_archive.gz

Then i indexed a file that is bigger than 1 Mo.
The script is not launched, no archive.txt file is created in the root directory.
I can see my logs with the search app. current size of the index is 2 Mo.

Any ideas?

Thx

M

Mahieu
Communicator

I could use an archive path, no problem about that.
Still, i'm not sure how to make it work together with the signtool.

Mahieu
Communicator

I'm not signing the data at the moment. I just need to sign it at the moment it is sent to my archive path.

I must have missed something here...

M

Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...