Getting Data In

Archiving and signing at the same time

Mahieu
Communicator

Hey guys,

I'm new on the Splunk planet.
I'm trying to find a script that would :
- Send archive data to a nas location when an index is full (as soon as an index is full, without the need of a human intervention)
- Sign this data automatically before it sends it to the distant location (i've generated keys already for IT data signing which works well, i guess i can use the same keys)

I've had a look at the sample script in the bin directory but i'm not "fluent" in python unfortunately.

If there's another solution than a script to do that, i'd work for me as well.

Thanks a lot in advance for your help.

M

Tags (3)
1 Solution

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

View solution in original post

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

mw
Splunk Employee
Splunk Employee

It sounds like you just want to specify the coldToFrozenDir. If you're signing the data already, that should be preserved during the migration.

http://www.splunk.com/base/Documentation/latest/admin/Automatearchiving#Let_Splunk_archive_the_data_...

If you do need a script, it doesn't have to be python, but the example coldToFrozenExample.py is a good start.

0 Karma

Mahieu
Communicator

I tried the following :

I created a brand new index called test, max size is set to 1 Mo.
In indexes.conf :
[test]
coldtoFrozenScript = "/bin/sh" "/opt/splunk/bin/archive_signing.sh"

In opt/splunk/bin, my archive_signing.sh script is :

!/bin/sh

echo "script execute" > /root/archive.txt
gzip -c $1/*.tsidx > /mnt/tmp/test_archive.gz
signtool -s /mnt/tmp/test_archive.gz

Then i indexed a file that is bigger than 1 Mo.
The script is not launched, no archive.txt file is created in the root directory.
I can see my logs with the search app. current size of the index is 2 Mo.

Any ideas?

Thx

M

Mahieu
Communicator

I could use an archive path, no problem about that.
Still, i'm not sure how to make it work together with the signtool.

Mahieu
Communicator

I'm not signing the data at the moment. I just need to sign it at the moment it is sent to my archive path.

I must have missed something here...

M

Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...