topic Re: Archiving and signing at the same time in Getting Data In

Archiving and signing at the same time

Mahieu — Fri, 29 Jul 2011 13:22:27 GMT

Hey guys,

I'm new on the Splunk planet.
I'm trying to find a script that would :
- Send archive data to a nas location when an index is full (as soon as an index is full, without the need of a human intervention)
- Sign this data automatically before it sends it to the distant location (i've generated keys already for IT data signing which works well, i guess i can use the same keys)

I've had a look at the sample script in the bin directory but i'm not "fluent" in python unfortunately.

If there's another solution than a script to do that, i'd work for me as well.

Thanks a lot in advance for your help.

Re: Archiving and signing at the same time

mw — Fri, 29 Jul 2011 15:51:12 GMT

It sounds like you just want to specify the coldToFrozenDir. If you're signing the data already, that should be preserved during the migration.

http://www.splunk.com/base/Documentation/latest/admin/Automatearchiving#Let_Splunk_archive_the_data_for_you

If you do need a script, it doesn't have to be python, but the example coldToFrozenExample.py is a good start.

Re: Archiving and signing at the same time

Mahieu — Wed, 03 Aug 2011 14:39:48 GMT

I'm not signing the data at the moment. I just need to sign it at the moment it is sent to my archive path.

I must have missed something here...

Re: Archiving and signing at the same time

Mahieu — Wed, 03 Aug 2011 14:50:25 GMT

I could use an archive path, no problem about that.
Still, i'm not sure how to make it work together with the signtool.

Re: Archiving and signing at the same time

Mahieu — Mon, 28 Sep 2020 09:46:39 GMT

I tried the following :

I created a brand new index called test, max size is set to 1 Mo.
In indexes.conf :
[test]
coldtoFrozenScript = "/bin/sh" "/opt/splunk/bin/archive_signing.sh"

In opt/splunk/bin, my archive_signing.sh script is :

!/bin/sh

echo "script execute" > /root/archive.txt
gzip -c $1/*.tsidx > /mnt/tmp/test_archive.gz
signtool -s /mnt/tmp/test_archive.gz

Then i indexed a file that is bigger than 1 Mo.
The script is not launched, no archive.txt file is created in the root directory.
I can see my logs with the search app. current size of the index is 2 Mo.

Any ideas?

Thx

Re: Archiving and signing at the same time

Mahieu — Mon, 28 Sep 2020 11:56:45 GMT

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH