How to test data integrity?

[aatik5u]

Hello there,

Here is the context, I have a Splunk test environment, one indexer one search head and one forwarder. I'm in charge of finding a way to guarantee the integrity of the events available on the search head.

My first question is, how to test data integrity control? I implemented it based on Splunk documentation, I tried to run Splunk clean and use the delete command (now I know that the event is not deleted from the index using delete),  and I edited the log files. But the integrity check is always successful. In an other words, in what case does the integrity check becomes unsuccessful? 

My second question is, I changed the auth.log file, I mean this can be super dangerous but Splunk just displays both events, before the edit and after the edit. How can I use Splunk to detect such changes?

Any help would be appreciated, thank you so much for your time 

[gcusello · Accepted solution]

Hi @aatik5u,

as you can read at https://docs.splunk.com/Documentation/Splunk/9.0.1/Security/Dataintegritycontrol and https://docs.splunk.com/Documentation/Splunk/9.0.1/Admin/ChecktheintegrityofyourSplunksoftwarefiles ,Data integrity is check on Indexers (that contain data) and not on Search Heads.

So deleting an event by CLI you don't modify Data integrity because the events remain in the index with a deleted status.

If you want to check Data Integrity, you have to go in the folder of one index with Data Integrity Check enabled and manually modify some row data.

Then performing the Integrity Check you'll have an error.

Ciao.

Giuseppe

[aatik5u]

Hello @gcusello 

Thank you very much for the answer I really appreciate it 🙂

May be I wasn't clear but yeah I totally used data integrity control on the indexer but thank you for the remarque.

I did what you said and I do have an unsuccessful security check, thank you very much for that. but since the files in raw data are either .dat or .zst files I can't really understand what m deleting. is there a way to understand what i'm deleting ?

thank you again 

[gcusello]

Hi @aatik5u,

raw data are in $SPLUNK_DB/<index>/colddb/db_xxxxxxx_xxxxx_x/rawdata or in $SPLUNK_DB/<index>/db/db_xxxxxxx_xxxxx_x/rawdata

Ciao.

Giuseppe