topic Re: How can one have visibility through searches of Data Integrity checks operations in Splunk? in Getting Data In

How can one have visibility through searches of Data Integrity checks operations in Splunk?

skender27 — Sat, 06 Jun 2020 23:25:49 GMT

Hi,

Me and my collegue are thinking of a report (then placed in a dashboard) about having visibility on data integrity check Splunk performs.
Once activated the stanza in indexes.conf and executed the Splunk comand: splunk check-integrity –index <index_name>, how can we in search time have some visualization about such activity?

Thanks,
Skender

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

gcusello — Tue, 29 Sep 2020 11:03:25 GMT

You have to do the following steps:
- create a script with your command,
- put it into your app's bin directory,
- create an input stanza in your inputs.conf

for example:
script file check_integrity.sh
splunk check-integrity –index index_to_check

inputs.conf
[script://$SPLUNK_HOME/etc/apps/yourapp/bin/check_integrity.sh]
disabled = 0
index = your_index
interval = 60.0
sourcetype = your_sourcetype
source = your_source

index_to_check is the index to check integrity
your_index is the index you create where you store the script results

Bye.
Giuseppe

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

skender27 — Tue, 20 Sep 2016 14:43:29 GMT

Hi Giuseppe,

I tried, but I get only indexed the Splunk comands and no output ("Total buckets checked... etc etc").
Since I am using Windows OS there should be smth to modify with the .bat file to get the standard output...

Is it possible to get the indexed events in the _audit index (and not a custom one)?

Thanks,
Skender

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

gcusello — Tue, 20 Sep 2016 17:01:07 GMT

you have to give the execution rights to the script: if you use operative commands like "query session" the script is correctly executed and output indexed in Splunk.
everyway, you could schedule om Windows the execution of the script as an administrator writing results in a txt file that you can take with Splunk.
Bye.
Giuseppe

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

skender27 — Fri, 14 Oct 2016 07:59:53 GMT

I understand.
Thanks again!

Skender

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

gurlest — Mon, 13 May 2019 18:37:57 GMT

We are trying to do something similar with a loop to catch all of our indexes:

#!bin/bash
for path in /opt/data/*/*; do splunk check-integrity -index ${path##*/} -verbose; done

However, I have noticed that the results of the command are only output to splunkd.log. I can find the output of the commands in the "_internal" index, but not in the index specified in inputs.conf

Do you have any guidance on how to get the results ingested into an index other than _internal?

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

potnuru — Tue, 02 Jun 2020 17:27:58 GMT

Hi @gurlest
Did you find the solution of the above query?
I am looking for the solution for the same query.

I need to capture the output of check-integrity command (which runs through script ) to specified index in Splunk.

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

gurlest — Wed, 30 Sep 2020 05:35:39 GMT

Yes, we did. We have been able to get it working like a champ. However, I would be a poor Splunk Admin if I didn't say this, as well: THIS IS HUGELY RESOURCE INTENSIVE. Ok. Now that that's out of the way... 🙂

Inputs > data_integrity_checks/local/inputs.conf

[script://./bin/integrity_checks.sh]
disabled = 0
interval = 3 10 * * 6
index = system_events
sourcetype = audit:integrity

Script > data_integrity_checks/bin/integrity_checks.sh

#!/bin/bash
IDX='/opt/splunk/etc/slave-apps/data_integrity_checks/local/dataintegrity_indexes.txt'
for index in $(<$IDX); do $SPLUNK_HOME/bin/splunk check-integrity -index $index -verbose 2>&1; done

The script runs based on the list of indexes on the dataintegrity_indexes.txt file, or you could just pass a list of index names.

Re: How can one have visibility through searches of Data Integrity checks operations in Splunk?

potnuru — Thu, 04 Jun 2020 15:20:22 GMT

@gurlest Thank you for the response, I will try the above given solution.