Getting backup job failed messages

[ansif]

My license master (which is used as dmc,deployment server,Cluster Master,Deployer) is throwing a lot of messages like below:

Scheduled backup job failed.The next scheduled backup job will run in an hour.
12/5/2017, 3:46:39 PM

Backup job "Default Scheduled Backup" has failed. Error: expected string or buffer
12/5/2017, 3:46:35 PM

Help me to identify and resolve this issue.

[twollenslegel_s]

This is a known issue for ITSI 3.0.X
2017-07-03 ITOA-8115 ITSI app throws Errors during installation ITSI components on License server

http://docs.splunk.com/Documentation/ITSI/3.0.1/ReleaseNotes/Knownissues

[aditya0473]

I had the same issue. Mine was a dedicated license server.

I update inputs.conf in SA-ITOA/local/ dir with below settings.

[itsi_backup_restore://itsi_backup_restore]
disabled = 1

[itsi_scheduled_backup_caller://itsi_scheduled_backup]
disabled = 1

This solved the issue. As this is a license server there is no need of backup here.
Hope this helps.

NOTE: If license server is same as search head do not update these settings.

[ansif]

Thanks @aditya0473 ,do you know where exactly this backup needed? Indexer? Search head?

[aditya0473]

Where ever ITSI is setup. Mostly on search heads. These settings are by default enabled, so where ITSI is up and running it shouldn't throw any error.

[nickhills]

I presume you are running ITSI?

I think I would be tempted to raise a ticket with Splunk for that issue, as it seems it should be configured out of the box
https://docs.splunk.com/Documentation/ITSI/3.0.0/Configure/BackupandRestoreITSIconfig

However, the first thing I would check is that the KV store is running ok.

[ansif]

Hi nickhillscpl,

As per the document http://docs.splunk.com/Documentation/ITSI/3.0.0/Configure/InstallSplunkITServiceIntelligence
I have installed only A-ITSI-Licensechecker, SA-ITOA, and SA-UserAccess on License master.

Please find the below kvstore status:

[root@xxxxxxxxxxxxx ~]# /opt/splunk/bin/splunk show kvstore-status

 This member:
                                     date : Wed Dec  6 04:22:09 2017
                                  dateSec : 1512534129.847
                                 disabled : 0
                                    guid : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
                     oplogEndTimestamp : Wed Dec  6 04:21:10 2017
                     oplogEndTimestampSec : 1512534070
                      oplogStartTimestamp : Sun Dec  3 23:40:04 2017
                     oplogStartTimestampSec : 1512344404
                                     port : xxxx
                               replicaSet : xxxxxxx-xxxx-xxxx-xxxxxxxxxxx
                        replicationStatus : KV store captain
                               standalone : 1
                                   status : ready

 KV store members:
        127.0.0.1:xxxx
                            configVersion : 1
                             electionDate : Wed Nov 15 13:49:35 2017
                          electionDateSec : 1510753775
                              hostAndPort : 127.0.0.1:xxxx
                               optimeDate : Wed Dec  6 04:21:10 2017
                            optimeDateSec : 1512534070
                        replicationStatus : KV store captain
                                   uptime : 1780355

Does License Master requires to run kvstore and backup jobs?