Dell EMC Isilon App - Adding multiple clusters is failing

[raynold_peterso]

Ok, 

We had a need to monitor our Isilon Clusters.  I looked around and loe and behold, there's an app or that!

I downloaded the Dell EMC Isilon App, v2.5.0, and the Add-on, v2.7.0.  All went well, I followed the instructions and I had my first of five clusters added, no problem.   My second thru fourth addition worked flawlessly.   Then came my fifth and LAST Cluster. 

All of my clusters have the same userid/password for authorization.  The only thing I changed was the IP address.   

I received the following:

What in the wide world of sports is "list index out of range"?

I have tried everything.  I have stopped and restarted splunk.  I  have removed that IP from the config files, stopped and restarted splunk.  And my only response is this message.

The isilonappsetup.conf is getting updated with the device.  The password.conf is NOT getting the update for the encrypted password.

Where is the fix?

Any help at this point would be great!

[xguerraz]

Hello

I ran into the same error message while trying to setup the add-on for onefs 9.1.0.8

the issue is located in the setup script as you can see in this log from /opt/splunk/var/log/splunk/python.log

2021-08-24 17:46:19,721 ERROR list index out of range
Traceback (most recent call last):
File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 199, in dispatch
methodOutput = method()
File "/opt/splunk/etc/apps/TA_EMC-Isilon/bin/isilon_setup.py", line 155, in handle_POST
major_version = release_version.split('v')[1].split('.')[0]
IndexError: list index out of range

When i tried to connect to https://cluster:8080/platform/1/cluster/config and search for "release:" . it has this format

"release" : "9.1.0.8"

I guess the python script expects a format like "release" : "v9.1.0.8" so i commented line 155 and replaced it with major_version = release_version.split('.')[0] and the setup now works as expected

The heavy forwarder is on version 8.2 and the add-on on version 2.7

[raghavnaidu]

spot on !! Thank you.

[Tyrian01]

Just updated one of our clusters to 9.1.0.8 - No issues with the Splunk app at all

[raynold_peterso]

Thanks for the response.  We have 4 clusters defined currently,  I wonder if there is a limit on the number of clusters you can add.    You know, a config file setting to only allow 4 currently.  

[Tyrian01]

No changes needed to be made to exceed 4 clusters. Perhaps check the permissions on the config files in TA_EMC-Isilon/local folder to ensure you can write to them.  

Look at your splunkd.log for detailed errors on why it's failing. 

Another option is to copy the lines of check from an existing host & paste/change IP to the new cluster you're trying to add directly in the TA_EMC-Isilon/local/inputs.conf file:

Eg below:

[isilon://10.xxx.xxx.xxx::/platform/1/statistics/current?substr=true&keys=node.clientstats.active&devid=all]
endpoint = https://10.117.66.97:8080/platform/1/statistics/current?substr=true&keys=node.clientstats.active&dev...
sourcetype = emc:isilon:rest
interval = 120
index = isilon
response_handler = IsilonResponseHandler

The new cluster will need to be added to the passwords.conf & isilonappsetup.conf (the password may not work if it's unencrypted but worth a try).

[Roy_9]

Hello @raynold_peterso 

When i am trying to configure the app, i am seeing the index isilon doesn't exist error where i have already created the isilon index, i tried it on our HF and SH but seeing the same error. Any thoughts on this?

[wellsjp]

Hello @Roy_9 

Installing this TA now and ran into the same issue.  I had to create an indexes.conf on the HF for the TA to be able to verify the index exists.  As long as your outputs.conf for the HF has outputs.conf configured correctly, it will not write to the local index.

Outputs.conf:

[indexAndForward]
index = false

I just created TA_EMC-Isilon/local/indexes.conf to keep it with the app. 

[davvik]

Did you just have to create an empty index.conf or did it have to contain som info?

[wellsjp]

It has to be an actual index definition.  And splunk will create the directory structure on the HF but will not write any data there.

indexes.conf:

[storage]
homePath = $SPLUNK_DB/storage/db
coldPath = $SPLUNK_DB/storage/colddb
thawedPath = $SPLUNK_DB/storage/thaweddb
maxDataSize = auto

And the empty directories it creates:

$ du -h storage
0 storage/db/GlobalMetaData
8.0K storage/db
0 storage/colddb
0 storage/thaweddb
0 storage/datamodel_summary
8.0K storage

[raynold_peterso]

After speaking with the Isilon fellows, the cluster I am attempting to add in is a oneFS v9 device.  I saw in the documentation that v8.1.x is the last supported oneFS that is documented.

I am not sure that makes a difference but thought you should at least have that information.

[Tyrian01]

Thanks for the info - we have 6 Isilon clusters (8.2.2) in Splunk but am starting the oneFS 9.1.0.8 upgrades this week. Will see why the app isn't compatible after the upgrade & see if it can be modified to work with 9.x OneFS. 
Will let you know how it goes.

[aparicio]

Hello got any results after an update?

Thanks

[Tyrian01 · Accepted solution]

Sorry I forgot about this post. My Splunk app worked fine after the OneFS upgrade to 9.1.0.8 - no action required.

[aparicio]

Hi, raynold_peterso.

I am using OneFS version 9.1.0.0.
Do not create passwords.conf and last_session_call_info.pos files.

I think that's why it appears -> list index out of range

thanks