After migrating an app to a new Splunk server searching on an account w/ SSO is failing

[abeeber_2]

Hi Folks,

search in panel fails with SSO account with admin role, but works with local admin and power user account

Working on an app migration to a new splunk server and am running in to a problem with couple of views that wont populate correctly.

Some of the panels fail with an Search process did not exit cleanly, exit_code=-1, description="exited with code -1". Please look in search.log for this peer in the Job Inspector for more info.

When I look at the internal logs, I don't see any errors except for a GET ..../configs/conf-visualations?output_mode=json&search=disabled.

Using my SSO account, when I run the search in a separate window. But if I hit enter again, the search works.

If I use local admin or a test account with power role, the panel/xml view works.

The app also works on the original search head with the same SSO account and same roles.

Any thoughts/suggestions on where to look?

Thx

[abeeber_2]

More intel.

Turns out there is a bug/fix in Splunk 6.4.5 where they shortened a temp file from 30 characters to 16.

We installed 6.4.9 on the index tier and the problem went away.

[abeeber_2]

Here is my after action report.

It turns the problem was due to the index tier running Windows 2008R2, which has a character limit.

Using SSO AD accounts that have FQDN meant the hashed value of the search sid exceeded the character limit of the server. This was identified by using the | history command to see the difference.

[abeeber_2]

More info.

SHC cluster is running on Linux. Indexers (to be migrated) are on Windows.

Search Panels have joins in them.

The error from the search.log is can not find runtime.csv and info.csv

Windows pathing for the remote search is below 260 characters.

[abeeber_2]

another update...

more analysis indicates a problem due to windows and character length limitations. Windows index servers are on Win2k08R2. Will test again when data sources migrated to new RHEL index servers