I have a search head cluster and I set up SSO with reverse proxy authentication and set scripted authorization using a Python script. This is working very fine. This setup is within intranet.
Later I added another reverse proxy which is exposed to internet and internally route to intranet server. This also works andSplunk homepage does open after successful authentication and authorization.
However, none of my searches are working. For every search, the message says "Server Error".
In short, Splunk searches are working when it is opened using direct Splunk server URL and using intranet URL. But, it none of the searches are working using internet URL.
I checked splunkd.log and found below message:
ERROR UiAuth - Request from <SH_IP_address> to "/splunk/en-US/splunkd/__raw/servicesNS/<user_name>/search/search/search/jobs" failed CSRF validation -- expected "4647222401877220", but instead cookie had "4647222401877220" and header had ""
SH_IP_address - User_Name [Date Time] "POST /splunk/en-US/splunkd/__raw/servicesNS/User_Name/search/search/jobs HTTP/1.1" 401 104 "https://Internet_URL/en-US/app/search/search?q=search%20index%3D_internal" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_4 like Mac OS X) ...." - 53f012485f2fb9d 0ms
Sample success message in splunkd_ui_access.log from Intranet URL
SH_IP_address - User_Name [Date Time] "POST /splunk/en-US/splunkd/__raw/servicesNS/nobody/search/search/jobs/1454684456.680_EB62A3AA-75CD-40-A71C-DA6DDB53F181/control HTTP/1.1" 200 59 "https://Intranet_URL/en-US/app/search/search?q=search%20index%3D_internal&display.page.search.mode=smart&earliest=&latest=&sid=14546456.680_EB62A3AA-75CD-4600-A71C-DA6DDB53F181" "Mozilla/5.0 (Windows NT 6.1; WOW64) ...." - 1e371eb6 3ms
How should I rectify the problem with internet URL?
Were you able to fix this?
I've got the exact same issue :
failed CSRF validation -- expected "123456", but instead cookie had "123456" and header had ""
I've got Splunk 7.2.7 behind a Reverse proxy.
We have a similar setup and we are using IIS 8.5 as Reverse Proxy....But only redirection happens and SSO didn't work. We got see only Splunk login page. No value for X-Remote-User variable when checked in Splunk SSO debug page. Can you please share the steps to configure SSO with IIS reverse proxy authentication?
How are you controlling the route persistence from the reverse proxy to the search head cluster? Make sure you follow the guide here (http://docs.splunk.com/Documentation/Splunk/6.3.3/DistSearch/UseSHCwithloadbalancers)
Also how have you configured the load balancer? Are you doing any header / cookie rewriting?