Deployment Architecture

I have search head clustering and SSO set up with reverse proxy authentication, but why do my searches produce "Server Error"?

ishaanshekhar
Communicator

I have a search head cluster and I set up SSO with reverse proxy authentication and set scripted authorization using a Python script. This is working very fine. This setup is within intranet.

Later I added another reverse proxy which is exposed to internet and internally route to intranet server. This also works andSplunk homepage does open after successful authentication and authorization.

However, none of my searches are working. For every search, the message says "Server Error".

In short, Splunk searches are working when it is opened using direct Splunk server URL and using intranet URL. But, it none of the searches are working using internet URL.

I checked splunkd.log and found below message:

ERROR UiAuth - Request from <SH_IP_address> to "/splunk/en-US/splunkd/__raw/servicesNS/<user_name>/search/search/search/jobs" failed CSRF validation -- expected "4647222401877220", but instead cookie had "4647222401877220" and header had ""

splunkd_ui_access.log:

SH_IP_address - User_Name [Date Time] "POST /splunk/en-US/splunkd/__raw/servicesNS/User_Name/search/search/jobs HTTP/1.1" 401 104 "https://Internet_URL/en-US/app/search/search?q=search%20index%3D_internal" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_4 like Mac OS X) ...." - 53f012485f2fb9d 0ms

Sample success message in splunkd_ui_access.log from Intranet URL

SH_IP_address - User_Name [Date Time] "POST /splunk/en-US/splunkd/__raw/servicesNS/nobody/search/search/jobs/1454684456.680_EB62A3AA-75CD-40-A71C-DA6DDB53F181/control HTTP/1.1" 200 59 "https://Intranet_URL/en-US/app/search/search?q=search%20index%3D_internal&display.page.search.mode=smart&earliest=&latest=&sid=14546456.680_EB62A3AA-75CD-4600-A71C-DA6DDB53F181" "Mozilla/5.0 (Windows NT 6.1; WOW64) ...." - 1e371eb6 3ms

How should I rectify the problem with internet URL?

Thanks
Ishaan

davietch
Path Finder

Hi,

Were you able to fix this?
I've got the exact same issue :

failed CSRF validation -- expected "123456", but instead cookie had "123456" and header had ""

I've got Splunk 7.2.7 behind a Reverse proxy.

0 Karma

deepashri_123
Motivator

Hi,

I am facing the same issue , and i have a single search head. Did you find any solution?

0 Karma

JHudson_CVX
New Member

Can you share you email Id? I have created detailed documentation in word for Setting up IIS as a reverse proxy with Splunk. I will email you...Thanks...

0 Karma

mukuldang08
New Member

CAn you send me the same? mukul94dang@gmail.com

0 Karma

deepashri_123
Motivator

hey@JHudson_CVX,
Appreciate ur help!!!
my email id is: deepashri.amrutkar@smartcirqls.com

0 Karma

JHudson_CVX
New Member

I sent you an email. Hope you got it...

0 Karma

vidyadharms
New Member

We have a similar setup and we are using IIS 8.5 as Reverse Proxy....But only redirection happens and SSO didn't work. We got see only Splunk login page. No value for X-Remote-User variable when checked in Splunk SSO debug page. Can you please share the steps to configure SSO with IIS reverse proxy authentication?

0 Karma

jplumsdaine22
Influencer

Hi @ishaanshekhar

How are you controlling the route persistence from the reverse proxy to the search head cluster? Make sure you follow the guide here (http://docs.splunk.com/Documentation/Splunk/6.3.3/DistSearch/UseSHCwithloadbalancers)

Also how have you configured the load balancer? Are you doing any header / cookie rewriting?

0 Karma

ishaanshekhar
Communicator

Someone please help!!!!

0 Karma

deepashri_123
Motivator

Hi@ishaanshekhar,
Can u help how u resolved this issue

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...