All Apps and Add-ons

Why am I receiving a 404 error on a postprocess search using SideView where it states the CID cannot be found after upgrading to Splunk 6.6.6?

dmarling
Builder

We have recently upgraded to Splunk 6.6.6 from 6.5.2. After the upgrade completed we started having issues with one of our SideView dashboards that is used to modify single records on a lookup table. The problem that we run into is when we click button on the dashboard to save the changes, a 404 error is happening on a post processing search that is used to validate the data that the user is attempting to insert. Here is the error:

{"messages": [{"type": "ERROR", "time": "2018-04-28T22:01:07", "message": "job sid=1524966194.14471_B4ABD78F-1B36-472A-A31D-D2D18FC64BA6 not found"}], "data": null, "total": 0, "success": false, "offset": 0, "count": 0}

Obviously the sid is different each time someone uses this dashboard, but the results are the same. Here is the URI that is being called:

Request URL: /en-US/api/search/jobs/1524966194.14471_B4ABD78F-1B36-472A-A31D-D2D18FC64BA6/results?search=%7C%20rest%20%2Fservices%2Fauthentication%2Fcurrent-context%2Fcontext%0A%7C%20search%20NOT%20username%3D%22splunk-system-user%22%0A%7C%20eval%20APR%3D%22Unknown%22%0A%7C%20eval%20ASKIT%3D%22Unknown%22%0A%7C%20eval%20INC%3D%22Unknown%22%0A%7C%20eval%20PRB%3D%22Unknown%22%0A%7C%20eval%20PRB_MSG%3Dif((match(PRB%2C%22PRB%5B0%5D%5Cd%7B6%7D%22)%20AND%20len(PRB)%3D10)%20OR%20(PRB%3D%22Unknown%22)%2C%22%22%2C%22PRB%20ticket%20must%20be%20in%20format%20PRB0012345%2C%20%27Unknown%27%20entered%20above.%22)%0A%7C%20eval%20INC_MSG%3Dif((match(INC%2C%22INC%5B012%5D%5Cd%7B6%7D%22)%20AND%20len(INC)%3D%3D10)%20OR%20(INC%3D%22Unknown%22)%2C%22%22%2C%22INC%20ticket%20must%20be%20in%20format%20INC0012345%2C%20%27Unknown%27%20entered%20above.%22)%0A%7C%20eval%20APR_MSG%3Dif((match(APR%2C%22(APR%7CCHG)%5B0%5D%5Cd%7B6%7D%22)%20AND%20len(APR)%3D10)%20OR%20(APR%3D%22Unknown%22)%2C%22%22%2C%22CHG%2FAPR%20must%20be%20in%20format%20CHG0012345%20OR%20APR0012345%2C%20%27Unknown%27%20entered%20above.%22)%0A%7C%20eval%20ASKIT_MSG%3Dif((match(ASKIT%2C%22%5B10%5D%5B0-9%5D%5Cd%7B13%7D%22)%20AND%20len(ASKIT)%3D15)%20OR%20(ASKIT%3D%22Unknown%22)%20%2C%22%22%2C%22ASKIT%20article%20IDs%20must%20be%20a%20total%20of%2015%20digits%20and%20start%20with%2004%2C%2002%2C%20or%2009.%20%27Unknown%27%20was%20entered%20above.%22)%0A%7C%20eval%20INVALIDFIELD%3DPRB_MSG.INC_MSG.APR_MSG.ASKIT_MSG%0A%7C%20eval%20INVALIDTICKET%3DPRB_MSG.INC_MSG.APR_MSG%0A%7C%20eval%20INVALIDASKIT%3DASKIT_MSG%0A%7C%20eval%20VALIDFIELD%3Dif(INVALIDFIELD%3D%22%22%2C%22VALID%22%2CNULLER)%20%7C%20fields%20VALIDFIELD%20INVALIDFIELD%20PRB_MSG%20INC_MSG%20APR_MSG%20ASKIT_MSG%20INVALIDTICKET%20INVALIDASKIT&outputMode=json
Request Method: GET
Status Code: 404 Not Found

And here is the query parsed out:

| rest /services/authentication/current-context/context
| search NOT username="splunk-system-user"
| eval APR="Unknown"
| eval ASKIT="Unknown"
| eval INC="Unknown"
| eval PRB="Unknown"
| eval PRB_MSG=if((match(PRB,"PRB[0]\d{6}") AND len(PRB)=10) OR (PRB="Unknown"),"","PRB ticket must be in format PRB0012345, 'Unknown' entered above.")
| eval INC_MSG=if((match(INC,"INC[012]\d{6}") AND len(INC)==10) OR (INC="Unknown"),"","INC ticket must be in format INC0012345, 'Unknown' entered above.")
| eval APR_MSG=if((match(APR,"(APR|CHG)[0]\d{6}") AND len(APR)=10) OR (APR="Unknown"),"","CHG/APR must be in format CHG0012345 OR APR0012345, 'Unknown' entered above.")
| eval ASKIT_MSG=if((match(ASKIT,"[10][0-9]\d{13}") AND len(ASKIT)=15) OR (ASKIT="Unknown") ,"","ASKIT article IDs must be a total of 15 digits and start with 04, 02, or 09. 'Unknown' was entered above.")
| eval INVALIDFIELD=PRB_MSG.INC_MSG.APR_MSG.ASKIT_MSG
| eval INVALIDTICKET=PRB_MSG.INC_MSG.APR_MSG
| eval INVALIDASKIT=ASKIT_MSG
| eval VALIDFIELD=if(INVALIDFIELD="","VALID",NULLER) | fields VALIDFIELD INVALIDFIELD PRB_MSG INC_MSG APR_MSG ASKIT_MSG INVALIDTICKET INVALIDASKIT

When hitting that same cid with no query I get the information I expected minus the validation that the above query is attempting to perform on the inputted values:

Request URL: /en-US/api/search/jobs/1524966194.14471_B4ABD78F-1B36-472A-A31D-D2D18FC64BA6/results?outputMode=json
Request Method: GET
Status Code: 200 OK

We upgraded to SideView Utilities 3.4.1 as we had been running 3.3.11 and we are still receiving the same results.

If this comment/answer was helpful, please up vote it. Thank you.
0 Karma
1 Solution

dmarling
Builder

I have resolved this issue based on feedback from SideView Support. Thank you Nick Mealy!

I simply needed to remove the first two lines of the post processing search:

| rest /services/authentication/current-context/context
| search NOT username="splunk-system-user"

As for why that fixed it, here is the explanation I was provided as to why I got a 404:

Postprocess 404's are confusing
because they are never actually 404's.
(Well, almost never) In some security
review, it was decided that if
anything at all went wrong with a
postprocess request, it should return
404 and never 50x.

This means that the issue was my SPL causing an error when the call to the api is done and SideView was returning a 404 as a result of that error. In a release between Splunk version 6.5.2 and 6.6.6, when having a rest call as the first line event of a post processing search, it will cause the search to fail. I believe this was always the intended results as a rest call should always be the first part of a search and not in the middle of a search. We were exploiting a bug unintentionally with the way it was setup when this was originally built several years ago.

If this comment/answer was helpful, please up vote it. Thank you.

View solution in original post

0 Karma

dmarling
Builder

I have resolved this issue based on feedback from SideView Support. Thank you Nick Mealy!

I simply needed to remove the first two lines of the post processing search:

| rest /services/authentication/current-context/context
| search NOT username="splunk-system-user"

As for why that fixed it, here is the explanation I was provided as to why I got a 404:

Postprocess 404's are confusing
because they are never actually 404's.
(Well, almost never) In some security
review, it was decided that if
anything at all went wrong with a
postprocess request, it should return
404 and never 50x.

This means that the issue was my SPL causing an error when the call to the api is done and SideView was returning a 404 as a result of that error. In a release between Splunk version 6.5.2 and 6.6.6, when having a rest call as the first line event of a post processing search, it will cause the search to fail. I believe this was always the intended results as a rest call should always be the first part of a search and not in the middle of a search. We were exploiting a bug unintentionally with the way it was setup when this was originally built several years ago.

If this comment/answer was helpful, please up vote it. Thank you.
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...