HiddenSavedSearch - internals

[jonuwz]

I'm trying to figure out how to update a HiddenSavedSearch (and subsequently the SimpleResultsTable) by sending pushContextToChildren from an upstream module.

I can sort of see what happening by adding console.log statements to HiddenSavedSearch.js

The initial context contains the savedSearch, and I need to replace this savedSearch when the HiddenSavedSearch received 'getModifiedContect'. If I knew how the initial savedSearch is formed, then I could use the same process to update it. (replacing the job sid so it finds the newest job results)

Any pointers would be greatly appreciated.

John

[jonuwz]

Hi. Thats exactly what I'm trying to do.

The problem is that i've got a very heavy dashboard destined for plasma screens in public areas.

A full refresh takes between 3-6 seconds, and only some components need to be refreshed every minute.

Refreshing SavedSearches on an interval was altogether straightforward.

For HiddenSavedSearches it stores the search definition even before $super(container) gets called.

I can see there's already a mechanism in getModifiedContext for it to inherit a sid, and it'll swap out the search - I have no idea how to exploit this in a dashboard though.

if (this.sid) {
            inheritedSearch.job = Splunk.Globals.Jobber.buildJob(this.sid);
            context.set("search", inheritedSearch);
            return context;
        }

This does work. As I test, I set this.sid to another valid job sid, and it overrides the savedsearch in the view.xml. Promising.

What I think I need to do is

• set up something in initialise() to do a refresh every X seconds
• override Refresh() to do an ajax call to get the latest sid and set this.sid
• Call getModifiedContext and pushContextToChildren
• Profit

Unfortunately, i'm very new to splunk, and while the documentation (including the app framework stuff) is great, and the js and py files actually have comments that describe the intention of the code - There's an awful lot of it to get through.

I guess what I'm asking is where / what do I query to get the latest run instance of a job, and is there already a pattern to do this ?

Cheers

John

[sideview · Accepted solution]

I'm afraid I need to find out what you're actually trying to do, in order to properly advise you.

As a wild guess, possibly you want to have a dashboard stay up for hours at a time, and have the HiddenSavedSearch automatically pick up the new search id's when the schedule generates new search results for that saved search. And although this would be solved by refreshing the whole page, you don't want to just put refresh="-1" into the <view> tag.

Again, that's kind of a wild guess, but I can't think of any situation where you'd want to reach in and tinker with the data coming out of a HiddenSavedSearch. Can you add more detail either as an update to your question or as a comment on this answer?

To understand how the module framework works, if you are already ready to invest more than an hour or so reading things and you're able to read javascript you should just go read AbstractModule.js, particularly the pushContextToChildren method. getContext will always get the contextual data that the given module inherited from it's parent. getModifiedContext on the other hand will get that same context data, plus whatever additions or edits that given module has itself made to the data to pass on to it's children.

There's also a fairly weighty set of docs on the site although since it's not technically on the 'docs' site, you'll never find it by searching from docs. http://dev.splunk.com/view/SP-CAAADW2
There's tons of methods documented there though and far more than just the core methods of getContext/getModifiedContext/pushContextToChildren/onContextChange... If you're comfortable reading code I'd just go read the comments and code in AbstractModule and in a few concrete modules.

UPDATE ------------

Sounds like my wild guess was surprisingly accurate.

OK. It's a tricky path you have to go down. There are many pitfalls. It's very tricky and if I'm saying that then you know it's bad. However you seem to know your way around.

You've probably already figured out that the sid is baked into the page at page-load time. It's buried in the loadParams and you can see them jumbled into some JSON near the bottom of the page. HiddenSavedSearch begins and ends its life using that one magic sid that was baked into it's load params.

So you have to modify its behavior. There's the framework-ey way, and the quick way. Let's do the quick way. (I'll worry about the framework-ey way and I might bake it into a future version of the Sideview Utils module "SavedSearch")

1) define a new method on SavedSearch that will make an ajax call to check the history of the given saved search name. When the response of that call returns, if it turns out there is a newer job up there, a) modify the this.sid property directly, and b) call this.pushContextToChildren().

2) override the constructor, ie initialize such that the aforementioned new method will run on an timeout/interval.

( If you haven't found a good set of examples showing how to patch an existing module class email me and I'll give you those pointers. )

Now to the actual implementation of #1,

1) good news: under the "/saved/searches/" part of the REST api, there's a history endpoint ( /servicesNS/nobody/MY_APP_NAME/saved/searches/MY_SAVED_SEARCH_NAME/history ). This endpoint, assuming the saved search is exported to app-level or system-level (use the little permissions links in manager), will return atom-feed XML. You can then parse these out to yield the sid of the most recent search results.

2) bad news: That's all well and good but of course this is a URL on the splunkd management port, NOT a URL on the splunkWeb port. Browser-security being what it is, it does us no good from the javascript.

3) good news: There is a base endpoint on splunkWeb called "/splunkd" that is enabled by default, and that theoretically gives you the ability to hit anything on splunkd's REST API.

4) bad news: the /splunkd endpoint now has an xsl file listed, but it's an xsl file that's only on splunkd, and when splunkWeb tries to do server side XSL it blows up. The XML is not actually returned when you make the request, only the XSL error. Also SplunkWeb doesn't like spaces in url-segments so your saved search name would have to be devoid of spaces.

5) good news: You can write your own python controller, use sample REST API code to make the EAI call to /saved/searches, thus get you the latest sid for a given saved search, and return that sid as JSON to the client.

[jonuwz]

If I could use the sideview utils, believe me - I would.
Using text replacement in postprocessing would be worth it alone.

Unfortunately, for the app i'm building this would mean removing all/most the sideview app views. I've read the T&C, and feel this would be bad karma.

Any view with a SimpleResultsTable not Gimp-ed upstream can expose data just by sticking /?q=search... on the url.
It even bypasses the role-restrictions.

I can't have users seeing non-relevant data, but cannot split out the data into seperate indexes.

I'll mail you regarding other licensing options - and thanks again.

[sideview]

Aha. Of course going the other way and using the jobs feed to parse every single job, and then find the savedsearch you're looking for... that's probably a decent option. The cleanest way in my opinion, is instead of patching SavedSearch, is to use a CustomBehavior from Sideview Utils. Make that CB module do the timeout, make it go get the sid, and then in getModifiedContext make it do what HiddenSavedSearch does and return a search+job for that current sid. I'd be wary of using JobManager and HiddenSearchSwapper source code too much but maybe that's just me.

[jonuwz]

Thanks for this. I suspected as much.

I was looking for a 'clean' way to get the latest sid for a search. I can probably rework HiddenSearchSwapper.py to do as you've suggested

Temporarily I've resorted to parsing the html returned from :

/en-GB/module/system/Splunk.Module.JobManager/render ?client_app=search&count=1&offset=0&sortKey=dispatch_time&sortDir=desc&jobStatus=Done&app=xxxx&label=yyyyyyyyyyy#

The comments in the job_manager js suggest format changes are coming, so I really should be doing this with a controller.

re spaces: - HiddenFieldPicker doesn't like spaces either.