How do I retrieve all query results in a custom visualization without “No results found” hiding the visualization?

[cartersmith]

I am creating a custom visualization for Splunk Enterprise. I want the visualization to see all the data rows returned from the query, so within my updateView() I call updateDataParams() to set the offset to the total number of rows received so far. However, I have run into several issues with this approach.

If the offset reaches the total number of rows returned by the query, there is an additional call to updateView() with zero rows. Regardless of what code executes in updateView() for that last call, the visualization is replaced by the text “No results found.” The only way I have found to prevent this “No results found” text from hiding my visualization is to detect the last non-empty batch of results and not call updateDataParams() in that case.

I tried following the example of the blog post at https://www.splunk.com/blog/2016/04/11/show-me-your-viz.html, which suggests only calling updateDataParams() if data.rows.length is exactly 50,000, but this can stop asking for results too soon. With both large and small result sets, there can be multiple calls to updateView() with fewer than 50,000 rows before all the results have been passed to updateView().

There is also a “data.meta.done” flag which looks promising, but for a large result set there can be several calls to updateView() with the flag set. I’m guessing the flag means that all the query results have been received from the server, but they may not have been passed to updateView() yet. Also, this flag is not always defined, so checking it requires a few extra contortions.

For large result sets, I can usually detect the last batch of rows by checking for both the “done” flag and that the batch size is smaller than 50,000. This will break if the last batch just happens to be exactly 50,000 rows, however.

This approach also fails if the last non-empty batch of rows is passed in with the “done” flag still set to false. This can happen with small result sets. I haven’t figured out a way to work around this last problem.

So the question: How do I reliably get the full set of query results sent to updateView() without having my custom visualization hidden by the “No results found” error message?

[cartersmith · Accepted solution]

After testing this more, I see the behavior described above when getInitialDataParams() asks for data in either ROW_MAJOR_OUTPUT_MODE or COLUMN_MAJOR_OUTPUT_MODE, but not with RAW_OUTPUT_MODE, so my fix is to switch to raw output mode.

[FloSwiip]

Hello,

Not the answer you are looking for, but the workaround I use to avoid that “No results found” horrible display that kill dashboards formatting, is to append an empty row at search level to get always something displayed...

example (the first line is supposed to give no results here) :
index=notanindex | table host
| appendpipe [ stats count | eval host="" | fields - count ]

Still interested by a smarter solution 🙂

Best regards.