Splunk 8.0.2 report acceleration broken for reports using inputlookup command in subsearches

[orion44]

Prior to updating to Splunk Enterprise 8.0.2 scheduled accelerated reports ran extremely fast:
Report A
Duration: 37.166
Record count: 314

After updating to Splunk Enterprise 8.0.2 the report ran extremely slow:
Report A
Duration: 418.621
Record count: 300

Given the patch notes for 8.0.2 – I'm not seeing any changes to acceleration or summary indexing, so is it safe to assume this is a fluke?

The massive increase in report generation (job) time of the scheduled accelerated reports appears to be caused by them no longer accessing the corresponding report acceleration summary. The "Access Count" never goes up when the scheduled reports are run.

Guess we'll wait for 8.0.3 to fix this.

Troubleshooting steps attempted:
Manually rebuild Report Acceleration Summaries
Delete all affected Report Acceleration Summaries
Delete and recreate affected production reports – recreated schedule and checked box for acceleration
Check filesystem permissions of inputlookup csv - confirmed -rw-rw-r-- splunk splunk

[nvanderwalt_spl]

It would be useful to know which version you were on before when it was working, and whether the inputlookup is done as a subsearch.

[orion44]

Version 8.0.1 was used prior to the issue starting after upgrading to 8.0.2. The inputlookup function used is [| inputlookup filename.csv | fields name1 | rename name1 as my_search_value | format] and comes after index=index_name in the search query.

[orion44]

Downgrading to 8.0.1 resulted in acceleration working correctly again. Do you know if there's a full list of changes in 8.0.2 that we can review for a possible cause?

[gjanders]

Did you log a support case? As that way the support team will actually confirm that its a bug/fix the issue in a future release...

[orion44]

"It appears you do not have an active Support Contract or entitlement and as a result, cannot open a Support case. If you believe this is an error, please contact 1-855-SPLUNK S, or consult https://www.splunk.com/en_us/about-us/contact.html#customer-support for a country specific Support phone number and we can resolve any contractual data integrity issues."

[gjanders]

Oh ok, if your not on a customer site you cannot raise an issue, if i get spare time I'll try it 🙂

[orion44]

Were you able to report a support case? Has anyone else reported this issue?

[gjanders]

Will try and replicate it today...where does the summarisation load screen come from?

[orion44]

Thanks!

It can be found at https://:8000/en-US/manager/system/summarization

[gjanders]

So I did this, search 1:

index=_internal | stats count by _time

Search 2:

index=_internal | stats count by _time | inputlookup append=true rest_api_test.csv

Report acceleration confirms they will both use the same acceleration job.
When I run the reports they advise they both used the same acceleration job.

So did not replicate it, is that similar to what you are doing?

[gjanders]

Search 2 is scheduled and working as expected, access count has increased to 6 now (did not replicate your issue)!
Splunk 8.0.2

[orion44]

Try doing the inputlookup as a subsearch. In my case, it looks like this:

 [| inputlookup file.csv | fields title | rename title as TITLE | format]

[gjanders]

Replicated as a subsearch via:
index=_internal | stats count by _time | append [ inputlookup rest_api_test.csv ]

[orion44]

Thanks, I'll update OP to indicate subsearch requirement.

[gjanders]

Support confirmed "According to the search.log, there is a change in the behavior of the search using append between 8.0.1 and 8.0.2." investigation will now go to sustaining...

[orion44]

Fantastic, thanks for submitting the case!

[orion44]

Schedule the second search you created to run at some interval and see if the "Access Count" number increases after each scheduled report runs. In my case, the count doesn't go up and the report generates slowly – despite showing that it is using the acceleration (summary) index on the Report Acceleration Summaries page.

[nvanderwalt_spl]

And is the summary showing as complete? Perhaps the summary is not in a usable state yet?

[orion44]

Yes, the summary was 100% complete as shown in OP screenshot. Prior to the 8.0.2 upgrade the report acceleration was working as intended.

[orion44]

It could, that would be greatly appreciated. Thanks!

[gjanders]

Sorry I do not have good news, basically embedded subsearches + newer Splunk versions + report acceleration do not work together, I believe it relates to fixing a previous issue...

The fix will come in a future version but no ETA so I cannot help further!

The only solution is to not have embedded sub searches....or don't accelerate 🙂