Reporting

Splunk 8.0.2 report acceleration broken for reports using inputlookup command in subsearches

orion44
Communicator

Prior to updating to Splunk Enterprise 8.0.2 scheduled accelerated reports ran extremely fast:
Report A
Duration: 37.166
Record count: 314

After updating to Splunk Enterprise 8.0.2 the report ran extremely slow:
Report A
Duration: 418.621
Record count: 300

Given the patch notes for 8.0.2 – I'm not seeing any changes to acceleration or summary indexing, so is it safe to assume this is a fluke?

The massive increase in report generation (job) time of the scheduled accelerated reports appears to be caused by them no longer accessing the corresponding report acceleration summary. The "Access Count" never goes up when the scheduled reports are run.

alt text

Guess we'll wait for 8.0.3 to fix this.

Troubleshooting steps attempted:
Manually rebuild Report Acceleration Summaries
Delete all affected Report Acceleration Summaries
Delete and recreate affected production reports – recreated schedule and checked box for acceleration
Check filesystem permissions of inputlookup csv - confirmed -rw-rw-r-- splunk splunk

Labels (2)
0 Karma

nvanderwalt_spl
Splunk Employee
Splunk Employee

It would be useful to know which version you were on before when it was working, and whether the inputlookup is done as a subsearch.

0 Karma

orion44
Communicator

Version 8.0.1 was used prior to the issue starting after upgrading to 8.0.2. The inputlookup function used is [| inputlookup filename.csv | fields name1 | rename name1 as my_search_value | format] and comes after index=index_name in the search query.

0 Karma

orion44
Communicator

Downgrading to 8.0.1 resulted in acceleration working correctly again. Do you know if there's a full list of changes in 8.0.2 that we can review for a possible cause?

0 Karma

gjanders
SplunkTrust
SplunkTrust

Did you log a support case? As that way the support team will actually confirm that its a bug/fix the issue in a future release...

0 Karma

orion44
Communicator

"It appears you do not have an active Support Contract or entitlement and as a result, cannot open a Support case. If you believe this is an error, please contact 1-855-SPLUNK S, or consult https://www.splunk.com/en_us/about-us/contact.html#customer-support for a country specific Support phone number and we can resolve any contractual data integrity issues."

0 Karma

gjanders
SplunkTrust
SplunkTrust

Oh ok, if your not on a customer site you cannot raise an issue, if i get spare time I'll try it 🙂

0 Karma

orion44
Communicator

Were you able to report a support case? Has anyone else reported this issue?

0 Karma

gjanders
SplunkTrust
SplunkTrust

Will try and replicate it today...where does the summarisation load screen come from?

0 Karma

orion44
Communicator
0 Karma

gjanders
SplunkTrust
SplunkTrust

So I did this, search 1:

index=_internal | stats count by _time

Search 2:

index=_internal | stats count by _time | inputlookup append=true rest_api_test.csv

Report acceleration confirms they will both use the same acceleration job.
When I run the reports they advise they both used the same acceleration job.

So did not replicate it, is that similar to what you are doing?

0 Karma

gjanders
SplunkTrust
SplunkTrust

Search 2 is scheduled and working as expected, access count has increased to 6 now (did not replicate your issue)!
Splunk 8.0.2

0 Karma

orion44
Communicator

Try doing the inputlookup as a subsearch. In my case, it looks like this:

 [| inputlookup file.csv | fields title | rename title as TITLE | format]
0 Karma

gjanders
SplunkTrust
SplunkTrust

Replicated as a subsearch via:
index=_internal | stats count by _time | append [ inputlookup rest_api_test.csv ]

0 Karma

orion44
Communicator

Thanks, I'll update OP to indicate subsearch requirement.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Support confirmed "According to the search.log, there is a change in the behavior of the search using append between 8.0.1 and 8.0.2." investigation will now go to sustaining...

0 Karma

orion44
Communicator

Fantastic, thanks for submitting the case!

0 Karma

orion44
Communicator

Schedule the second search you created to run at some interval and see if the "Access Count" number increases after each scheduled report runs. In my case, the count doesn't go up and the report generates slowly – despite showing that it is using the acceleration (summary) index on the Report Acceleration Summaries page.

0 Karma

nvanderwalt_spl
Splunk Employee
Splunk Employee

And is the summary showing as complete? Perhaps the summary is not in a usable state yet?

0 Karma

orion44
Communicator

Yes, the summary was 100% complete as shown in OP screenshot. Prior to the 8.0.2 upgrade the report acceleration was working as intended.

0 Karma

orion44
Communicator

It could, that would be greatly appreciated. Thanks!

0 Karma

gjanders
SplunkTrust
SplunkTrust

Sorry I do not have good news, basically embedded subsearches + newer Splunk versions + report acceleration do not work together, I believe it relates to fixing a previous issue...

The fix will come in a future version but no ETA so I cannot help further!

The only solution is to not have embedded sub searches....or don't accelerate 🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...