Splunk PDF report works at "Send Test Email" but failed at scheduled runs

[wanling]

I recently upgraded our Splunk instance to Splunk 5.0.2. The Schedule PDF Delivery using the new integrated PDF feature works fine when I "send test email". However, there's no PDF report being attached in the email when it's run through the scheduler. The email carried an error message: "An error occurred while generating a PDF of this report. Please see python.log for details."

I have problem sending reports to management now. Splunk does not seem to fall back to the original PDF print server. Please advise.

here's the segment of python.log related to this scheduled run.

2013-03-21 21:00:02,247 INFO sendemail:mail sendPDF = True, pdfview = rpt_computer_status, searchid =
2013-03-21 21:00:02,247 DEBUG Preloading from 'D:\Program Files\Splunk\var\run\splunk\merged\server.conf'.

2013-03-21 21:00:02,249 DEBUG Preloading from 'D:\Program Files\Splunk\var\run\splunk\merged\web.conf'.

2013-03-21 21:00:02,250 DEBUG simpleRequest > GET https://127.0.0.1:8089/servicesNS/nobody/citi/admin/alert_actions/email [] sessionSource=direct

2013-03-21 21:00:02,648 DEBUG simpleRequest < server responded status=200 responseTime=0.3990s

2013-03-21 21:00:02,654 DEBUG simpleRequest > GET https://127.0.0.1:8089/servicesNS/huangwl/citi/data/ui/views/rpt_computer_status [] sessionSource=direct

2013-03-21 21:00:02,667 DEBUG simpleRequest < server responded status=200 responseTime=0.0130s

2013-03-21 21:00:02,668 INFO sendemail pdfService = pdfgen

2013-03-21 21:00:02,668 INFO sendemail:mail effectiveTime=1363870800

2013-03-21 21:00:02,668 DEBUG simpleRequest > GET https://127.0.0.1:8089/services/pdfgen/render?input-dashboard=rpt_computer_status&owner=huangwl&now=... [] sessionSource=direct

2013-03-21 21:00:03,875 Malay Peninsula Standard Time WARNING pdfgen_endpoint:323 - Invalid argument passed to pdfgen/render. arg=owner value=huangwl

2013-03-21 21:00:04,444 Malay Peninsula Standard Time INFO pdfgen_endpoint:308 - pdf time-of-report=1363870800.0

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:405 - could not dispatch search view: No of Active Computers (Total: 149)

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:405 - could not dispatch search view: Computers Named Incorrectly

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:405 - could not dispatch search view: Computers sending system data

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:173 - Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:173 - Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available

2013-03-21 21:00:04,510 Malay Peninsula Standard Time ERROR pdfgen_endpoint:173 - Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available

2013-03-21 21:00:04,512 Malay Peninsula Standard Time ERROR pdfgen_endpoint:173 - Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available

2013-03-21 21:00:04,512 Malay Peninsula Standard Time ERROR pdfgen_endpoint:179 - No views prepared without exceptions. Bailing out of PDF render.

2013-03-21 21:00:05,194 DEBUG simpleRequest < server responded status=400 responseTime=2.5250s

2013-03-21 21:00:05,194 ERROR An error occurred while generating a PDF of this report: Failed to fetch PDF (status = 400): Unable to render PDF.

• Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available
• Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available
• Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available
• Exception raised while trying to prepare "Computer Status" for rendering to PDF. No search job available

2013-03-21 21:00:07,604 INFO Sending email. subject="Splunk Alert: rpt_computer_status", results_link="https://xxx.xxx.xxx.xxx:8000/app/citi/@go?sid=", recipients="['xxx@xxx.xxx']"

[wanling]

I've further findings on this issue, upon troubleshooting another report having similar problems. The PDF report is available, but "No results found" is shown for each panel. On the other hand, test run is okay.

It turned out that all the knowledge objects (macro, field extraction, events, tags etc) used in the reports must be given global permission, otherwise there will be no result returned during the scheduled run. I suppose the scheduler somehow is also subject to the access control in this case.

[wanling]

After comparing a working copy, I realised the tag used to specify the search string makes a difference. Now the report schedule works after changing tag to in the XML. Unfortunately, this cause is not reflected in the python.log, making the initial troubleshooting rather clueless.