Reporting
Highlighted

Where should I check for python.log error messages about generating pdf of scheduled reports?

Contributor

Hi,

I get the following error while receiving some scheduled reports in Splunk 6.2.

An error occurred while generating the PDF. Please see python.log for details.

But, NOT all reports generate the pdf, only some of them do.
And the following are the lines of the python.log under opt/splunk//var/log/splunk:

2016-01-01 09:01:41,038 +0100 ERROR     pdfgen_endpoint:230 - No views prepared without exceptions. Bailing out of Integrated PDF Generation.
2016-01-01 09:01:41,172 +0100 ERROR     sendemail:971 - An error occurred while generating a PDF: Failed to fetch PDF (status = 400): Unable to render PDF.&lt;br/&gt;<ul><li>Exception raised while trying to prepare "MY OTHER REPORT" for rendering to PDF. Splunkd daemon is not responding: ('Error connecting to /servicesNS/admin/MY_APP/search/parser: The read operation timed out',)</li></ul>
2016-01-01 09:01:57,389 +0100 ERROR     __init__:477 - Socket error communicating with splunkd (error=The read operation timed out), path = /servicesNS/admin/servicedesk_plus/search/parser
2016-01-01 09:01:57,390 +0100 ERROR     pdfgen_endpoint:224 - Exception raised while trying to prepare "MY REPORT" for rendering to PDF. Splunkd daemon is not responding: ('Error connecting to /servicesNS/admin/MY APP/search/parser: The read operation timed out',)
2016-01-01 09:01:57,390 +0100 ERROR     pdfgen_endpoint:230 - No views prepared without exceptions. Bailing out of Integrated PDF Generation.
2016-01-01 09:01:57,514 +0100 ERROR     sendemail:971 - An error occurred while generating a PDF: Failed to fetch PDF (status = 400): Unable to render PDF.&lt;br/&gt;<ul><li>Exception raised while trying to prepare "MY REPORT" for rendering to PDF. Splunkd daemon is not responding: ('Error connecting to /servicesNS/admin/MY APP/search/parser: The read operation timed out',)</li></ul>

What is exactly the issue?

Thanks for any help,
Skender

Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Contributor

Could it be because of some restart of splunk daemon?

Skender

0 Karma
Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Explorer

I had the same issue. I finally figured it out. To find out if you're having the same issue look splunkdaccess.log in $SPLUNKHOME/var/log/splunk.

user@server:/opt/splunk/var/log/splunk# grep Cisco splunkd_access.log
127.0.0.1 - splunk-system-user [17/May/2016:15:26:34.142 -0400] "POST /servicesNS/nobody/search/saved/searches/Audit%20Cisco%20Changes/notify?trigger.condition_state=1 HTTP/1.0" 200 1928 - - - 10ms
127.0.0.1 - splunk-system-user [17/May/2016:15:26:34.339 -0400] "GET /servicesNS/user/search/saved/searches/Audit%20Cisco%20Changes?output_mode=json HTTP/1.0" 200 34334 - - - 15ms
127.0.0.1 - splunk-system-user [17/May/2016:15:26:34.730 -0400] "GET /servicesNS/user/search/saved/searches/Audit%20Cisco%20Changes HTTP/1.0" 200 57101 - - - 11ms
127.0.0.1 - splunk-system-user [17/May/2016:15:26:34.498 -0400] "GET /services/pdfgen/render?paper-size=legal-landscape&sid=scheduler_cm9uLm9nbGU__search__RMD5ec6c9fa239e4c78b_at_1463513100_76&input-report=Audit%20Cisco%20Changes&now=1463513100&owner=user&namespace=search HTTP/1.0" **400** 272 - - - **60452ms**

In the log above, you can see the last line has a "400" status code and a time value of "60452ms". The status code will show up in the python.log.

user@server:/opt/splunk/var/log/splunk# grep Cisco python.log
2016-05-17 15:26:34.498 -0400 ERROR sendemail:996 - An error occurred while generating a PDF: Failed to fetch PDF (**status = 400**): Unable to render PDF.<br/><ul><li>Exception raised while trying to prepare "Audit Cisco Changes" for rendering to PDF. Splunkd daemon is not responding: ("Error connecting to /servicesNS/user/search/search/parser: ('The read operation timed out',)",)</li></ul>
2016-05-17 15:26:34,979 -0400 INFO  sendemail:112 - Sending email. subject="Splunk Report: Audit Cisco Changes", results_link="https://server.admin.local:8000/app/search/@go?sid=scheduler_cm9uLm9nbGU__search__RMD5ec6c9fa239e4c78b_at_1463462400_63889", recipients="[u'hosted@service-now.com']", server="smtp.admin.local"

The error will also show up in the pdfgen.log:

user@server:/opt/splunk/var/log/splunk# grep -i cisco pdfgen.log
2016-05-17 15:26:34,885 -**0400** ERROR pdfgen_utils:481 - Exception raised while trying to prepare "Audit PCI Cisco Changes New" for rendering to PDF. Splunkd daemon is not responding: ("Error connecting to /servicesNS/user/search/search/parser: ('The read operation timed out',)",)

So that is how to verify that you have the same problem that I had. To fix this problem, you have to change the splunkdConnectionTimeout value in web.conf to something bigger than the time found in the splunkd_access.log file. In my case, I changed the value to 90 (seconds). The default value is 30 seconds.

To make the change, copy or create the web.conf file from etc/system/default/web.conf to etc/system/local/web.conf. Change the splunkdConnectionTimeout= where is greater than the number is seconds from the splunkdaccess.log. Note that the splunkdaccess.log value is in milliseconds.

Restart the Splunkd service.

View solution in original post

Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Explorer

great write up; thx

0 Karma
Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Explorer

Well explained. Thank you..

0 Karma
Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Champion

Very informative writeup. Thank you for taking the time. I'm awarding you points for your effort!

0 Karma
Highlighted

Re: Where should I check for python.log error messages about generating pdf of scheduled reports?

Community Manager
Community Manager

Hi @skender27

Just wanted to follow up to see if this answer by @ronogle worked for you? If yes, please be sure to resolve the post by clicking "Accept" directly below the answer. This is a pretty useful topic that would benefit the community. Thanks!

Patrick

0 Karma