Rapid7 Nexpose Add-On not pulling all assets

[stephen_beach]

For my onsite Splunk deployment with the Rapid7 Add-on, it will pull assets from all of my sites with less than ~300 assets. However for any site with more than ~300 assets the site fails to load and no data is ever sent to Splunk.

From what I can see from the Nexpose console it shows that it receives the request to pull the data and begins constructing the table. Then nothing happens and it stops pulling data. Here is what I have from the Splunk /opt/splunk/var/log/splunk/TA-rapid7_nexpose.log

2017-10-24 11:10:01,164 INFO    nx_logger:38 - Processing vuln report for site(s) <['50']>
2017-10-24 11:10:03,394 INFO    nx_logger:38 - Finished processing vuln report for site(s) <['50']>
2017-10-24 11:10:03,491 INFO    nx_logger:38 - Updating scan data historical file
2017-10-24 11:10:03,492 INFO    nx_logger:38 - Loading last scan data from file.
2017-10-24 11:10:03,492 INFO    nx_logger:38 - Parsing scan data from CSV: /opt/splunk/var/lib/splunk/modinputs/rapid7nexpose/last_scan_data.csv
2017-10-24 11:10:03,492 INFO    nx_logger:38 - CSV file </opt/splunk/var/lib/splunk/modinputs/rapid7nexpose/last_scan_data.csv> parsed.
2017-10-24 11:10:03,492 INFO    nx_logger:38 - Writing scan data historical file
2017-10-24 11:10:03,493 INFO    nx_logger:38 - Historical file written.
2017-10-24 11:11:36,160 INFO    nx_logger:38 - Processing vuln report for site(s) <['23']>
2017-10-24 11:11:37,375 INFO    nx_logger:38 - Finished processing vuln report for site(s) <['23']>
2017-10-24 11:11:37,452 INFO    nx_logger:38 - Updating scan data historical file
2017-10-24 11:11:37,453 INFO    nx_logger:38 - Loading last scan data from file.
2017-10-24 11:11:37,453 INFO    nx_logger:38 - Parsing scan data from CSV: /opt/splunk/var/lib/splunk/modinputs/rapid7nexpose/last_scan_data.csv
2017-10-24 11:11:37,454 INFO    nx_logger:38 - CSV file </opt/splunk/var/lib/splunk/modinputs/rapid7nexpose/last_scan_data.csv> parsed.
2017-10-24 11:11:37,454 INFO    nx_logger:38 - Writing scan data historical file
2017-10-24 11:11:37,455 INFO    nx_logger:38 - Historical file written.
2017-10-24 11:12:43,538 INFO    nx_logger:38 - Connecting Nexpose client
2017-10-24 11:12:43,766 INFO    nx_logger:38 - Executing asset query for site(s) <['22']>
2017-10-24 11:12:43,766 INFO    nx_logger:38 - In AdHoc generate
2017-10-24 11:12:43,767 INFO    nx_logger:38 - Making Query:
<ReportAdhocGenerateRequest session-id="***************************" sync-id="***"><AdhocReportConfig format="sql"><Filters><filter type="version" id="2.0.1"/><filter type="query" id="SELECT statement removed for brevity sake as it is large      "/><filter type="site" id="22"/></Filters></AdhocReportConfig></ReportAdhocGenerateRequest>

As shown in the above log nothing has happened since yesterday, despite setting it up to pull at least every 8 hours. Site 22 contains 3166 assets.

I see no issues in any logs on the Nexpose side which leads me to believe it is an error in the app. Has anyone ran into this before? Is there a fix or some other log files that I should be looking at?

[Rapid7_Integrat]

Hello, Stephen.
In order to diagnose this issue we require snippets from two log files from the folder $SPLUNK_HOME/var/log/splunk (where $SPLUNK_HOME is your Splunk installation directory):
- splunkd.log
- TA-rapid7_nexpose.log

In addition, it would be helpful to receive the Nexpose logs from the time period in which the query was running.

Lastly, it would also be helpful if you could take the query that you've removed from the log and run it as a SQL export within Nexpose. Here is a video which explains how to run a report within Nexpose:
https://www.rapid7.com/resources/sql-reports-in-nexpose/

If you could forward the logs and the results (time taken, file size and so on) from the SQL export to support@rapid7.com, we'll be able to assist you further.

Thanks in advance.