Import from Splunk Tenable data takes long

[benlc]

Everything works fine with the import. But it takes a long time to import all my data via API into Splunk.
Per 15min approximitly 1000 new Events after the scan is finished. So if I have about 600'000 Scan-Events it takes almost a week.
Is this normal? Where can I improve it?

Any idea? I can't find any errors in the log.
thanks for helping.
Ben

[nickhills]

Importing scans from Security Center is time consuming, but in theory, once you are up to date, its only the delta your importing on each run - unless your saying you have 600,00 results per scan?

If you don't need to import all events you can change the window from which Splunk will read from the API.

From time-time the Nessus scripts fall over, and I have to restart them - When I do so, i tend to bring the window forward (until just before it stopped) This keeps the delay down.

Another problem I have is that importing large numbers of events with the same time stamp (because that's how nessus does it), I get the following at search:

[indexerName] Events may not be returned in sub-second order due to search memory limits configured in limits.conf:[search]:max_rawsize_perchunk. See search.log for more information.

I suspect this has similar performance implications at index time too, which may well be contributing to the slow import. times.

Sadly, in my experience this is normal, and I have not found a way to improve it. (yet)

[benlc]

Thanks for your answer and very sorry for my delay.
We don't use the security center, we use the api of the Nessus Professional v6.
Do you have any idea why we get all the Events every time..? Is there some kind of option?

The second problem does not occur, I guess because we only get 1000 Events per 15 min ;-).