All Apps and Add-ons

Import from Splunk Tenable data takes long

Path Finder

Everything works fine with the import. But it takes a long time to import all my data via API into Splunk.
Per 15min approximitly 1000 new Events after the scan is finished. So if I have about 600'000 Scan-Events it takes almost a week.
Is this normal? Where can I improve it?

Any idea? I can't find any errors in the log.
thanks for helping.

0 Karma

Ultra Champion

Importing scans from Security Center is time consuming, but in theory, once you are up to date, its only the delta your importing on each run - unless your saying you have 600,00 results per scan?

If you don't need to import all events you can change the window from which Splunk will read from the API.

From time-time the Nessus scripts fall over, and I have to restart them - When I do so, i tend to bring the window forward (until just before it stopped) This keeps the delay down.

Another problem I have is that importing large numbers of events with the same time stamp (because that's how nessus does it), I get the following at search:

[indexerName] Events may not be returned in sub-second order due to search memory limits configured in limits.conf:[search]:max_rawsize_perchunk. See search.log for more information.

I suspect this has similar performance implications at index time too, which may well be contributing to the slow import. times.

Sadly, in my experience this is normal, and I have not found a way to improve it. (yet)

If my comment helps, please give it a thumbs up!
0 Karma

Path Finder

Thanks for your answer and very sorry for my delay.
We don't use the security center, we use the api of the Nessus Professional v6.
Do you have any idea why we get all the Events every time..? Is there some kind of option?

The second problem does not occur, I guess because we only get 1000 Events per 15 min ;-).

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...