Ingesting all WIZ vulnerabilities

[_joe]

I am trying to use this app to ingest vulnerabilities but it seems like I am getting rate-limited. Anyone else having this problem?

[nachimistry]

Thanks, that detail helps. Counts landing on exact thousands make me think this isn't a 429 at all. It looks more like the pull stops paginating early, either because it hits a page or result cap or because a run times out partway through, so you get N full pages and nothing after.

A few things to check:

1. Compare what landed with what Wiz shows. Run this over one full pull window, and swap in your index, sourcetype and vulnerability ID field:

index=<your_wiz_index> sourcetype=<your_wiz_vuln_sourcetype> earliest=-24h | stats count dc(<vuln_id_field>) as unique_vulns by severity

If unique_vulns sits on a round number while the Wiz console shows more, the pull is getting cut off.

1. Find out where the add-on writes its own log. It may not match source=*wiz* in _internal, which would explain the empty result. Check $SPLUNK_HOME/var/log/splunk/ for a Wiz-named file, then run:

index=_internal source=*wiz* | stats count latest(_time) as last_seen by source | convert ctime(last_seen)

1. If the add-on has a log-level setting, set it to DEBUG for one run and see whether the last request still reported more pages, and whether the run ended with an error or a timeout.
2. The four inputs probably share one Wiz service account, so they share its API quota. Stagger their start times, and make sure each run finishes before its next interval, so two pulls of the same severity never overlap.

If DEBUG shows it stopping while Wiz still reports more pages, that's worth raising with the add-on maintainer, along with the exact counts from both sides.

[_joe]

Thanks for the feedback.

It seems the problem was an app bug that was supposed to have been resolved in 1.4.8. Prior, it seems that the token would timeout while the app was going through pagination, then it wouldn't try to re-authenticate.

[nachimistry]

Thanks for closing the loop, that explains the round numbers: each run got through some full pages before the token expired, then quietly stopped. Since you're already on 1.4.8, it's worth confirming the fix actually landed for you. After the next full pull, compare per-severity counts with the Wiz console:

index=<your_wiz_index> sourcetype=<your_wiz_vuln_sourcetype> earliest=-24h
| stats dc(<vuln_id_field>) as unique_vulns by severity

If Low still stops at a round number like 75,000, the re-auth isn't kicking in on long runs, and splitting into four severity inputs is mostly helping by keeping each run shorter than the token lifetime. Did the counts match after the upgrade?

[_joe]

Thanks for the feedback.

I am on Wiz Add-on For Splunk Version: 1.4.8 and the latest  Splunk Enterprise10.4.x. Originally I had one input pulling once an hour, currently I have four inputs broken out by severity (Critical, High, Medium, and Low).

I am not getting from your search. I assumed it was being rate limited on the WIZ side because my results were coming by rounded off to the nearest thousand.  (i.e. 75,000 Low severities) .

[nachimistry]

I haven't seen the add-on documentation spell this out, but full vulnerability pulls from the Wiz API are heavy, and a few things usually help. First, confirm it really is rate limiting and see when it happens:

index=_internal source=*wiz* (429 OR "rate limit" OR "Too Many Requests") | timechart span=15m count

Then:

• Lengthen the input interval. A full pull every few minutes will hit limits. Run the full pull daily and use incremental or updated-since filtering, if the add-on offers it, for anything more frequent.
• Avoid overlapping inputs. Two inputs using the same Wiz service account share one rate limit.
• Reduce page size or scope. Filter by severity or project if the input supports it.
• Check the add-on's own log for the exact API call that fails. That tells you whether it's the vulnerability query or the auth or token endpoint.

Which add-on version are you on, and what interval is the input set to?