Getting Data In

DMC ALert - Missing forwarder for zombie entry

afx
Contributor

Hi,
I am getting a DMC alert for a missing forwarder even though that one (at least by client name, but not UUID) exists.

Backgroud story:
Forwarder was deleted completely on source system (don't ask...).
Re-Installed from scratch with same parameters (of course, still different UUID, but same client name) and slightly newer code version.
Rebuilt the forwarders DB on the deployment server (Monitoring Console -> Settings -> Forwarder Monitoring Setup -> Rebuild forwarder assets).
Still I get errors about the missing forwarder and it shows the old version number for the code.

So where is this Zombie stored and how do I get rid of it?

thx
afx

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi afx,
did you tried to rebuild the lookup "dmc_forwarder_assets" [Settings -- Forwarder Monitoring Setup -- Rebuild Forwarder Assets].
In this way you rebuild your lookup with only the UFs that are really sending logs in the last period (configurable).

Bye.
Giuseppe

0 Karma

afx
Contributor

Hi Guiseppe,
yes, as described in my post.

But, your post suddenly triggered my leaky memory.
It is not the deployment server where these alerts are generated but the search head for various strange reasons (modifed DMC alert with some lookups that are only maintained on the search head that make the alert more informative).

So once I did the DB refresh on the search head the world went quiet again.

thx
afx

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi afx,
if this answer solved your problem (or helped to do), please accept and/or upvote it.
Bye and see next time.
Giuseppe

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...

Monitoring AI Agents with Splunk Observability Cloud

Let’s say I’m running a travel planning AI app in production. A user asks for three concise hotel options in ...

[Puzzles] Solve, Learn, Repeat: Tiling

This puzzle (first published here) is based on finding groups of tessellated tiles (inspired by floor tiles I ...