For all reported universal forwarders ( Linux and Windows) how do I verify the sources and sourcetype?

[kagamalai]

Hi,

There are more than 1000 UF Windows and Linux systems installed. It is a distributed environment with around 100 systems at each location, one indexer deployed, and each indexer connected to a search head.

Our next step is to verify that all the hosts have been configured properly and are reporting to the indexer.

In cases where a host does not have the source or sourcetype, we need to update the list to match host and not match host against the below lookup table.

Could someone please suggest the spl.

source	sourcetype
WinEventLog:Security	WinEventLog
WinEventLog:Application	WinEventLog
WinEventLog:System	WinEventLog
/var/log/haproxy/haproxy.log	haproxy
/var/log/audit/audit.log	audit
/var/log/maillog	postfix_syslog
/var/log/messages	linux_messages_syslog
/var/log/cron	cron

Thanks

Manickam

[gcusello]

Hi @kagamalai,

your question contains two items:

• reporting of connected/Not connected systems;
• configuration checks.

About the first problem, you could use the alert contained in the Monitoring Console that checks the missing Forwarders, 

Otherwise, you could create a lookup containing the hosts in the perimeter to monitor (called e.g. perimeter.csv) and schedule an alert like this:

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter.csv | eval host=lower(host), count=0 | fields host count]
| stats sum(count) AS total By host
| where total=0

This second way gives you more control but is heavier than the other because you have to manually maintain the lookup.

About the second part, how do you manage your forwarders? are you using a Deployment Server?

If yes, you have to check the deployed configurations, if not I hint to use it.

Anyway, you should check, one by one, your flows (identified by sourcetype) anche check if you have parsing errors.

To avoid them, try to use the standard addons for Windows and Linux.

Ciao.

Giuseppe

[kagamalai]

Thank you for your reply,

• reporting of connected/Not connected systems;  --- I have spl to check and installed and not installed UF status.
• configuration checks. What is the best way to check all three or 5 logs reporting to indexer hostwise? We need to identify the host if one of the logs is missing on any host. 

Thanks

Manickam

[gcusello]

Hi @kagamalai,

as I said, if you deploy the standard Technical Add-Ons (TA) for Windows and Linux using the Deployment Server, you're sure that the deployed configurations are OK and that they are deployed to all the systems.

then you could run some simple search as

index=wineventlog
| stats dc(sourcetype) AS dc_sourcetype values(sourcetype) AS sourcetype count BY host

so you're sure that you're receiving all the sourcetypes in wineventlog; the dc_sourcetype value immediately identifies if there are someone missing.

Then running the same search on the windows and the os indexes you're sure that all the sourcetypes are present.

Ciao.

Giuseppe

[kagamalai]

Thank you, but we do not manage the UF using the deployment server, TA Add-on.

We have manually installed the UF and configured the log files, so the reason we want to verify each has all logs reporting to the indexer is that all the logs need to be configured correctly.

[gcusello]

Hi @kagamalai,

Forwarders installation must be manual, eventually using a script (for Linux) and a tool as Ansible (for Windows).

Recently two apps to upgrade forwarders were released but I didn't still used them. 

But anyway for configurations, this isn't a good practice and it's a very heavy additional work for you, so I hint to plan the use of DS.

Ciao.

Giuseppe