dest=unknown in ES

[richardphung]

We are having an issue with our Splunk ES instance where notables that have dest = unknown, all show up in our ESS Incident Review page as having the same IP address and MAC Address under the "Additional Fields" section.
Most of these notables have destination = null, meaning that the original log message (e.g. opsec:antibot or opsec:antivirus) does not contain any fields pertaining to user OR destination IP address.

For example:
Host With Recurring Malware correlation search returns $dest$ = null/unknown,
But in ESS, it shows "Destination IP Address: xxx.xxx.xxx.xxx"

I think this is because of our asset lookup definition.

We have 3-4 different asset lookups that are powered by scheduled searches against our:
-- Network Access Control system
-- DHCP registrations
-- Patch Management system

However, I can't seem to find a place where unknown would be defined.
Could anyone point me in the right direction?

[lakshman239]

For ES to correlate against assets and identities, it would be good to have clean asset data with no 'unknown' values in the host fields -eg nt_host.

Also, when you have multiple assets lookups [ populated by one or more saved searches, LDAP queries etc..], it would be good if you can merge them together to one master asset table for ES to refer to. Pls refer to SA-IdentityManagement/default/macros.conf and refer to asset_sources macro. You will need to define a file under local and add your inputs.conf and macros.conf.

Pls refer to https://docs.splunk.com/Documentation/ES/5.3.0/Admin/Howassetandidentitydataprocessed and following sections to create and validate merged assets.

[richardphung]

I found this:
When looking at the asset lookup list, I see the offending IP mapped to unknown nt_host:

| inputlookup dhcp_assets  | search nt_host="Unknown"

I also see another host with Unknown in the nt_host field...

I think I should add an eval to the DHCP Asset Lookup Gen saved search so that it will rename "Unknown" to something else.. like "Unknown Host"...

Here is my lookup definition:

index=os sourcetype="isc:dhcp" action=added
| stats latest(dest_host) as nt_host latest(dest_ip) as ip by dest_mac
| rename dest_mac AS mac
| fields ip, mac, nt_host
| rex field=nt_host "^[^\.]+\.(?P<dest_domain>.+)"
| rex field=nt_host "^(?<nt_host>[^\.]+)"
| eval dns=case(isnotnull(dest_domain),nt_host+"."+dest_domain)
| eval city="" 
| eval country="" 
| eval pci_domain=""  
| eval is_expected=""  
| eval should_timesync=""  
| eval should_update=""  
| eval requires_av=""  
| eval owner="" 
| eval priority=case(
     [TRUNCATED EVAL CASES]
    ,"high")
| eval category=case(
         [TRUNCATED EVAL CASES]
    isnotnull(dns), "Domain joined device") 
| table ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av
| outputlookup create_empty=false createinapp=true  dhcp_assets.csv

Maybe add something like...

| eval nt_host= replace(nt_host,"Unknown", "Unknown Host")
After: "| rex field = ...."?