Asset Lookup in Malware Datamodel

[richardphung]

We are using ES with a datamodel that has the base constraint:

    (`cim_Malware_indexes`) tag=malware tag=attack

This drives correlation searches like: Endpoint - Recurring Malware Infection - Rule

| tstats summariesonly=false allow_old_summaries=true dc(Malware_Attacks.date) as "day_count",values(Malware_Attacks.user) as "user",values(Malware_Attacks.category) as "category",values(Malware_Attacks.vendor_product) as "source",count from datamodel="Malware"."Malware_Attacks"   by "Malware_Attacks.dest","Malware_Attacks.signature" | rename "Malware_Attacks.dest" as "dest","Malware_Attacks.signature" as "signature" | where 'day_count'>3

This creates notables, as expected, however, I see that many notables do not have a user associated with them.
They simply come-up as "unknown"

For example:
Signature JS/Spigot.B
User unknown

Our Datamodel has the user field with an eval expression:

if(isnull(user) OR user="","unknown",user)

This all makes sense.

However, we do have asset lookups, and often, I find myself going into the ES Incident Review page, finding the affected system, and then doing my own search against the asset and identity lookups to get this information.

The question is can we perform the asset and/or identity lookup at the time the notable is being generated?
If so, where would we write this? In the datamodel definition? or in the correlation search?

Thanks in advance.

--RP

[rashid47010]

Referring above and against malware correlation rules.
most of the time the triggered notable events have dest="unknown".
Upon investigation, i found that the unknown value events belong to sourcetype= symantec:ep:risk:file

how to fix this problem.

[richardphung]

We actually DO have this information.
We have several Asset Lookups, such as:

| inputlookup patchmgmt_assets 
| inputlookup dhcp_assets 
| inputlookup nac_assets 
| inputlookup vmware_assets

In which the "dest" field could be matched with either ip or nt_host (according to CIM), and the owner would be the "user" in the context of the Malware notable.

[lakshman239 · Accepted solution]

You could check a few things to narrow down the issue and help resolve them, to the extent possible.

If you run cim_Malware_indexes tag=malware tag=attack (same as your search) in the search within ES, are you seeing user=unknown? if so, you can narrow down the events that are not having a proper user field OR not getting correctly extracted and mapped to 'user'. Resolving them will improve your coverage and reduce 'unknown'.

On your query, yes, you can use asset and identities search as part of your correlation search (notable) , by using macro like
| eval src="1.2.3.4" | get_asset(src)

eval user="abc" |get_identity4events(user)

[rashid47010]

hi lakshman,

I am facing the same issue for user and dest field. most of the time either dest OR user filed is unknown. when I drill down further I found that those "unknown" field events belongs to sourcetype=symantec:ep:risk:file.

interestingly, for some events the dest field ( i think which is being extracted from "Computer name") is present there.

please support here to conclude this.

[lakshman239]

you would need to ensure that your asset lookup table has all the assets in your organization [ which can come in any events, e.g Malware, network etc..]. Similary you need to capture all users in identity. Once this is available, when you use any TA's [ e.g. symantec], you can validate if the data is getting extracted using CIM /datamodel, e.g. | from datamodel:"Authentication.Authentication" will show all fields extracted. If a field [ e.g user or dest] is not extracted, you will need to add/update in local/props.conf of the add-on/app which you are using [ in this case for you symantec TA]. once this is done, all should work.

[richardphung]

I am unsure how to use the macro you refer to: get_asset(src)

It occurs to me that the assets are included in a lookup, couldn't this be done as a subsearch?

e.g. | search [| inputlookup assets.csv | rename host AS dest]

[richardphung]

Or something like...

 |   eval user=replace(user, "unknown", "") | join type="left" dest user [| inputlookup assets.csv | rename host AS dest | rename owner AS user | fields dest user]

[lakshman239]

The macros help you to link back to your assets and identities (lookup files defined in your environment). e.g   index=yourindex user=* | `get_identity4events(user)`  will match the user to your identity lookup and output fields like user_first, user_last, user_category, and other user_* fields. The same approach will hold for asset macro. hope this helps.

[richardphung]

OK. I had the macro syntax incorrect.
get_asset(src) does return some values, e.g. dest_asset_id, dest_asset_tag, and so forth..
however, "user" still appears as "unknown" despite at least 2 of our asset lookups containing "owner" information

So back to the original issue...

[lakshman239]

use get_identity4events(user) to link back to identities lookups

[richardphung]

get_identity4events(user) if user="unknown" will not produce anything.

However, if dest = "computer_name"
And I do:
| inputlookup dhcp_assets | search nt_host="computer_name"
OR
if dest = "computer_ip", I can find the identity by doing: | inputlookup dhcp_assets | search ip="computer_ip"
I get owner=someusername

In this context, owner=user

[lakshman239]

just to clarify terminology, asset is a computer/IP/src/dest and identity is user/src_user. So you populate asset lookups with only assets and identity lookups with only identities. Both are different. In your datamodel searches, you can either link back to asset or identity or both to match fields in the events to your assets or identities.

[richardphung]

Thank you for the clarification!

In this context, the user information is contained in one of our asset lookups as "owner"
This would match the notable event dest as either nt_host OR ip

So something like...
index="notable" search_name="*Malware*" user="unknown" | join dest [|inputlookup nac_assets | rename ip AS dest | rename owner AS user]
Appropriately finds & assigns the "owner" as "user"

However, I would ideally like to perform this look-up/match at the time the correlation search runs.

[lakshman239]

ok, so you need the asset owner. so, you could do something like which will take your dest and connect to your asset and provide the owner field.

index="notable" search_name="*Malware*" | `get_asset(dest)` | table dest, dest_owner

[richardphung]

this did it!
get_asset(dest) | rename dest_owner AS user

I added this to the correlation search:
| tstats summariesonly=false allow_old_summaries=true dc(Malware_Attacks.date) as "day_count",values(Malware_Attacks.user) as "user",values(Malware_Attacks.category) as "category",values(Malware_Attacks.vendor_product) as "source",count from datamodel="Malware"."Malware_Attacks" by "Malware_Attacks.dest","Malware_Attacks.signature" | rename "Malware_Attacks.dest" as "dest","Malware_Attacks.signature" as "signature" | where 'day_count'>3 | get_asset(dest) | rename dest_owner AS user