topic How to make Stream logs CIM compliant in Splunk Enterprise Security

How to make Stream logs CIM compliant

akhalfan — Wed, 30 Sep 2020 04:28:09 GMT

I've used Splunk Stream app to get DNS logs from a Windows DNS server. I got the logs to a Search Head instance that has the Enterprise Security app. However, I can't seem to the data, which is in json format CIM compliant.

Below is a sample message raw log:

What would be the best way to make the query field CIM compliant with the query field in the DNS as mentioned here:

https://docs.splunk.com/Documentation/CIM/4.15.0/User/NetworkResolutionDNS
{"endtime":"2020-03-04T16:13:55.892181Z","timestamp":"2020-03-04T16:13:55.886950Z","bytes":237,"bytes_in":35,"bytes_out":202,"dest_ip":"8.8.8.8","dest_mac":"00:15:5D:FA:54:6B","dest_port":53,"flow_id":"d53fcb9a-ea29-4761-ac1a-de6ca66d31e4","host_addr":["104.115.41.252"],"hostname":["www.microsoft.com-c-3.edgekey.net","www.microsoft.com-c-3.edgekey.net.globalredir.akadns.net","e13678.dspb.akamaiedge.net"],"message_type":["QUERY","RESPONSE"],"name":["www.microsoft.com","www.microsoft.com-c-3.edgekey.net","www.microsoft.com-c-3.edgekey.net.globalredir.akadns.net","e13678.dspb.akamaiedge.net"],"protocol_stack":"ip:udp:dns","query":["www.microsoft.com"],"query_type":["A"],"reply_code":"NoError","reply_code_id":0,"response_time":5231,"src_ip":"14.33.31.16","src_mac":"AA:AA:BB:BB:00:51","src_port":65031,"time_taken":5231,"transaction_id":4481,"transport":"udp","ttl":[2265,4012,461,20]}

Re: How to make Stream logs CIM compliant

zacharychristen — Fri, 06 Mar 2020 15:43:50 GMT

The Splunk Stream Add-on automatically extracts the query field.

Run the following search in your dns index and verify that the "dns, network, and resolution" tags appear:

index=<YOUR DNS INDEX> sourcetype="stream:dns" | stats count by tag

If not, verify that you have the Splunk Stream Add-on installed: Navigate to Apps > Manage apps. The Splunk Stream Add-on is in the same packaged app as the Splunk Stream app but sometimes the add-on does not properly install.

If you have the Add-on installed and the tags are showing up, verify that you have your DNS index whitelisted in the CIM setup. This can be done in Enterprise Security by Navigating to Configure > CIM Setup. Click the Network Resolution Data Model and verify the DNS index is whitelisted.

Re: How to make Stream logs CIM compliant

akhalfan — Fri, 06 Mar 2020 19:49:26 GMT

For this query
index= sourcetype="stream:dns" | stats count by tag
only the DNS tag appears (makes sense since it is the only one I actually tagged in the event types.

The index is whitelisted. However, I noticed that the following searh:

index=<dns_index> datamodel=Network_Resolution.DNS  where DNS.message_type=QUERY

I get "no results found"

Re: How to make Stream logs CIM compliant

zacharychristen — Fri, 06 Mar 2020 19:52:28 GMT

When you look at your installed apps do you see the Splunk Stream Add-on or do you just see the App?

Re: How to make Stream logs CIM compliant

akhalfan — Fri, 06 Mar 2020 22:11:00 GMT

installing the add-on fixed it, I don't need to install the app
thanks

Re: How to make Stream logs CIM compliant

mokuso — Wed, 10 Jun 2020 21:35:29 GMT

I had a similar issue. The results of the above query only returned the "communicate" and "network" tags.

It appears that the default/eventtypes.conf is ignoring "stream:Splunk_*" which excludes "stream:Splunk_DNS*" events:

[stream_dns] search = sourcetype=stream:dns NOT source=stream:Splunk_*

I added the following to local/eventtypes.conf and it resolved the issue:

[stream_dns] search = sourcetype=stream:dns

Now the dns events return all 4 tags: communicate, dns, network, resolution