How to make Stream logs CIM compliant

[akhalfan]

I've used Splunk Stream app to get DNS logs from a Windows DNS server. I got the logs to a Search Head instance that has the Enterprise Security app. However, I can't seem to the data, which is in json format CIM compliant.

Below is a sample message raw log:

What would be the best way to make the query field CIM compliant with the query field in the DNS as mentioned here:

https://docs.splunk.com/Documentation/CIM/4.15.0/User/NetworkResolutionDNS
{"endtime":"2020-03-04T16:13:55.892181Z","timestamp":"2020-03-04T16:13:55.886950Z","bytes":237,"bytes_in":35,"bytes_out":202,"dest_ip":"8.8.8.8","dest_mac":"00:15:5D:FA:54:6B","dest_port":53,"flow_id":"d53fcb9a-ea29-4761-ac1a-de6ca66d31e4","host_addr":["104.115.41.252"],"hostname":["www.microsoft.com-c-3.edgekey.net","www.microsoft.com-c-3.edgekey.net.globalredir.akadns.net","e1367...}

[zacharychristen · Accepted solution]

The Splunk Stream Add-on automatically extracts the query field.

Run the following search in your dns index and verify that the "dns, network, and resolution" tags appear:

index=<YOUR DNS INDEX> sourcetype="stream:dns" | stats count by tag

If not, verify that you have the Splunk Stream Add-on installed: Navigate to Apps > Manage apps. The Splunk Stream Add-on is in the same packaged app as the Splunk Stream app but sometimes the add-on does not properly install.

If you have the Add-on installed and the tags are showing up, verify that you have your DNS index whitelisted in the CIM setup. This can be done in Enterprise Security by Navigating to Configure > CIM Setup. Click the Network Resolution Data Model and verify the DNS index is whitelisted.

[mokuso]

I had a similar issue. The results of the above query only returned the "communicate" and "network" tags. 

It appears that the default/eventtypes.conf is ignoring "stream:Splunk_*" which excludes  "stream:Splunk_DNS*" events:

[stream_dns]
search = sourcetype=stream:dns NOT source=stream:Splunk_*

I added the following to local/eventtypes.conf  and it resolved the issue:

[stream_dns]
search = sourcetype=stream:dns

 Now the dns events return all 4 tags: communicate, dns, network, resolution

[akhalfan]

For this query
index= sourcetype="stream:dns" | stats count by tag
only the DNS tag appears (makes sense since it is the only one I actually tagged in the event types.

The index is whitelisted. However, I noticed that the following searh:

index=<dns_index> datamodel=Network_Resolution.DNS  where DNS.message_type=QUERY

I get "no results found"

[zacharychristen]

When you look at your installed apps do you see the Splunk Stream Add-on or do you just see the App?

[akhalfan]

installing the add-on fixed it, I don't need to install the app
thanks