Searching DNS queries into reports from Splunk Stream

[brian1_tate]

So I know there is a newer app called Stream. It has a massive amount of DNS queries from 100 hosts at least in Stream. If I need to pull data from that to generate the report, how can I narrow the DNS queries that Stream has captured since malware is using internal DNS and we have no clue which to look for. This may be a case of DNS tunneling but Steam doesn't have friendly means to 'search this from within ES'. While I haven't been given a time of day or the days in question, its probably daily. I need to be able to not only have myself do this by users that are not going to use Splunk SPL. The are used to using ArcSight and it constantly displays current, live data, not static data that has been search. Then they build a report or a trend and get that automatically sent to them via email in ArcSight. This is the expectation for enterprise management, incident response or even MSSP who do not want their clients accessing Splunk!

Why Splunk is at the top of Gartner Quadrant (I have no idea why) - their certs training focus on too much sales scenarios. Rarely does a SIEM serve the purpose of sales (lack of insight on cert training). What we need is to, show me DNS records over a period of time, display the src host and where are they sending that request to because I can't t urn on debugging modes in a network that spans the nation and globe. I need some practical scenarios where one would be able to search for DNS logs performed on what host, to where, by requested by what client because we have a serious incident that we need to figure out across 1 million hosts or more where this potential DNS tunneling is coming to. I can't get the Stream app to really dive down into this as most people want reports, they do not or cannot log into Splunk. Has anyone used Stream and and ES that can comment how they have done this?

Thx

[jbrodsky_splunk]

Hello @brian1_tate, thanks for checking out Stream.

I think Splunk Stream + core Splunk is probably the best way forward for your particular use case of finding malicious DNS communications on your internal network. ES is not going to really add much more functionality here.

We have developed several bits of material that will be directly relevant to what you want to do. They all revolve around using Stream to capture the data (query logging from your DNS servers, while that will accomplish the same thing at the end of the day, will likely greatly annoy your server administrators.)

Here's what I would look at:

"Hunting the Known Unknowns with DNS" by Ryan Kovar and Steve Brant from our security practice:
https://conf.splunk.com/session/2015/conf2015_SBrant_RKovar_Splunk_SecurityCompliance_HuntingKnownUn...

"Random Words on Entropy and DNS" also by Ryan Kovar
http://blogs.splunk.com/2015/10/01/random-words-on-entropy-and-dns/

The recent SplunkLive security hands-on sessions, delivered by myself and others - which have a section on DNS exfil:
http://www.slideshare.net/Splunk/splunk-enterprise-for-infosec-handson-breakout-session-65534395

On that last one, there's a complementary document that you can review that has some good example searches to find malicious DNS comms - see the text on the bottom of slide 61. Hope that helps.

In my humble opinion - this part NOT speaking for Splunk - it makes sense for us to roll some of this functionality into ES in the "Advanced Threat" section of that product. I'll bring this up with our product managers.

[rkovar_splunk]

Hey Brian! You may also want to take a look at this SANS whitepaper where the author goes into quite a bit of detail on DNS tunneling and using Splunk to detect it

SANS whitepaper on detecting DNS tunneling

[Splunker]

Hey Brian,

There are a few different ways to skin this cat.

You've gone in the Stream direction, another way to go is you could turn on "query logging" on your DNS servers and use a Splunk Universal-Forwarder to collect the query-logs, with the appropriate TA, and ES will see them (assuming the TA you use is CIM-compliant, which most are).

If you cannot collect DNS query-logs (say, in the case of a DNS appliance) then Stream would be your next option.

You essentially need to bring all those logs into Splunk, wether via a UF or Splunk Stream, then ES can monitor it. Additionally you can do your own analytics as well (if you wish to).

If you decide to go with Stream, naturally it will need to tap (sniff) the right part of your network, and you'll need to create a new DNS Stream (Configure Streams->New Stream->DNS) (apologies dont have a Stream installation handy, but hopefully you get the idea..), give it a name, and tick the fields you want Stream to collect for this particular stream. Stuff like src_ip, host_addr, query, message_type etc.

By ticking those fields, your really saying which parts of a DNS packet you want Stream to log (the query, the source IP, the TTL, etc..)

Once that stuff is indexed in your Splunk, search for 'index=* tag=dns' if it's CIM-compliant, you should see results from this search. You can substitute the index-name of your Stream DNS logs.

Here's a useful use-case from the ES docs, about how to use DNS to identify "patient-zero" for a malware infection.

http://docs.splunk.com/Documentation/ES/4.2.0/Usecases/PatientZero

Hope it helps 🙂

[Splunker]

You will need to make a new Stream (DNS) configuration (Configuration->Configure Streams) in the Stream app.

You can add all the fields you need, including the src_ip, query, message_type, etc, for later reporting/dashboarding/alerting.

Enterprise-Security has the Advanced-Threat->Protocol-Intelligence->DNS Activity (and DNS Search) views to see an overview of DNS traffic.

You could also add your own view, showing src_ip and domains-queried as you need, in a summarised (readable) format, and add it into the views of ES, so people don't have to mess with SPL.

[brian1_tate]

Forgive me but I'm a newbie with Stream. I've got 46 configured streams for just about everything.

The problem is that we have no idea what hosts are infected (or maybe). We have 100 internal DNS servers so I have no idea where to begin. I need a simple way to generate a scheduled report perhaps for each DNS server with its source IP, the query it performed and so on so we can start narrowing down what we are looking for. This malware hits the internal DNS and that's just about all we know at this time.

Any suggestions on a query I might use to start this off until we can determine what hosts are the cause of the problem? Once I get more information I can start creating alerts and let the IRT sort though 75k of records that are generated in 15 minutes....

Thx