Splunk Enterprise Security

Splunk Enterprise Security Alexa Domain NOT lookup failing

Explorer

Hello,

I am trying to create alerts for all outbound DNS queries which do not match the top one million domains as per Alexa top 1 million which comes shipped with ES.

I am using the following search which I am fairly certain is correct as per Splunk base and previous examples of creating searches with lookups.

'index=externaldns | rex field=query "(?[A-z0-9]+.[A-z0-9]+$)"
| fields domain | search NOT [ | inputlookup alexa_by_str.csv ]| stats count by domain'

The result of this search was showing what looked to be the queries which DID match the top 1 million alexa domains. When looking into the job status of the search I saw an error message saying that the results were truncated to maxout 10000 (Splunks default) for searches: https://docs.splunk.com/Documentation/Splunk/7.2.3/Admin/Limitsconf

I then changed both the maxout for the search AND subsearch to 12000000 in the limits.conf local file in my sh_cluster app to override the default and I am now receiving the following error message:

'[subsearch]: Search Processor: Subsearch produced 1002192 results, truncating to maxout 50000.'

Can someone tell me if I have changed the wrong config or if there is anything more than I should do to increase the maxout?

0 Karma
1 Solution

Motivator

I would suggest to use a slightly different method here, as increasing the subsearch limits may clog your system. If I understand your use case, you want to get a list of all domains which are NOT included in the Alexa Top 1Mio and count these.

This can be done like:

'index=externaldns | rex field=query "(?[A-z0-9]+.[A-z0-9]+$)"
| fields domain 
| append [
    | inputlookup alexa_by_str.csv
    | eval is_alexa=1 ]
| stats count as count, sum(is_alexa) as is_alexa by domain
| where is_alexa=0
| table domain count

View solution in original post

Motivator

I would suggest to use a slightly different method here, as increasing the subsearch limits may clog your system. If I understand your use case, you want to get a list of all domains which are NOT included in the Alexa Top 1Mio and count these.

This can be done like:

'index=externaldns | rex field=query "(?[A-z0-9]+.[A-z0-9]+$)"
| fields domain 
| append [
    | inputlookup alexa_by_str.csv
    | eval is_alexa=1 ]
| stats count as count, sum(is_alexa) as is_alexa by domain
| where is_alexa=0
| table domain count

View solution in original post

Explorer

That's worked perfectly DMohn. Many thanks!

Motivator

You're welcome, glad to be of assistance!

0 Karma

Explorer

Ah, just had another run through of the search there. The search works fine when setting the where = 1, as expected it displays all the dns queries made which match the alexa domain lookup. However, when setting the where = 0 there are no results found.

At first glace I'd thought that maybe it was possible that there no DNS queries made which aren't in the alexa lookup. However, I tested this by doing an nslookup on a domain NOT in the alexa domain lookup and then ran the search and still no results were found. The events for the nslookup are in the index but aren't showing up in the lookup search where alexa = 0.

Any thoughts?

0 Karma

Motivator

Try changing the where is_alexa=0 to where isnull(is_alexa)

0 Karma