Splunk Search

How do I create a time chart that predicts based on the values inside the search field?

aflick2486
Explorer

I am trying to create a search that looks through some logs and creates a time chart based on the search field which is a domain address like 'global.ymtracking.com' in Splunk Enterprise 6.4.3. I have a search already started which is:

index=dns source="/var/log/named/rpz.log"   | search src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24 | timechart count by query

I want to do something like this:

index=dns source="/var/log/named/rpz.log"   | search src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24 | timechart count by query as foo | predict foo

And then create an alert if one of the values goes above the predicted value on any given day. The error I get with the above is that you can't do timechart count by query as foo you would have to do timechart count(query) as foo, which is not exactly what I am looking for. I am not sure how this would be possible and any help would be greatly appreciated.

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try (may not perform best). The predict command is non-streaming command which can work on single,fixed name field.

index=dns source="/var/log/named/rpz.log"   src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24
| stats  count by query | table query 
|map search="search index=dns source=\"/var/log/named/rpz.log\"  src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24  query=\"$query$\" | timechart count | predict count | rename count as \"$query$\"" | timechart values(*) as *

View solution in original post

somesoni2
Revered Legend

Give this a try (may not perform best). The predict command is non-streaming command which can work on single,fixed name field.

index=dns source="/var/log/named/rpz.log"   src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24
| stats  count by query | table query 
|map search="search index=dns source=\"/var/log/named/rpz.log\"  src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24  query=\"$query$\" | timechart count | predict count | rename count as \"$query$\"" | timechart values(*) as *

View solution in original post

aflick2486
Explorer

This is close to what I am looking for, however all of the predictions are equal to 0 and it seems to predict on all of the domains counted together. Is it possible to predict based on only one value inside the query field?

0 Karma

cmerriman
Super Champion

wouldn't this work:
| timechart count as foo by query | predict foo

https://docs.splunk.com/Documentation/Splunk/6.6.0/SearchReference/Predict

0 Karma

aflick2486
Explorer

When I go to the visualization tab, this gives me the error:

command="predict", Unknown field: foo

0 Karma

niketnilay
Legend

Predict command can work on single time series data which means you can either aggregate all domain address together or pass on only one address at a time i.e

| timechart count as foo
| predict foo

PS : I would also remove | search from the query so that src_ips to be excluded are removed from the base search itself.

Do read about various algorithms for predict command like LLP LLT etc.

If you want to predict categorical field then check out Machine Learning Toolkit app on Splunkbase, which uses algorithms like LogisticRegression, SVM etc.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

aflick2486
Explorer

So that would pass in the first domain and then how would you pass in the others?

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.