topic Re: How do I create a time chart that predicts based on the values inside the search field? in Splunk Search

How do I create a time chart that predicts based on the values inside the search field?

aflick2486 — Thu, 18 May 2017 18:16:20 GMT

I am trying to create a search that looks through some logs and creates a time chart based on the search field which is a domain address like 'global.ymtracking.com' in Splunk Enterprise 6.4.3. I have a search already started which is:

index=dns source="/var/log/named/rpz.log"   | search src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24 | timechart count by query

I want to do something like this:

index=dns source="/var/log/named/rpz.log"   | search src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24 | timechart count by query as foo | predict foo

And then create an alert if one of the values goes above the predicted value on any given day. The error I get with the above is that you can't do timechart count by query as foo you would have to do timechart count(query) as foo, which is not exactly what I am looking for. I am not sure how this would be possible and any help would be greatly appreciated.

Re: How do I create a time chart that predicts based on the values inside the search field?

cmerriman — Thu, 18 May 2017 18:42:47 GMT

wouldn't this work:
| timechart count as foo by query | predict foo

https://docs.splunk.com/Documentation/Splunk/6.6.0/SearchReference/Predict

Re: How do I create a time chart that predicts based on the values inside the search field?

aflick2486 — Thu, 18 May 2017 18:46:53 GMT

When I go to the visualization tab, this gives me the error:

command="predict", Unknown field: foo

Re: How do I create a time chart that predicts based on the values inside the search field?

niketn — Thu, 18 May 2017 20:35:57 GMT

Predict command can work on single time series data which means you can either aggregate all domain address together or pass on only one address at a time i.e

| timechart count as foo
| predict foo

PS : I would also remove | search from the query so that src_ips to be excluded are removed from the base search itself.

Do read about various algorithms for predict command like LLP LLT etc.

If you want to predict categorical field then check out Machine Learning Toolkit app on Splunkbase, which uses algorithms like LogisticRegression, SVM etc.

Re: How do I create a time chart that predicts based on the values inside the search field?

aflick2486 — Thu, 18 May 2017 20:45:15 GMT

So that would pass in the first domain and then how would you pass in the others?

Re: How do I create a time chart that predicts based on the values inside the search field?

somesoni2 — Thu, 18 May 2017 21:16:56 GMT

Give this a try (may not perform best). The predict command is non-streaming command which can work on single,fixed name field.

index=dns source="/var/log/named/rpz.log"   src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24
| stats  count by query | table query 
|map search="search index=dns source=\"/var/log/named/rpz.log\"  src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx AND src_ip!=xxx.xxx.xxx.xxx/24 AND src_ip!=xxx.xxx.xxx.xxx/24  query=\"$query$\" | timechart count | predict count | rename count as \"$query$\"" | timechart values(*) as *

Re: How do I create a time chart that predicts based on the values inside the search field?

aflick2486 — Fri, 19 May 2017 15:55:41 GMT

This is close to what I am looking for, however all of the predictions are equal to 0 and it seems to predict on all of the domains counted together. Is it possible to predict based on only one value inside the query field?