How to test a correlation search?

[echojacques]

So this is the pre-configured correlation search called "substantial increase in port activity". I'd like to tweak it to our needs... but to tweak it I need to test it. When I copy and paste the actual correlation search into the Splunk Search bar it doesn't work. What am I missing? This is exactly what I'm pasting into the Search bar:

| `tstats` sum(count) from sa_port_proto groupby _time,transport,dest_port span=30m | stats sum(count) as count by _time,transport,dest_port | `timeDiff` | appendpipe [search timeDiff<=86400 | stats max(_time) as _time,sum(count) as count by transport,dest_port | eval group="Last 24 hours"] | eval group=if(_time<relative_time(time(),"@d") AND timeDiff<=5184000,"Last 60 days",group) | bin _time span=1d | stats sum(count) as count by _time,group,transport,dest_port | eval temp=if(group="Last 60 days",transport.dest_port,null()) | eventstats stdev(count) as stdev,avg(count) as avg by temp | eventstats max(stdev) as stdev,max(avg) as avg by transport,dest_port | dedup transport,dest_port sortby -_time | eval limit=(3*stdev)+avg | eval diff=count-limit | search diff>0

[chetandravid]

I have a question here how migrate correlation search to data model?

[sowings · Accepted solution]

What do you mean by "doesn't work"? Gives no results? Take off the "search diff>0" at the end--that's a filtering search, as I indicated last time. It could simply be that you've got "normal" levels of activity. Try changing the 3*stdev term to 2, and see if you have results then.

[sowings]

I believe that this search is expected to "learn" over time what the usual behavior is, so you'll only see results (now that it has learned) if something truly does exceed the averages that have previously been observed.

[echojacques]

I think I have other problems, I'm getting "splunkd daemon not responding" now. So it's probably not the search that is the problem. Thanks for the info, I'll keep testing.

[echojacques]

When I run the search, I don't get any results. I had disabled the search last week because I was getting 500+ results every time it ran. And now today, I get no results.

I tested with 1*stdev and 2*stev and removed the search diff>0 and still no results. I am also searching last 30 days.

Just confused because last week it was finding a lot and then this week nothing.