topic Re: Search Commands: analyzefields in Splunk Search

Search Commands: analyzefields

mw — Sun, 13 Feb 2011 23:44:41 GMT

I'm trying to wrap my head around some of the more advanced/esoteric search commands. It seems like there's a lot of power there if you know how to harness it (i.e. you're familiar with statistics, probability, and data mining techniques). So, seeing as I'm very much a lay person, and the documentation is a little light sometimes, I'm hoping that someone can educate us all about this command, what it does exactly, and cases where it would be useful in the real-world. My ultimate hope is to post further questions like this about... well, a lot of the search commands, in order to augment the docs a bit and make us all more powerful splunkers. So, are you using this command and, if so, for what?

Here's a related post: Question about analyzefields search command

Re: Search Commands: analyzefields

Ron_Naken — Mon, 14 Feb 2011 00:49:11 GMT

AF helps you determine how accurately each field predicts the specified field. As an example, look at the following data:

02/03/2011 01:00,st=CA,state_no=1,vote_no=1
02/03/2011 14:00,st=CA,state_no=1,vote_no=1
02/03/2011 01:00,st=MA,state_no=2,vote_no=2
02/03/2011 02:00,st=MA,state_no=2,vote_no=2
02/03/2011 07:00,st=MO,state_no=4,vote_no=1
02/03/2011 08:00,st=MO,state_no=4,vote_no=1

If you run the following search:

* | af classfield=vote_no

You can see that there is a 100% chance (1.0) that my state (state_no) will predict my vote (vote_num), by looking at the accuracy field (acc). You can also see that state is always declared for a vote (cocur = 1).

The use case here is to determine if we can use the data to predict which state will vote for which candidate and with what accuracy we might make a prediction. This is too small a dataset to make accurate predictions, but given a much more representative dataset, I could, with reasonable confidence, predict that a CA or MO voter will pick candidate #1.

HTH
ron

Re: Search Commands: analyzefields

mw — Thu, 17 Feb 2011 08:31:49 GMT

Thanks Ron. Good stuff! I'm going to post some more of these, so please keep your eyes peeled and chime in if you can.

Re: Search Commands: analyzefields

mw — Thu, 17 Feb 2011 08:43:52 GMT

If you have any searches which utilize this command, please chime in and let us know what it's doing for you.