When using the MAP and TAIL commands in my search, why do events not match the statistics table?

[motobeats]

When I run the MAP search below, the events that I get back do not match the ones used to generate the statistics table. However, if I switch the TAIL to HEAD, that events and statistics table match.

 host=<my_host> "monitor status"
 |rex field=_raw "Pool\s\/(?<Pool>.*)\smember"
 |rex field=_raw "member\s\/(?<Member>.*)\smonitor"
 |rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s"
 |rex field=_raw "for\s(?<DownTime>.*)\s\]"
 |dedup Member
 |table Member
 |map [
  search host=<my_host> "monitor status"
  |rex field=_raw "member\s\/(?<Member>.*)\smonitor"
  |rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s"
  |rex field=_raw "for\s(?<DownTime>.*)\s\]"
  |search Member=$Member$
  |tail 1
  ]
 |table _time,Member,Status,DownTime

When using TAIL, events are the most recent events but statistics table uses the oldest events from the time period
When using HEAD, both events and statistics table use most recent events. So only the statistics table changes.

Why does this happen?

[jkat54]

As for why tail works versus head... you've probably got one "blank" / "bad" event that comes to the top when you use head vs tail because one takes the 1st event and one takes the last event.

Try just using the search with head vs tail to see the results difffer:

   host=<my_host> "monitor status"
   |rex field=_raw "member\s\/(?<Member>.*)\smonitor"
   |dedup Member
   |rex field=_raw "Pool\s\/(?<Pool>.*)\smember"
   |rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s"
   |rex field=_raw "for\s(?<DownTime>.*)\s\]"
   |table Member Status DownTime
   |head 1



   host=<my_host> "monitor status"
   |rex field=_raw "member\s\/(?<Member>.*)\smonitor"
   |dedup Member
   |rex field=_raw "Pool\s\/(?<Pool>.*)\smember"
   |rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s"
   |rex field=_raw "for\s(?<DownTime>.*)\s\]"
   |table Member Status DownTime
   |tail 1

[jkat54]

This search makes no sense. The map search is the same as your original search minus the member name extraction that's occurred in the original search but plus a search for member=member from first search.

The map command is used to take something from the first search, and use it in the second search. So generally you would be searching a different index in the map command... or at least a different search string all together.

This is your search simplified & optimized:

  host=<my_host> "monitor status"
  |rex field=_raw "member\s\/(?<Member>.*)\smonitor"
  |dedup Member
  |rex field=_raw "Pool\s\/(?<Pool>.*)\smember"
  |rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s"
  |rex field=_raw "for\s(?<DownTime>.*)\s\]"
  |table Member Status DownTime

[motobeats]

Yes, this is a much better search. Overlooked that dedup could do the head/tail filtering I was looking for. I also combined all of my rex statements into one.

[jplumsdaine22 · Accepted solution]

The events you are seeing (I assume you have verbose mode turned on) are those returned from the first part of your search. You are using dedup, which by default returns the most recent event. That is why the events are always the most recent.

By contrast tail returns the last event in your series, which as you don't have any other ordering, will be the oldest event. Head in turn will give you the most recent. Note head and tail return N results from the beginning and end of the result set - if you ordered your results by something else eg if you had |sort -downtime | head 1 your statistics table would display the event with the highest downtime.

I am curious what you're trying to achieve with the search - you're only feeding a single result to map and then running an identical search. The following search should be logically equivalent to what you have posted

host=<my_host> "monitor status" 
|rex field=_raw "member\s\/(?<Member>.*)\smonitor"
|dedup Member
|rex field=_raw "monitor\sstatus\s(?<Status>.*)\.\s
|rex field=_raw "for\s(?<DownTime>.*)\s\]" 
|table _time,Member,Status,DownTime
|head 1

Although swap head with tail depending on which result you want.

See: http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/Map
See: http://docs.splunk.com/Documentation/Splunk/6.3.1511/Search/Aboutsubsearches

[motobeats]

Yes, this search is better. Realize now that dedup does everything I need so map is unnecessary.

And thanks for the info on head v. tail. Old habit to assume tail=most recent where in Splunk tail is the bottom event which is usually the oldest one, assuming no sorting.
Thanks again.