I have a dashboard which counts the number of times a user performed an action. I have 3 time frames (last 24h, 7d, 30d) and thought I would try using three base searches since I do more transformation with each "set" of data.
For testing, I left my 3 original panels and wrote 3 new base searches and added new panels to use those. I noticed that the results from my original searches and the base searches don't match up. But if you click the magnifying glass, "Open in Search" then it matches the non-base search. The 24h search matches the non-base but the other two (7d & 30d) don't and I have no idea why.
My XML
<dashboard>
<label>base_testing</label>
<search id="base24h">
<query>
index=foo
| fields _time file user
</query>
<earliest>-24h@h</earliest>
<latest>now</latest>
</search>
<search id="base7d">
<query>
index=foo
| fields _time file user
</query>
<earliest>-7d@h</earliest>
<latest>now</latest>
</search>
<search id="base30d">
<query>
index=foo
| fields _time file user
</query>
<earliest>-30d@h</earliest>
<latest>now</latest>
</search>
<row>
<panel>
<title>24 hours</title>
<table>
<search>
<query>index=foo
| fields _time file user
| stats dc(file) AS "File Count" by user
| sort - "File Count"
| head 20</query>
<earliest>-24h@h</earliest>
<latest>now</latest>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
<panel>
<title>7 days</title>
<table>
<search>
<query>index=foo
| stats dc(file) AS "File Count" by user
| sort - "File Count"
| head 20</query>
<earliest>-7d@h</earliest>
<latest>now</latest>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
<panel>
<title>30 days</title>
<table>
<search>
<query>index=foo
| stats dc(file) AS "File Count" by user
| sort - "File Count"
| head 20</query>
<earliest>-30d@d</earliest>
<latest>now</latest>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
</row>
<!-- BASE ROW -->
<!-- BASE ROW -->
<!-- BASE ROW -->
<row>
<panel>
<title>BASE 24 hours</title>
<table>
<search base="base24h">
<query>| stats dc(file) AS "File Count" by user | sort - "File Count" | head 20</query>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
<panel>
<title>BASE 7 days</title>
<table>
<search base="base7d">
<query>| stats dc(file) AS "File Count" by user | sort - "File Count" | head 20</query>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
<panel>
<title>BASE 30 days</title>
<table>
<search base="base30d">
<query>| stats dc(file) AS "File Count" by user | sort - "File Count" | head 20</query>
</search>
<option name="count">5</option>
<option name="drilldown">none</option>
</table>
</panel>
</row>
</dashboard>
How many results are returned by the 7d and 30d base searches? If it is in excess of 500,000 then Splunk will silently ignore all but the first 500k. The documentation also recommends using a transforming search as your base search.
@robinsonalex88 - both the 7d and 30d non-base search panels are able to list the correct 20. I'm curious why the base search wouldn't be able to. Either way, 7d has 40k events, 30d has 100k. When I click (open in search) on the panel using the base search, it will run the search and its values will be correct (differing from what it displays on the dashboard panel).