Assuming that your drilldown takes you to a different dashboard, you could change the permissions for the drilldown dashboard to only grant the admin role access to read and write, and have no other role have any access to it.
If you are using a saved search for the drilldown you can similarly change the permissions for the saved search to allow the admin role read and write and whatever role the "User" has to have neither read nor write to that particular saved search.
If you are using neither a dashboard nor a saved search for the drilldown, then I don't believe you can limit the access using roles.
