I save dashboards from both search and report, and it appears that the dashboards run the search every time it is brought up, but does not refresh after that. I have read that it is supposed to display the cached search. What is the proper way to create dashboards so that they update properly. Is there a way to do this from Splunk Web or only in XML?
Save your searches as reports (saved searches) and use the saved search name in your dashboards. The results of a saved search are kept in the dispatch directory for twice as long as your search timeframe, i.e. if you search over the past 4 hours, search results will be cached for 8 hours (by default) before being reaped.
Not sure what you meant with your second question in your comment.