SimpleXML: Inherent brokenness regarding autorun and Submit button?

[woodcock]

I know that I am the king of "clearly explain what you really need" in Splunk Answers but I am just going to have to be a hypocrite this time. This same kind of thing happens to me over and over and over and I never can get to the bottom of it. I think the problem is that I need some things to happen automatically but searches to happen mainly/only when the Submit button is pushed and I frequently get into a situation where I just can't make anything work right. For example, below you will find a run anywhere example which I would like to work like this (but it doesn't):
ForSource of domian(s)?:
When Ad-hoc (free-form text input) is checkmarked, the left panel is used and the Domain string text control is unhidden/enabled but when it is unchecked, the left panel and the Domain string text control should be hidden.
When Command and Control hotlist is checkmarked, the right panel should be shown/used but when it is unchecked, the right panel should be hidden. I would like all of that to happen as soon as the boxes are clicked, but I would like the searches to run only when Submit is clicked. The Wildcard the domain (suffix)? changes some of the SPL in the left panel. Here is what I have that DOES NOT WORK mostly because the Domain string control seems to disappear for no reason:

<form theme="dark">
  <init>
    <unset token="which_urls_ad_hoc_eval_token"></unset>
    <unset token="which_urls_c_and_c_eval_token"></unset>
    <unset token="domain_for_term_set_token"></unset>
    <unset token="domain_for_match_set_token"></unset>
    <!--set token="domain_for_term_set_token">$domain_text_token$</set>
    <set token="domain_for_match_set_token">$domain_text_token$</set-->
  </init>
  <label>URL DNS Search (RFC1918 sources only)</label>
  <search>
    <query>
        | makeresults
        | fields - _time
        | eval data="$which_urls_checkbox_token$"
        | eval which_urls_ad_hoc                = if(match(data, "which_urls_ad_hoc"),       "YES", "NO")
        | eval which_urls_c_and_c               = if(match(data, "which_urls_c_and_c"),      "YES", "NO")
      </query>
    <done>
      <condition match="$job.resultCount$!=0">
        <eval token="which_urls_ad_hoc_eval_token">nullif($result.which_urls_ad_hoc$,  "NO")</eval>
        <eval token="which_urls_c_and_c_eval_token">nullif($result.which_urls_c_and_c$, "NO")</eval>
      </condition>
      <condition>
        <unset token="which_urls_ad_hoc_eval_token"></unset>
        <unset token="which_urls_c_and_c_eval_token"></unset>
      </condition>
    </done>
  </search>
  <description>Author: Gregg Woodcock, woodcock@Splunxter.com</description>
  <fieldset submitButton="true" autoRun="false">
    <input type="checkbox" token="which_urls_checkbox_token" searchWhenChanged="true">
      <label>Source of domian(s)?</label>
      <choice value="which_urls_ad_hoc">Ad-hoc (free-form text input)</choice>
      <choice value="which_urls_c_and_c">Command and Control hotlist</choice>
      <initialValue>which_urls_ad_hoc</initialValue>
      <default>which_urls_ad_hoc</default>
    </input>
    <input type="time" token="time_token" searchWhenChanged="false">
      <label>Time picker</label>
      <default>
        <earliest>-24h</earliest>
        <latest>+10m</latest>
      </default>
    </input>
    <input depends="$which_urls_ad_hoc_eval_token$" type="text" token="domain_text_token" searchWhenChanged="true">
      <label>Domain String</label>
    </input>
    <input type="checkbox" token="domain_wildcard_checkbox_token"  searchWhenChanged="true">
      <label>Wildcard the domain (suffix)?</label>
      <choice value="domain_suffix_wildcard">Yes</choice>
      <change>
        <condition match="$domain_wildcard_checkbox_token$==&quot;Yes&quot;">
          <set token="domain_for_term_set_token">$domain_text_token$*</set>
          <set token="domain_for_match_set_token">$domain_text_token$.*</set>
        </condition>
        <condition>
          <set token="domain_for_term_set_token">$domain_text_token$</set>
          <set token="domain_for_match_set_token">$domain_text_token$</set>
        </condition>
      </change>
    </input>
  </fieldset>
  <row>
    <panel depends="$which_urls_ad_hoc_eval_token$">
      <title>DNS search for domain='$domain_for_term_set_token$'</title>
      <table>
        <search>
          <query>index=win_dns sourcetype=win_dns TERM(*$domain_for_term_set_token$)
| where NOT match(Client_IP, "^(?!10\.|192\.168\.|172\.(?:1[6-9]|2[0-9]|3[01])\.)(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}$")
| rex "(?&lt;url&gt;\S+)$$"
| where match(url, "(?&lt;=\.|^)$domain_for_match_set_token$$$")
| eval domain="$domain_for_term_set_token$"
| stats count min(_time) AS earliestTime max(_time) AS latestTime BY url domain Client_IP
| sort 0 - count
| stats list(*) AS * sum(count) AS count_total BY domain Client_IP
| lookup dnslookup clientip AS Client_IP OUTPUT clienthost AS Client_Host
| fieldformat earliestTime = strftime(earliestTime, "%m/%d/%y %H:%M:%S")
| fieldformat latestTime = strftime(latestTime, "%m/%d/%y %H:%M:%S")
| sort 0 - count_total
| table Client* domain count_total count *Time url *</query>
          <earliest>$time_token.earliest$</earliest>
          <latest>$time_token.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">20</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">none</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
    <panel depends="$which_urls_c_and_c_eval_token$">
      <title>DNS search for domains in Command and Control Hot List</title>
      <table>
        <search>
          <query>|makeresults</query>
          <earliest>$time_token.earliest$</earliest>
          <latest>$time_token.latest$</latest>
        </search>
        <option name="count">20</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">none</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>

I am looking at you, @niketnilay!

[woodcock]

The simpleXML submit button is horribly broken and for complex usecases, cannot be made to work.  Try this dahsboard's approach instead:

<form>
  <label>Proof-of-Concept (PoC) demonstrating a fully-functioning "SUBMIT" button (actually checkbox).</label>
  <description>Notice that some of the controls interact with one-another but the actual search does not run until the SUBMIT box is checked.  For very intricate technical reasons, the "real" "SUBMIT" button is fundamentally broken and CANNOT BE MADE TO WORK (so please do not even try).  Fortunately, this alternative method does everything that anyone could desire!  WARNING!  Although bookmarks will correctly set all pickers, there is no way to make the dashboard autorun on first load.  This is the only downside.  Here are the pieces that are required. 1: DO NOT CHANGE ANYTHING ABOUT THE "SUBMIT" checkbox other than the description text which currently reads, "Check the box below to execute your search".  2: Make sure that EVERY OTHER PICKER has searchWhenChanged="true".  3: Make sure that EVERY OTHER PICKER has a "&lt;change&gt;...&lt;/change&gt;" section that unsets BOTH these tokens: {"SUBMIT_CHECKBOX", "form.SUBMIT_CHECKBOX"}.  4: Ensure that 1 search in every chain of searches uses the do-nothing token "$SUBMIT_CHECKBOX$" somewhere (e.g. if all searches are driven by a base search, this only has to be done in the base search, not the post-process searches).  5: "Autorun dashboard" does not matter but the safest thing to do is to turn this on.  For support contact woodcock@splunxter.com</description>
  <fieldset submitButton="false" autoRun="true">
    <input type="multiselect" token="INDEX" searchWhenChanged="true">
      <label>Select Index value(s)</label>
      <fieldForLabel>label</fieldForLabel>
      <fieldForValue>value</fieldForValue>
      <default>"*"</default>
      <search>
        <query>|tstats count dc(host) WHERE index=* AND $SOURCETYPE$ BY index
| rename index AS value
| eval label=value
| append [|makeresults | eval label = "All", value="*"]
| table label value</query>
        <earliest>$timepicker.earliest$</earliest>
        <latest>$timepicker.latest$</latest>
      </search>
      <delimiter> OR index=</delimiter>
      <valuePrefix>"</valuePrefix>
      <valueSuffix>"</valueSuffix>
      <prefix>(index=</prefix>
      <change>
        <eval token="form.INDEX">case(mvcount($form.INDEX$) == 2 AND mvindex($form.INDEX$, 0) == "*", mvindex($form.INDEX$, 1), mvfind($form.INDEX$, "^[*]") == mvcount($form.INDEX$) - 1, "*", true(), $form.INDEX$)</eval>
        <unset token="SUBMIT_CHECKBOX"></unset>
        <unset token="form.SUBMIT_CHECKBOX"></unset>
      </change>
      <suffix>)</suffix>
    </input>
    <input type="multiselect" token="SOURCETYPE" searchWhenChanged="true">
      <label>Select sourcetype value(s)</label>
      <fieldForLabel>label</fieldForLabel>
      <fieldForValue>value</fieldForValue>
      <default>*</default>
      <search>
        <query>|tstats count dc(host) WHERE $INDEX$ AND sourcetype=* BY sourcetype
| rename sourcetype AS value
| eval label = value
| append [|makeresults | eval label = "All", value="*"]
| table label value</query>
        <earliest>$timepicker.earliest$</earliest>
        <latest>$timepicker.latest$</latest>
      </search>
      <delimiter> OR sourcetype=</delimiter>
      <valuePrefix>"</valuePrefix>
      <valueSuffix>"</valueSuffix>
      <prefix>(sourcetype=</prefix>
      <change>
        <eval token="form.SOURCETYPE">case(mvcount($form.SOURCETYPE$) == 2 AND mvindex($form.SOURCETYPE$, 0) == "*", mvindex($form.SOURCETYPE$, 1), mvfind($form.SOURCETYPE$, "^[*]") == mvcount($form.SOURCETYPE$) - 1, "*", true(), $form.SOURCETYPE$)</eval>
        <unset token="SUBMIT_CHECKBOX"></unset>
        <unset token="form.SUBMIT_CHECKBOX"></unset>
      </change>
      <suffix>)</suffix>
    </input>
    <input type="text" token="COUNT">
      <label>Name for "count" field</label>
      <default>count</default>
      <change>
        <unset token="SUBMIT_CHECKBOX"></unset>
        <unset token="form.SUBMIT_CHECKBOX"></unset>
      </change>
    </input>
    <input type="time" token="timepicker" searchWhenChanged="true">
      <label>Timepicker</label>
      <default>
        <earliest>$timepicker.earliest$</earliest>
        <latest>$timepicker.latest$</latest>
      </default>
      <change>
        <unset token="SUBMIT_CHECKBOX"></unset>
        <unset token="form.SUBMIT_CHECKBOX"></unset>
      </change>
    </input>
    <input type="checkbox" token="SUBMIT_CHECKBOX">
      <label>Check the box below to execute your search</label>
      <choice value="">SUBMIT</choice>
    </input>
  </fieldset>
  <row>
    <panel>
      <title></title>
      <table>
        <title>Searches ONLY on checking the SUBMIT box (which resets itself)</title>
        <search>
          <query>|tstats count AS $COUNT|s$ dc(host) WHERE $INDEX$ AND $SOURCETYPE$ BY index sourcetype $SUBMIT_CHECKBOX$</query>
          <earliest>$timepicker.earliest$</earliest>
          <latest>$timepicker.latest$</latest>
        </search>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</form>

[niketn · Accepted solution]

@woodcock yes this is slightly tricky situation because of the behavior of Default Token Model and Submitted Token Model.

When the Search When Changed option for form inputs is unchecked, the token value updates in default token model but not in submitted token model. Now default token model can be used to directly display value in the dashboard like panel title or html panels etc. But if the same token is used in search it will reflect value only when Submit Token Model is updated (in other words only when the Submit button is clicked... other was would be to handle this in Splunk JS and update Submit Token Model under required condition). I had the same confusion which was clarified by @rjthibod : https://answers.splunk.com/answers/679596/what-is-the-expected-behavior-for-submit-button-wh.html

Now coming back to your question, the reason why it is tricky is because multiple check box options do not work with built in change event handler, because of which you have used independent search to match selected values in the checkbox and accordingly set the required token. https://answers.splunk.com/answers/681330/can-i-hideunhide-specific-text-boxes-using-a-singl.html

So, if Search When Changed is disabled for the checkbox with multiple options, then the independent search which sets the required tokens will not execute until Submit Button is clicked. So that is one change to Enable Search When Changed for which_urls_checkbox_token .

Second change is with Single Option check box domain_wildcard_checkbox_token; you don't need independent search to resolve the token since <change> event handler for dropdown works with single value checkbox. If you do this change, the required Text Box will show up immediately on selection of the checkbox and will hide otherwise.

Please try with these two changes and see. If not, I will try later this week 🙂

[woodcock]

OK, I made the changes and updated the question. Now the problem is that the Wildcards the domain (suffix)? checkbox only works when it is checked (not when it is unchecked).

[woodcock]

OK, I figured it out. On top of your changes, I changed this:

<input type="checkbox" token="domain_wildcard_checkbox_token" searchWhenChanged="true">
  <label>Wildcard the domain (suffix)?</label>
  <choice value="Yes">Yes</choice>
  <change>
    <condition value="Yes">

Now it works the way that I need it to!