Dashboards & Visualizations
Highlighted

How to display all events of a source if the source contains a particular field?

Path Finder

I am getting my events from an xml file. In that xml file, there is tag which gives the BuildID of a software. For instance: <BuildID>SoftwareBuildVer.2.0.6</BuildID>. I have been able to extract the SoftwareBuildVer.2.0.6 as a BuildID field in Splunk using regex. I want to be able to use BuildID + DateTime of the file creation to uniquely identify the source in Splunk. I have been able to create a final field, BuildAndDate, which has values that looks like this: SoftwareBuildVer.2.0.6 March 22, 2018 12:05PM. I plan on using the values in this field in a dashboard dropdown menu so that when a user selects a particular BuildAndDate value from the dropdown, the source which contains that particular BuildID is returned, and a new dashboard is generated based on the newly determined source value. Here is my simple xml for the dashboard (changes in dropdown to use BuildAndDate field has not been added yet):

<form>
  <label>XXXX_Dashboard</label>
  <fieldset submitButton="false" autoRun="true">
    <input type="dropdown" token="source_tok" searchWhenChanged="true">
      <label>Selected BuildID/DateTime</label>
      <fieldForLabel>source</fieldForLabel>
      <fieldForValue>source</fieldForValue>
      <search>
        <query>| tstats max(_time) AS mostRecent where host=XXXX sourcetype=XXXX groupby source, sourcetype, host| sort -mostRecent
| fields source 
| rex mode=sed field=source "s/(\\\)/\1\1\1/g"</query>
        <earliest>0</earliest>
        <latest></latest>
      </search>
      <prefix>source="</prefix>
      <suffix>"</suffix>
    </input>
  </fieldset>
  <row>
    <panel>
      <html depends="$alwaysHideCSSOverride$">
        <style>
          .select2-container .select2-choice {width: 520px;} 

          .select2-drop .select2-search {width: 528px; background-color: white;} 

          .select2-drop .select2-results {width: 528px;}

          .select2-drop-active {width: 530px !important;}

          .select2-container-active {width: 530px !important;}

          .splunk-status-indicator {
            border-radius: 15px !important;
            border: 2px solid white;
            font-size: 10px !important;
            line-height: normal !important;
            white-space: normal !important;
            <!--word-spacing: 100vw !important;-->
            overflow-x: hidden !important;
            display: flex;
            justify-content: center;
            align-items: center;
            word-break: break-word;
            padding: 3px;
          }

          div.facet-label {
            visibility: hidden !important;
          }

          div.viz-facet {
            <!--border: 2px solid red !important;-->
            padding:0 !important;
            height: 30% !important;
            width: 5% !important;
            margin: 0 15px 15px 0 !important;
          }

          div.facets-container {
            <!--background-color: lightgrey;-->
            background-image: url("/static/app/status_indicator_app/images/arrow.png");
            background-repeat: no-repeat;
            background-size: 100% 45%;
            margin-top: 55px;
          }
        </style>
      </html>
    </panel>
  </row>
  <row>
    <panel>
      <viz type="status_indicator_app.status_indicator">
        <search>
          <query>host=XXXX index=XXXX $source_tok$ | eval NewTime=strptime(StartTime,"%Y-%m-%dT%H:%M:%S.%3N") | eval _time=NewTime | eventstats max(_time) AS latestScan by Description  | where _time=latestScan | stats  count by StepResult, Description | eval  color=case(StepResult=="Passed","green",StepResult=="Skipped","gold", StepResult=="Failed","red")   | eval  StepResult = Description</query>
          <earliest>$earliest$</earliest>
          <latest>$latest$</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="height">275</option>
        <option name="refresh.display">progressbar</option>
        <option name="status_indicator_app.status_indicator.colorBy">field_value</option>
        <option name="status_indicator_app.status_indicator.fillTarget">background</option>
        <option name="status_indicator_app.status_indicator.fixIcon">check</option>
        <option name="status_indicator_app.status_indicator.icon">field_value</option>
        <option name="status_indicator_app.status_indicator.precision">0</option>
        <option name="status_indicator_app.status_indicator.showOption">1</option>
        <option name="status_indicator_app.status_indicator.staticColor">#555</option>
        <option name="status_indicator_app.status_indicator.useColors">true</option>
        <option name="status_indicator_app.status_indicator.useThousandSeparator">true</option>
        <option name="trellis.enabled">1</option>
        <option name="trellis.size">small</option>
        <option name="trellis.splitBy">Description</option>
      </viz>
    </panel>
  </row>
</form>

The dashboard currently works, but it directly makes use of the source in the dropdown. I want the BuildAndDate to be what shows up in the dropdown, and it should be used to determine the source which is then used to produce the new dashboard.

The issue I am having is mapping the BuildAndDate to its source, and the source then displaying all events in the source based on that. RIght now, clicking on a BuildAndDate value only returns one event, i.e. one line in the xml file ( <BuildID>SoftwareBuildVer.2.0.6</BuildID> ) instead of returning all the events in the xml in which <BuildID>SoftwareBuildVer.2.0.6</BuildID> is contained in.

This query produces the BuildAndDate field and value as expected:

host=XXXX sourcetype=XXXX | eval  mostRecent=strftime(_time, "%B %d, %Y %I:%M%p")
| eval BuildAndDate = BuildID+" "+mostRecent 
| stats values(BuildAndDate ) as BuildAndDate

Goal: Click on BuildAndDate value in dropdown and generate new dashboard via that selection.

Any help would be appreciated.

Thanks in advance.

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Champion

I'm not going to lie, I'm not sure I completely follow what you're trying to accomplish. But it sounds like you want to display to the user in the dropdown a BuildID/Date field to select. But you when they select one, you want to populate the token with source associated with just the BuildID.

If that's the case, can you just replace that last stats with this?

| stats count by BuildAndDate, source

Then use BuildAndDate for the "label" field of your dropdown, and source as the "value" field. So then users are shown the BuildAndDate, but the token is populated with the associated source?

Highlighted

Re: How to display all events of a source if the source contains a particular field?

Path Finder

Yes, you're right. That's basically what I am trying to accomplish.

I replaced the last stats as you said, and it returns the expected result in the Search app, however when I insert the same search query in the Search found in the dropdown, I get a message saying 'Search produced no results', so the dropdown is not being populated. Any idea why the search does not work in the dropdown?

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Champion

Hmm, not sure. Is the time scope for the dropdown search the same as what you used in the search app?

And i don't think it matters, but might be worth trying to drop the count field. Just add | fields - count after the stats.

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Path Finder

Thanks, but it still doesn't work. Quite strange.

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Champion

does any search work?

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Path Finder

Yes, this works:

host=XXXX sourcetype=XXX| eval  mostRecent=strftime(_time, "%B %d, %Y %I:%M%p")
| eval BuildAndDate  = BuildID+" "+mostRecent 
| stats count by source

but whenever I put in the BuildAndDate or BuildID field in the stats command, it doesn't work. For instance, this doesn't work:

host=WP3L2M6VNX1 sourcetype=SAMBAXMLTEST | eval  mostRecent=strftime(_time, "%B %d, %Y %I:%M%p")
| eval BuildAndDate  = BuildID+" "+mostRecent 
| stats count by source, BuildAndDate  

This also doesn't work:

host=WP3L2M6VNX1 sourcetype=SAMBAXMLTEST | eval  mostRecent=strftime(_time, "%B %d, %Y %I:%M%p")
    | eval BuildAndDate  = BuildID+" "+mostRecent 
    | stats count by source, BuildID

These two queries (which don't work) work fine in the Search & Reporting App, it only fails to work when I insert it in the Search criteria in the dropdown.

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Champion

are you building this dashboard in the search&reporting app as well? Is it possible that your BuildID field extraction isn't shared in the same app as the dashboard your building?

Highlighted

Re: How to display all events of a source if the source contains a particular field?

Path Finder

I am building the dashboard in a different app, but the BuildID is globally shared.

I discovered where the issue was coming from. It had to do with the _time field. When I changed mostRecent=strftime(_time, "%B %d, %Y %I:%M%p") to mostRecent=strftime(mostRecent, "%B %d, %Y %I:%M%p") it kind of works (It returns the values for the BuildID in the dropdown). So this works: | stats count by source, BuildID but this still doesn't work: | stats count by source, BuildAndDate.

Any thoughts?

0 Karma
Highlighted

Re: How to display all events of a source if the source contains a particular field?

Champion

well, you say - These two queries (which don't work) work fine in the Search & Reporting App, it only fails to work when I insert it in the Search criteria in the dropdown.

I am assuming you have a dashboard panel for this (or create a dashboard panel to test), now when you run these 2 queries from the dashboard, click on the magnifying glass (search) button for the panel, append your query (remove stats command) with something like |table BuildAndDate,BuildID,mostRecent.

This should show you how the dropdown actually is interpreted by Splunk, results should open up in a new window (along with the search as well). Invesitgate both the search and the table, do you see any issues in the way the tokens are passed and is being interpreted in the dashboard search and of course your table results?