Splunk Search

Is there a better way I can search for an index using a pulldown?

rhayle
Path Finder

Is there a better way I can search for an index using a pulldown?
Thanks

e.g.

  <module name="URLLoader" layoutPanel="viewHeader" autoRun="True">
<param name="keepURLUpdated">True</param>
<module name="Search">
  <param name="search">| metadata type="hosts"</param>
  <module name="Pulldown">
    <param name="name">host</param>
    <param name="label">Host</param>
    <param name="template">$name$="$value$"</param>
    <param name="float">left</param>
    <param name="searchFieldsToDisplay">
      <list>
        <param name="label">host</param>
        <param name="value">host</param>
      </list>
    </param>
    <module name="Search">
      <param name="earliest">-2d</param>
      <param name="latest">now</param>
      <param name="search">$host$ index=* | dedup index | sort index</param>
      <module name="Pulldown">
        <param name="name">index</param>
        <param name="label">Tenant</param>
        <param name="template">$name$="$value$"</param>
        <param name="float">left</param>
        <param name="searchFieldsToDisplay">
          <list>
            <param name="label">index</param>
            <param name="value">index</param>
          </list>
        </param>
Tags (3)
1 Solution

sideview
SplunkTrust
SplunkTrust

Ordinarily, what you need here - to filter the list of indexes based on the value in the host pulldown, cannot be done without running a search to get lots of events off disk. (You can of course set up a summary index but the scheduled search that feeds that summary index would have to run searches to get lots of events off disk. )

To give some background - In 5.0 and earlier you have the metadata command which gives you back either hosts, sourcetypes or sources and which takes an index term you can use for filtering. Or you have the | eventcount summarize=false command which gives you back indexes and which takes an index term for filtering.

But no combination of the above could give you the list of indexes, given a host term to filter. You had to slog through the events yourself.

In 6.0, it appears (somewhat shockingly), that to some extent this has been fixed. I think there's a good chance the fix is inadvertent, and it may only apply to single-indexer deployments, I'm not sure.

But all index-time fields automatically get a little bit of representation in tsidxstats (aka "high performance analytics store"). Although it's only enough to group by and get counts, that's all we need here. host/sourcetype and source also get the same treatment and we can group and filter by any combination of those fields, in a tstats search.

In short, run this search:

| tstats prestats=t count WHERE host=foo index=* GROUPBY index | stats count by index | where count>0

and you will get the list of indexes that have data indexed with the host "foo". Since it's a tstats search and the distinct counts of host and index are probably not insanely high, the search should run extremely fast.

Again this only works on 6.0.

UPDATE: This appears to work with Splunk's distributed search.

View solution in original post

sideview
SplunkTrust
SplunkTrust

Ordinarily, what you need here - to filter the list of indexes based on the value in the host pulldown, cannot be done without running a search to get lots of events off disk. (You can of course set up a summary index but the scheduled search that feeds that summary index would have to run searches to get lots of events off disk. )

To give some background - In 5.0 and earlier you have the metadata command which gives you back either hosts, sourcetypes or sources and which takes an index term you can use for filtering. Or you have the | eventcount summarize=false command which gives you back indexes and which takes an index term for filtering.

But no combination of the above could give you the list of indexes, given a host term to filter. You had to slog through the events yourself.

In 6.0, it appears (somewhat shockingly), that to some extent this has been fixed. I think there's a good chance the fix is inadvertent, and it may only apply to single-indexer deployments, I'm not sure.

But all index-time fields automatically get a little bit of representation in tsidxstats (aka "high performance analytics store"). Although it's only enough to group by and get counts, that's all we need here. host/sourcetype and source also get the same treatment and we can group and filter by any combination of those fields, in a tstats search.

In short, run this search:

| tstats prestats=t count WHERE host=foo index=* GROUPBY index | stats count by index | where count>0

and you will get the list of indexes that have data indexed with the host "foo". Since it's a tstats search and the distinct counts of host and index are probably not insanely high, the search should run extremely fast.

Again this only works on 6.0.

UPDATE: This appears to work with Splunk's distributed search.

sideview
SplunkTrust
SplunkTrust

Looks like these guys have answered your question well. As a side note, if you're on Sideview Utils 2.X you can replace this block:

<param name="searchFieldsToDisplay">
  <list>
    <param name="label">index</param>
    <param name="value">index</param>
  </list>
</param>

with

<param name="valueField">index</param>
0 Karma

sowings
Splunk Employee
Splunk Employee

| rest /services/data/indexes count=0 | fields title | dedup title | rename title AS index

The rename is optional, it's just to make it clear to other consumers of the search that you're referring to an index. The REST API calls a lot of varied things "title".

Requires version 4.3 or higher of Splunk.

0 Karma

rhayle
Path Finder

I need the index pulldown to filter based on the host pulldown?

0 Karma

somesoni2
Revered Legend

You can use following for the Index dropdown search

| eventcount summarize=false index=* index=_* server=$host$

Remove "index=_*" if you dont want to list internal indexes.

rhayle
Path Finder

| eventcount summarize=false index=* server=$host$

The host filter part does not work with this?

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>