Grouping hosts based on their environment and create dashboard based on that.

[amer2885]

Hi,

I have to group set of servers based on the default search and present it in the dashboard. Based on the host field and different sourcetype.

Example: say we have below server list

Prod contains  server1, server2, server3.

QA  contains server4, server5, server6.

DEV  contains server7, server8, server9.

sourcetype: access_logs, catalina_logs.

Need to group these servers based on host and sourcetype.

[bowesmana]

Is there any way to determine the group based on the server name or something else. If the host name contains the environment in some form, e.g. 

WBPRDSRV01

WBDEVSRV01

and so on, then you could derive group from the host. In the absence of that, you could create a lookup table with server name and group, but that would need to be maintained

Are you looking to be able to select the group in the dashboard and then query according to that group or show all groups on the same dashboard. If you just want to select a group, then you just an a dropdown input, which could be populated from the groups on your lookup.

A little more detail would help.

[amer2885]

@bowesmana 

Lets say we have 2 groups in a team using different tools. Those tools need to be monitor on the same dashboard.

tool1 has dev, qa , prod environment

tool2 has dev, qa , prod environment

and has respective set of servers in them. Which does not contain env in host name. 
Both the tools have 2 sourcetype say access_log, catalina_log

This will be on event based scenario. On selection of tool1 from 1st drop-down list , env(dev,qa,prod) from 2nd dropdown list, sourcetype from 3rd dropdown list and select submit to show the desired result.

[bowesmana]

Can you provide a sample of the data so I can see how you expect to find the data rows for each tool/environment?

Dashboards can use cascading tokens in a search that populates the contents of the dropdown, for example 

<form>
  <label>Test</label>
  <fieldset submitButton="false">
    <input type="dropdown" token="tool" searchWhenChanged="true">
      <label>Tool</label>
      <choice value="tool1">Tool 1</choice>
      <choice value="tool2">Tool 2</choice>
    </input>
    <input type="dropdown" token="env" searchWhenChanged="true">
      <label>Environment</label>
      <fieldForLabel>envs</fieldForLabel>
      <fieldForValue>envs</fieldForValue>
      <search>
        <query>
| makeresults
| eval envs=case($tool|s$="tool1","Dev:QA:Prod",$tool|s$="tool2","Dev:QA:Prod")
| makemv delim=":" envs
| mvexpand envs</query>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </search>
    </input>
  </fieldset>
</form>

 shows how in the second dropdown it uses the value of the selected tool token to build a list of environments (in this case they are identical - but if your data supports it you can get this from the data or take it from a lookup as other options.

[amer2885]

@bowesmana 

Appreciate Your help.

There are 2 application say abc, xyz which runs on all env's  i.e dev, qa, prod.
It has respective servers list.  approx 50 servers in total for both application.
and have 2 sourcetype defined one is access_logs and other catalina_logs

I need 3 dropdown list 
example: 
Application(abc,xyz,ALL(option for 2apps))     Environment(dev,qa,prod,ALL(option for all 3 envs))   sourcetype(access_logs,catalina_logs,ALL(option for 2sourcetype))     submit button

when I select abc application from 1st dropdown list  then select dev env from 2nd dropdown list and access_logs from 3rd dropdown list  and click submit 
 
then I need to get the results for only those servers from dev env of application abc which contains sourcetype access_logs and show in the panel. 

and if we have the option of ALL in the 3 dropdown list and select it and submit , it should show all the 50 servers with 2 sourcetype in the panel.

and
lets say Application abc has 30 servers
10 for prod
10 for qa 
10 for dev

can we group the 10 servers with their hostnames ? all the servers have different hostname nothing mentioned in the hostname as these are prod to identify.

[bowesmana]

Here's a basic dashboard with static values for what you want and a basic search panel, but you have not explained how you can correlate the app and the environment against your data.

So, when the example search is searching for data, it is searching app and env fields, which may or may not exist in your data.

<form>
  <label>Test</label>
  <fieldset submitButton="true" autoRun="false">
    <input type="dropdown" token="app" searchWhenChanged="true">
      <label>Application</label>
      <choice value="*">All</choice>
      <choice value="abc">ABC</choice>
      <choice value="xyz">XYZ</choice>
    </input>
    <input type="dropdown" token="env" searchWhenChanged="true">
      <label>Environment</label>
      <fieldForLabel>envs</fieldForLabel>
      <fieldForValue>envs</fieldForValue>
      <choice value="*">All</choice>
      <choice value="prod">Prod</choice>
      <choice value="qa">QA</choice>
      <choice value="dev">Dev</choice>
    </input>
    <input type="dropdown" token="sourcetype">
      <label>Sourcetype</label>
      <choice value="*">All</choice>
      <choice value="catalina_logs">Catalina Logs</choice>
      <choice value="access_logs">Access logs</choice>
    </input>
  </fieldset>
  <row>
    <panel>
      <event>
        <search>
          <query>index=* app=$app|s$ env=$env|s$ sourcetype=$sourcetype|s$
| stats values(host) by sourcetype
          </query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="list.drilldown">none</option>
      </event>
    </panel>
  </row>
</form>

[amer2885]

@bowesmana 

First, Thank you for addressing this issue.

I have the basic setup.

I am having trouble grouping the servers based on the env.
Example:
If i have set of servers say
server1
server2
server3 
which are used for dev for one application abc

how can i group, this under one tag
i.e abc-dev-servers 
so that i can call this during the query or search.
and based on this, on selection it should show only logs of these servers(abc-dev-servers).

Thats what I'm looking for, when I select app abc and select dev env and access_logs then it should show only the logs of 3servers (under tag abc-dev-servers)

First I want to group the servers based on their application and environment, and use it in querying so that the result comes for only those servers.   

[bowesmana]

Please provide a data sample indicating how - from the data - you would be able to tell which logs relate to which context.

If you have nothing in the data that says what application or environment those logs come from, then you cannot make a search that will perform that.

If your only link from data to environment is by host, then you will need to maintain a lookup of hosts that represent each environment and you can then use that lookup to construct the hosts that will be used in the search.

However, that still leaves you with how to determine the application for a particular a log entry.

Data samples with explanation of how this is represented is needed.

Thanks