Splunk Search

Searching across multiple host combinations

larrydavid
New Member

Our team looks after 7 applications, we have 5 environments and each application sits on between 2 and 4 servers, depending on the environment.

Each app instance has its own dedicated server, so in other words, given a hostname, you can figure out exactly which application and which environment it is for.

 

At the moment, if we want to search for the logs of one of the applications (app1) in UAT, and if this app has 4 servers in UAT, the only way we can do this is by using the following search parameters

 

source=*app1.log host=host1 OR host=host2 OR host=host3 OR host=host4

 

Sometimes we have a few different applications talking to each other, so we end up having to mention a long list of host names and this gets quite tedious. 

We have a separate team that manages Splunk across the organisation.

 

Is there something we could be asking the Splunk team to do for us to make our searching easier?

Is there something they could do that would result in us being able to do something like

application=app1 environment=uat 

 

instead of having to specify host names for the environment that we are interested in?

 

Our team would appreciate any suggestions that can make our work easier.

 

Thank you

 

 

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @larrydavid ,

the easiest approach is to create a lookup (eventually an automatic one!) containing the combinations of apps and hosts to define the environments, so you can use the lookup in your searches, something like this:

environment app  host
env1        app1 host1
env1        app1 host2
env1        app1 host3
env2        app2 host4
env2        app2 host5
env2        app2 host6
env3        app3 host7
env3        app3 host8
env3        app3 host9

One additiona question: if each application uses some servers and there's a relation 1:n between apps and hosts, why you don't use only apps to define your environment?

then, remember the there's the IN() operator to use instead of OR:

source=*app1.log host IN (host1,host2,host3,host4)

it's smaller!

Ciao.

Giuseppe 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

You need to clarify your constraints.  The most obvious solution is to send a field "environment" along with log events.  There are a million ways to do this.

Then, if the deployment team is sympathetic to your course, they can name hosts according to environment in some way.  There is at least a dozen ways to do this. (One obvious way is to dedicate a special domain to environment.)  So, that's at least 1,000,012.

You can also do an automatic lookup on hostname.  That's at least 1,000,013 ways.

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...