Splunk Search

how to get data from multiple files(log files) and then display it together in a pie chart

poojadevadas
Explorer

I have multiple Deployment log files:
1. The first log file gives me all the logs related to the deployment in environment xxx.
2. The second log file gives me all the logs related to the deployment in environment yyy.
3. The third log file gives me all the logs related to the deployment in environment zzz.

I'm calculating the duration of deployment in each environment by finding the difference between the endTime and the startTime using
eval DurationSeconds = (endTime - startTime) . And using this I'm able to find the time duration taken in each environment.

Now I'm trying to collect data from all these 3 log files and then display all these data in the one pie chart so that we get to visualise the time taken for the deployment process in each environment in one single chart.
Could someone please help me out with this.

Tags (3)
0 Karma
1 Solution

renjith_nair
SplunkTrust
SplunkTrust

@poojadevadas,

Try

(source="xxx" OR source="yyy" OR source="zzz") |rex field=source "deploy(?<environment>\w+)"
|stats latest(startTime) as startTime,latest(endTime) as endTime by environment
|eval DurationSeconds = (endTime - startTime)
|stats values(DurationSeconds) by environment

Here is a runanywhere example

<dashboard>
  <row>
    <panel>
      <chart>
        <search>
          <query>|makeresults|eval env="xxx,yyy,zzz",duration="100,200,300"|makemv env delim=","|makemv duration delim=","
|eval x=mvzip(env,duration)|mvexpand x|eval x=split(x,",")|eval env=mvindex(x,0),duration=mvindex(x,1)|fields - x
|stats values(duration) by env</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="charting.chart">pie</option>
        <option name="charting.drilldown">all</option>
      </chart>
    </panel>
  </row>
</dashboard>
Happy Splunking!

View solution in original post

renjith_nair
SplunkTrust
SplunkTrust

@poojadevadas,

Try

(source="xxx" OR source="yyy" OR source="zzz") |rex field=source "deploy(?<environment>\w+)"
|stats latest(startTime) as startTime,latest(endTime) as endTime by environment
|eval DurationSeconds = (endTime - startTime)
|stats values(DurationSeconds) by environment

Here is a runanywhere example

<dashboard>
  <row>
    <panel>
      <chart>
        <search>
          <query>|makeresults|eval env="xxx,yyy,zzz",duration="100,200,300"|makemv env delim=","|makemv duration delim=","
|eval x=mvzip(env,duration)|mvexpand x|eval x=split(x,",")|eval env=mvindex(x,0),duration=mvindex(x,1)|fields - x
|stats values(duration) by env</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="charting.chart">pie</option>
        <option name="charting.drilldown">all</option>
      </chart>
    </panel>
  </row>
</dashboard>
Happy Splunking!

poojadevadas
Explorer

Hi @renjith.nair,

This is very close to what I'm looking for . But I have one doubt as I'm very new to Splunk. My environment name is present in the log file name. For example:
environment xxx -> deployxxx
environment yyy -> deployyyy

Is there a way I can extract the environment name from file name and then use in the query mentioned by you(above).

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@poojadevadas,
No problem. If you have the string in your "source" filename, then try this to extract the environment.

rex field=source "deploy(?<environment>\w+)"

Updated the answer with the change. If its not working, please provide a sample filename

Happy Splunking!
0 Karma

poojadevadas
Explorer

Hi @renjith.nair ,
I tried executing the query mentioned by you. I'm able to list down the environment names but the Time is shown as blank I.e.

Environment. values(DurationSeconds)
xxx
yyy
zzz

0 Karma

poojadevadas
Explorer

I used this query for only one environment(log file deployxxx.log) but was unable to find the duration between the first and last event in environment xxx.

So used this:
(source="deploy906.log") |rex field=source "deploy(?\w+)"
|stats earliest(_time) as startTime latest(_time) as endTime by environment
|eval DurationSeconds = (endTime - startTime)
|stats values(DurationSeconds) by environment

Using this I was able to display the environment name as well as the duration for environment xxx. But when I added source="deployyyy" or source="deployzzz" in the same query, I was unable to find the duration. It jus displays the environment names but duration is left blank. Could you please help me with this.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@poojadevadas, do you have these sources in your splunk environment? I have added the source just based on the assumption that you have these sources as your log file sources.
Just verify what are the source you are getting for this files xxx,yyy,zzz . Also are these from same index? if not you need to add the index as well. Would it be possible to add sample logs for each of this environment (mask any sensitive data ), so that we can have a better idea.
thanks!

Happy Splunking!
0 Karma

poojadevadas
Explorer

I loaded the files again with same index and this query worked fine. Thanks @renjith.nair.

0 Karma

poojadevadas
Explorer

I'm displaying the result as a pie chart. On click of a pie slice, I want another pie chart(based on each environment name) to be opened up and display some data specific to that pie slice(that specific environment). I checked in Splunk docs and found that I can use drilldown for this but unable to understand what field to mention.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@poojadevadas,In the pie chart options , add this

        <drilldown>
          <set token="env">$click.value$</set>
        </drilldown>
Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...