topic Re: Can I create a visual of the status of a file using Splunk visualization tool? in Dashboards & Visualizations

Can I create a visual of the status of a file using Splunk visualization tool?

chandana204 — Fri, 22 Dec 2017 17:49:59 GMT

Hi,

I have been working on data, this data has tracking information. I want to see all information with respect to file name in a single line with time stamp. In simple terms, if we check fedex order status it display the package status, where it is gone where it is now, what is the current status of package. I want to visualize my data in same as fedex tracking. Can i do using splunk visualization tool? If so, Please suggest me. My data doesn't have geo location information but it has some process keywords like process, send, transfer. I wanted to work on based on these keywords.

ex:
12/22/2017 processed
12/23/2017 send
12/23/2017 transferred

sol: processed----->send---------->transferred
12/22/2017 12/23/2017 12/23/2017

NOTE: I am new learner

Thanks,
Chandana

Re: Can I create a visual of the status of a file using Splunk visualization tool?

niketn — Fri, 22 Dec 2017 18:12:41 GMT

@chandana204, if you are on Splunk 6.6 or higher you can try Status Indicator with Trellis Layout to show a File Tracking status.

https://wiki.splunk.com/User_talk:Niketnilay#Topic_15:_Following_is_an_example_of_Orderflow_Status_Tracker_using_Status_Indicator_Custom_visualization_with_Trellis_Layout_to_split_Status_Indicators_by_status.

Re: Can I create a visual of the status of a file using Splunk visualization tool?

chandana204 — Fri, 22 Dec 2017 18:52:28 GMT

This info is good. I already worked with status indicator. i want to represent process in color mode not time stamp.

i have used below query. but it's coloring time stamp not process.

| stats values(process) by _time

Thanks,

Re: Can I create a visual of the status of a file using Splunk visualization tool?

niketn — Fri, 22 Dec 2017 21:31:32 GMT

@chandana204, can you add sample raw data for a file with various status? Also I am assuming above stats is for specific file. Please confirm. What is the fieldname for file?

Re: Can I create a visual of the status of a file using Splunk visualization tool?

niketn — Fri, 22 Dec 2017 22:37:24 GMT

@chandana204, please find below the run anywhere dashbaord built in Simple XML. Logic is similar to Wiki Talk above. You can also check out the Status Indicator Custom Visualization tutorial documents.

Following is the complete run anywhere Simple XML Dashboard code:

<dashboard>
  <label>Status Indicator Trellis for File Transfer</label>
  <row>
    <panel>
      <html depends="$alwaysHideCSS$">
        <style>
          .viz-panel>facet-label{
              top: 40px !important;
              color: white !important;
              font-size: 150% !important;
          }          
        </style>
      </html>
      <title>Panel 1 - processed --> send --> transferred</title>
      <viz type="status_indicator_app.status_indicator">
        <search>
          <query>| makeresults 
| eval data="filename=file1,process=processed,date=12/22/2017;filename=file1,process=send,date=12/23/2017;filename=file1,process=transferred,date=12/23/2017;filename=file1" 
| makemv data delim=";" 
| mvexpand data 
| rename data as _raw 
| KV 
| search filename="file1"
| stats values(date) as date by process 
| appendpipe
    [| makeresults 
    | eval process="processed,send,transferred" 
    | makemv process delim="," 
    | mvexpand process 
    | fields - _time 
    | eval date=""]
| dedup process
| eval color=case(isnull(date) OR len(date)=0,"#d93f3c",true(),"#65a637")
| eval icon=case(process=="processed","gears",process=="send","paper-plane",process=="transferred","gift")
| eval date=case(isnull(date) OR len(date)=0,"??/??/????",true(),date)
| stats values(date) as date values(icon) as icon values(color) as color by process</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="drilldown">none</option>
        <option name="height">180</option>
        <option name="status_indicator_app.status_indicator.colorBy">field_value</option>
        <option name="status_indicator_app.status_indicator.fillTarget">background</option>
        <option name="status_indicator_app.status_indicator.fixIcon">warning</option>
        <option name="status_indicator_app.status_indicator.icon">field_value</option>
        <option name="status_indicator_app.status_indicator.precision">0</option>
        <option name="status_indicator_app.status_indicator.showOption">1</option>
        <option name="status_indicator_app.status_indicator.staticColor">#65a637</option>
        <option name="status_indicator_app.status_indicator.useColors">true</option>
        <option name="status_indicator_app.status_indicator.useThousandSeparator">false</option>
        <option name="trellis.enabled">1</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">small</option>
        <option name="trellis.splitBy">process</option>
      </viz>
    </panel>
  </row>
  <row>
    <panel>
      <title>Panel 2 - processed --> send</title>
      <viz type="status_indicator_app.status_indicator">
        <search>
          <query>| makeresults 
| eval data="filename=file1,process=processed,date=12/22/2017;filename=file1,process=send,date=12/23/2017;filename=file1," 
| makemv data delim=";" 
| mvexpand data 
| rename data as _raw 
| KV 
| search filename="file1"
| stats values(date) as date by process 
| appendpipe
    [| makeresults 
    | eval process="processed,send,transferred" 
    | makemv process delim="," 
    | mvexpand process 
    | fields - _time 
    | eval date=""]
| dedup process
| eval color=case(isnull(date) OR len(date)=0,"#d93f3c",true(),"#65a637")
| eval icon=case(process=="processed","gears",process=="send","paper-plane",process=="transferred","gift")
| eval date=case(isnull(date) OR len(date)=0,"??/??/????",true(),date)
| stats values(date) as date values(icon) as icon values(color) as color by process</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="drilldown">none</option>
        <option name="height">180</option>
        <option name="status_indicator_app.status_indicator.colorBy">field_value</option>
        <option name="status_indicator_app.status_indicator.fillTarget">background</option>
        <option name="status_indicator_app.status_indicator.fixIcon">warning</option>
        <option name="status_indicator_app.status_indicator.icon">field_value</option>
        <option name="status_indicator_app.status_indicator.precision">0</option>
        <option name="status_indicator_app.status_indicator.showOption">1</option>
        <option name="status_indicator_app.status_indicator.staticColor">#65a637</option>
        <option name="status_indicator_app.status_indicator.useColors">true</option>
        <option name="status_indicator_app.status_indicator.useThousandSeparator">false</option>
        <option name="trellis.enabled">1</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">small</option>
        <option name="trellis.splitBy">process</option>
      </viz>
    </panel>
  </row>
  <row>
    <panel>
      <title>Panel 3 - processed</title>
      <viz type="status_indicator_app.status_indicator">
        <search>
          <query>| makeresults 
| eval data="filename=file1,process=processed,date=12/22/2017;filename=file1" 
| makemv data delim=";" 
| mvexpand data 
| rename data as _raw 
| KV 
| search filename="file1"
| stats values(date) as date by process 
| appendpipe
    [| makeresults 
    | eval process="processed,send,transferred" 
    | makemv process delim="," 
    | mvexpand process 
    | fields - _time 
    | eval date=""]
| dedup process
| eval color=case(isnull(date) OR len(date)=0,"#d93f3c",true(),"#65a637")
| eval icon=case(process=="processed","gears",process=="send","paper-plane",process=="transferred","gift")
| eval date=case(isnull(date) OR len(date)=0,"??/??/????",true(),date)
| stats values(date) as date values(icon) as icon values(color) as color by process</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="drilldown">none</option>
        <option name="height">180</option>
        <option name="status_indicator_app.status_indicator.colorBy">field_value</option>
        <option name="status_indicator_app.status_indicator.fillTarget">background</option>
        <option name="status_indicator_app.status_indicator.fixIcon">warning</option>
        <option name="status_indicator_app.status_indicator.icon">field_value</option>
        <option name="status_indicator_app.status_indicator.precision">0</option>
        <option name="status_indicator_app.status_indicator.showOption">1</option>
        <option name="status_indicator_app.status_indicator.staticColor">#65a637</option>
        <option name="status_indicator_app.status_indicator.useColors">true</option>
        <option name="status_indicator_app.status_indicator.useThousandSeparator">false</option>
        <option name="trellis.enabled">1</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">small</option>
        <option name="trellis.splitBy">process</option>
      </viz>
    </panel>
  </row>
</dashboard>

Re: Can I create a visual of the status of a file using Splunk visualization tool?

chandana204 — Thu, 28 Dec 2017 18:06:49 GMT

Hi, filednames are same for processed and send but transferred. Not only these status but also need to show some other statuses like failed, error, fatal. It is kind of tracking. let's say, if a file got processed and send but transferred, created error event. I wanted to show all these in a dashboard like processes and send in green color transferred in no color and error in red color. It should be like if i give a file name this dashboard should show all information related to the file name process.

NOTE: I am working on log files.

Re: Can I create a visual of the status of a file using Splunk visualization tool?

chandana204 — Tue, 29 Sep 2020 17:27:11 GMT

Finally I able to visualized process, send, transferred and fatals also. But I caught up with two issues
1. In my data one file has many processes, send and fatals. After using dedup, I able to pulled last occurred process, send and fatals but it's not sorting in ascending order.
2. This particular file doesn't have transferred event. As per the below code it should show transferred block in grey color but it's not showing.

Below code i used

index="o" eventtype="x" "file_name" Process="Processing"
| append
[ search index="o" eventtype="x" "file_name" Process="Sending" ]
| append
[ search index="o" eventtype="x" "file_name" success="transferred"
| append
[ search index="o" eventtype="x" "file_name" Fatal="Error"] | dedup Process
| eval icon=case(Process="Processing","calendar-check-o", Process="Sending","calendar-check-o", success="transferred", "calendar-check-o", Fatal="Error", "calendar-times-o")
| eval time=strftime(_time,"%m-%d-%y %H:%M:%S")
| eval time=case(isnull(time),"N/A",true(),time)
| eval color=case(Process="Processing" AND time=="N/A","grey", Process="Processing" AND time!="N/A","green", Process="Sending" AND time=="N/A","grey", Process="Sending" AND time!="N/A","green", success="transferred" AND time=="N/A","grey", success="transferred" AND time!="N/A","green", Fatal="Error" AND time=="N/A","grey", Fatal="Error" AND time!="N/A","red")
| table Process success Fatal time icon color
| eval status=case(Process="Processing","Processed", Process="Sending","Send", success="transferred","Transferred", Fatal="Error","FATAL: Error")
| stats values(time) as time values(icon) as icon values(color) as color by status
| sort time

Output:
processed FATAL: Error Send
11-23-17 03:30:04 11-23-17 03:30:04 11-21-17 01:20:04

Re: Can I create a visual of the status of a file using Splunk visualization tool?

niketn — Fri, 29 Dec 2017 04:17:06 GMT

@chandana204, can you please add a mock of what you need as output with both Progress and Status combined?

I think you should color based on Status and depict the progress as text, in your case.

Re: Can I create a visual of the status of a file using Splunk visualization tool?

chandana204 — Mon, 08 Jan 2018 19:56:57 GMT

Thank you Niketnilay. I able to solve my problem based on the status indicator documentation.