topic Re: How to write a search for continuous day count? in Splunk Search

How to write a search for continuous day count?

srinivasup — Wed, 10 May 2017 19:13:44 GMT

I have a scenario that when i write a search, i will get count for each day. But if there is no count that day, the row is not appearing

Ex:

Day count
Mon 1
Tue 2
Sat 3

This is the output, but here Wed, Thu, Friday days are missing. So i would like hard code the days or want to include missing dates with count zero

Something like this

Day count
Mon 1
Tue 2
Wed 2
Thu 3
Fri 5
Sat 6

Sometimes days may change , please help in writing a search

Re: How to write a search for continuous day count?

somesoni2 — Wed, 10 May 2017 19:30:15 GMT

What's your current query and time-range??

Re: How to write a search for continuous day count?

somesoni2 — Wed, 10 May 2017 22:14:09 GMT

Assuming that values of days are always Mon to Sat, you can try like this

your current search 
| append [| gentimes start=-1 | eval Day=split("Mon Tue Wed Thu Fri Sat"," ") | table Day | mvexpand Day | eval count=0 ]
| stats max(count) as count by Day

Re: How to write a search for continuous day count?

srinivasup — Thu, 11 May 2017 03:17:33 GMT

sorry for the confusion

Day count
Mon Mar.1.2017 1
Tue Mar.2.2017 2
Thu Mar.3.2017 4
Fri Mar.5.2017 7
Sat Mar.6.2017 8

now want to include missing day count as 0

Re: How to write a search for continuous day count?

somesoni2 — Thu, 11 May 2017 03:24:20 GMT

Give this a try

your current search 
 | append [| gentimes start=-1 | addinfo | eval Day=mvrange(info_min_time,info_max_time+86400,86400)  | table Day | mvexpand Day | eval Day=strftime(Day,"%a %b.%d.%Y") | eval count=0 ]
 | stats max(count) as count by Day

May need to adjust mvrange command upper limit.

Re: How to write a search for continuous day count?

niketn — Thu, 11 May 2017 04:08:11 GMT

What is the final transforming command you are using to create your statistical table?

Re: How to write a search for continuous day count?

cmerriman — Thu, 11 May 2017 12:04:36 GMT

can you use the |makecontinuous command on your _time field? followed by |fillnull count value=0

https://docs.splunk.com/Documentation/Splunk/6.6.0/SearchReference/Makecontinuous

Re: How to write a search for continuous day count?

srinivasup — Thu, 11 May 2017 18:19:16 GMT

my query |inputlookup Day.csv and output is as follows

Day count

Mon May.2.2017 1
Tue May.3.2017 2
Sat May.7.2017 3

i want to see the output

Mon May.2.2017 1
Tue May.3.2017 2
wed 0
Thu 0
Fri 0
Sat May.7.2017 3

tried above query but didnt work

Re: How to write a search for continuous day count?

srinivasup — Thu, 11 May 2017 18:23:33 GMT

my query |inputlookup Day.csv and output is as follows

Day count

Mon May.2.2017 1
Tue May.3.2017 2
Sat May.7.2017 3

i want to see the output

Mon May.2.2017 1
Tue May.3.2017 2
wed 0
Thu 0
Fri 0
Sat May.7.2017 3

tried above query but didnt work

Re: How to write a search for continuous day count?

somesoni2 — Thu, 11 May 2017 18:36:13 GMT

Using @cmerriman's solution, this should do it
Updated

|inputlookup Day.csv
| eval _time=strptime(Day,"%a %b.%d.%Y")
| makecontinuous | eval Day=strftime(_time,"%a %b.%d.%Y") | fields - _time
| fillnull value=0 count

Re: How to write a search for continuous day count?

srinivasup — Thu, 11 May 2017 18:58:10 GMT

Day _time count
Mon May.2.2017 2017-05-02 1
Tue May.3.2017 2017-05-03 2
2017-05-04 0
2017-05-05 0
2017-05-06 0
Sat May.7.2017 2017-05-07 3

This is how it is comming but under Day column it should display weekdays but it is showking blanks, please help

Re: How to write a search for continuous day count?

srinivasup — Thu, 11 May 2017 20:20:02 GMT

And its not displaying sunday

Re: How to write a search for continuous day count?

woodcock — Fri, 12 May 2017 05:32:05 GMT

Like this:

| makeresults 
| eval raw="Mon 1::Tue 2::Sat 3" 
| makemv delim="::" raw 
| mvexpand raw 
| rex field=raw "^(?<Day>\S+)\s+(?<count>\S+)$" 
| fields - _time raw 

| rename COMMENT AS "Everything above fakes sample (intermediate) data; everything below is your solution"
| rename COMMENT AS "YOUR EXISTING SEARCH HERE"

| append 
    [| makeresults count=7 
    | streamstats count AS serial 
    | eval Day=case(serial=1, "Mon", serial=2, "Tue", serial=3, "Wed", serial=4, "Thu", serial=5, "Fri", serial=6, "Sat", serial=7, "Sun", true(), "N/A")
    | fields - _time
    | eval count=0 ]
| stats sum(count) AS count first(serial) AS serial BY Day
| sort 0 serial
| fields - serial
| eval count=if(count=0, null(), count)
| filldown count
| fillnull count value="0"

Re: How to write a search for continuous day count?

HiroshiSatoh — Fri, 12 May 2017 10:29:26 GMT

try this!

|gentimes start=-7|eval day=strftime(endtime,"%a") ,count=0|fields day,count
  |join type=outer day [search (your search)|eval day=strftime(_time,"%a")|stats count by day]

Re: How to write a search for continuous day count?

srinivasup — Sun, 21 May 2017 17:11:50 GMT

this is working but we need to append time for weekdays

Re: How to write a search for continuous day count?

woodcock — Sun, 21 May 2017 18:30:12 GMT

I do not understand. What is your output now and what should it be?

Re: How to write a search for continuous day count?

srinivasup — Sun, 21 May 2017 18:48:26 GMT

Input:

Day count
Mon May.2.2017 1
Tue May.3.2017 2
Sat May.7.2017 3

Output:

Day count
Mon 1
Tue May.3.2017 2
Wed ...............

Thu ...............

Fri ...............

Sat May.7.2017 3
Sun ...............

As per the previous query we could able to add week days but now we want to add days corresponding to each week day

Re: How to write a search for continuous day count?

srinivasup — Sun, 21 May 2017 18:50:32 GMT

Input:

Day count
Mon May.2.2017 1
Tue May.3.2017 2
Sat May.7.2017 3

Output:

Day count
Mon 1
Tue May.3.2017 2
Wed ...............
Thu ...............
Fri ...............
Sat May.7.2017 3
Sun ...............

As per the previous query we could able to add week days but now we want to add days corresponding to each week day

Re: How to write a search for continuous day count?

woodcock — Tue, 23 May 2017 00:00:40 GMT

OK, try this:

| makeresults 
| eval raw="Mon 1::Tue 2::Sat 3" 
| makemv delim="::" raw 
| mvexpand raw 
| rex field=raw "^(?<Day>\S+)\s+(?<count>\S+)$" 
| fields - _time raw 
| rename COMMENT AS "Everything above fakes sample (intermediate) data; everything below is your solution"
| rename COMMENT AS "YOUR EXISTING SEARCH HERE"
| append 
    [| makeresults count=7 
    | streamstats count AS serial 
    | eval serial = serial - 1
    | eval Day=case(serial=0, "Sun", serial=1, "Mon", serial=2, "Tue", serial=3, "Wed", serial=4, "Thu", serial=5, "Fri", serial=6, "Sat", true(), "N/A")
    | eval time = relative_time(now(), "@w") + (serial  * 60 * 60 * 24)
    | eval count=0]
| stats sum(count) AS count first(serial) AS serial first(time) AS Day_Detail BY Day
| sort 0 serial
| fields - serial
| eval count=if(count=0, null(), count)
| filldown count
| fillnull count value="0"
| fieldformat Day_Detail = strftime(Day_Detail, "%a %b %d %Y")

It may not be exactly perfect but it has all the moving parts that you should need to massage it to your needs.

Re: How to write a search for continuous day count?

srinivasup — Wed, 31 May 2017 14:19:46 GMT

|gentimes start=-7 -- it looks for last 7 days.

I want to update start time like earliest time

ex: earliest=-w@w latest=@w, usually it will fetch the records for last week,

How can i pass these earliest and latest values to gentimes start =-w@w end=@w