topic Re: date generation is wrong using gentimes in Splunk Search

date generation is wrong using gentimes

angelinealex — Thu, 15 Mar 2018 04:38:17 GMT

Hi,

I am using below code snippet to generate previous 12 months.

I see two problems here,

its generating 2017-11-05 twice
its missing 2018-03-11

What am I missing? Please help.

Re: date generation is wrong using gentimes

p_gurav — Thu, 15 Mar 2018 04:45:16 GMT

Your query seems ok. I run that query and it giving me proper 365 results.

Re: date generation is wrong using gentimes

MousumiChowdhur — Thu, 15 Mar 2018 04:50:10 GMT

Hi!

I checked the same code but it's giving the correct result. It's printing 2017-11-05 once and 2018-03-11 does exist.

Thanks.

Re: date generation is wrong using gentimes

angelinealex — Thu, 15 Mar 2018 04:58:32 GMT

ahh ok.. then why i am getting wrong result with duplicate date and missing date. Any guess?

Re: date generation is wrong using gentimes

angelinealex — Thu, 15 Mar 2018 04:58:52 GMT

ahh ok.. then why i am getting wrong result with duplicate date and missing date. Any guess?

Re: date generation is wrong using gentimes

p_gurav — Thu, 15 Mar 2018 05:01:08 GMT

Try rerunning search, if still it gives duplicates use dedup command.

Also use below query :

| gentimes start=-365 end=-0 increment=0d | eval Date=strftime(starttime,"%Y-%m-%d") |sort - Date| table Date

Re: date generation is wrong using gentimes

angelinealex — Thu, 15 Mar 2018 16:04:04 GMT

The above query resulted the same. If I use dedup command then i am not getting duplicate date. But still its missing 2018-03-11.

Re: date generation is wrong using gentimes

niketn — Thu, 15 Mar 2018 17:38:04 GMT

@angelinealex can you try the following search and check results?

| gentimes start=-365 end=-0 increment=1d 
| eval Date=strftime(starttime,"%Y-%m-%d") 
| reverse
| fields Date

Re: date generation is wrong using gentimes

angelinealex — Thu, 15 Mar 2018 18:42:39 GMT

I get the same result for the above query as well.

I figured out the problem. but looking for the solution.
Because of day light saving changes its duplicating 5th Nov, 2017 and missing 11th Mar, 2018.

I am in PST timezone.

Re: date generation is wrong using gentimes

niketn — Fri, 16 Mar 2018 02:50:17 GMT

@angelinealex First off... "good catch". I think you have found a bug with gentimes command. You should add bug tag to this question and also report to Splunk Support if you have valid Splunk Entitlement.

Meanwhile, you can try the following search:

| gentimes start=-365 end=-0 increment=1d
| eval Date=strftime(starttime,"%Y-%m-%d")
| fields Date
| dedup Date
| append [| makeresults | eval Date="2018-03-11" | fields - _time]
| sort Date
| reverse

Re: date generation is wrong using gentimes

angelinealex — Sat, 17 Mar 2018 01:36:48 GMT

This gives my correct result. But i cant hard code March 11 since it changes every year.

Re: date generation is wrong using gentimes

angelinealex — Sat, 17 Mar 2018 01:46:11 GMT

    | gentimes start=1/16/17 end=-0 increment=0d 
    | eval Date=strftime(starttime,"%Y-%m-%d") 
    | sort Date 
    | reverse 
    | table Date

The above sample code gives me the proper result.. i mean without duplicate and missing date from 1/16/17. But how do i pass the dynamic date to start?
Daily I should get previous 12 months dates.

Re: date generation is wrong using gentimes

niketn — Sat, 17 Mar 2018 03:49:00 GMT

Following two work (Starting with Jan/Feb)

| gentimes start="01/16/2017" end=-0 increment=0d

| gentimes start="02/16/2017" end=-0 increment=0d

But following does not (starting with March). If you try previous 12 month dates you will have a date in March which gives same issue as what you have noticed. Definitely a BUG with gentimes command.

| gentimes start="03/16/2017" end=-0 increment=0d

Re: date generation is wrong using gentimes

niketn — Sat, 17 Mar 2018 04:02:43 GMT

@angelinealex, you can use map to get the first day of last year i.e. 01/01/2017 and then pass to gentimes to get all the dates. Finally after reverse, you can retain only first 365 results using head 365

| makeresults
| eval startDateLastYear=strftime(relative_time(_time,"-1y@y"),"%m/%d/%Y")
| fields - _time
| map search="| gentimes start=\"$startDateLastYear$\" end=-0 increment=0d 
| eval Date=strftime(starttime,\"%Y-%m-%d\") 
| sort Date 
| reverse 
| table Date
| head 365"

You can also try the following dashboard code on similar lines to set the token for first day of last year i.e. 01/01/2017 and then pass on the token to search query with gentimes

<dashboard>
  <label>gentimes last one year daylight saving issue</label>
  <search>
    <query>| makeresults
| eval startDateLastYear=strftime(relative_time(_time,"-1y@y"),"%m/%d/%Y")
| fields - _time
    </query>
    <earliest>-1s@s</earliest>
    <latest>@s</latest>
    <done>
      <set token="tokStartDate">$result.startDateLastYear$</set>
    </done>
  </search>
  <row>
    <panel>
      <table>
        <search>
          <query>| gentimes start="$tokStartDate$" end=-0 increment=0d 
| eval Date=strftime(starttime,"%Y-%m-%d") 
| sort Date 
| reverse 
| table Date
| head 365
          </query>
        </search>
      </table>
    </panel>
  </row>
</dashboard>

Please try out and confirm.

Re: date generation is wrong using gentimes

angelinealex — Tue, 20 Mar 2018 02:13:34 GMT

saved search just worked like champ. Thank you very much for all your help

Re: date generation is wrong using gentimes

niketn — Tue, 20 Mar 2018 13:32:34 GMT

Glad you found this working mean while I have submitted a feedback on Splunk Docs that daylight saving start and end dates are not working as expected for gentimes command!

Re: date generation is wrong using gentimes

angelinealex — Tue, 20 Mar 2018 18:15:34 GMT

Great, Please let me also know if you get any response for the feedback 🙂

Re: date generation is wrong using gentimes

niketn — Thu, 29 Mar 2018 11:04:17 GMT

Splunk team has reopened an existing JIRA ticket for this issue JIRA # SPL-146219.

Re: date generation is wrong using gentimes

angelinealex — Fri, 30 Mar 2018 03:07:13 GMT

Cool. Thank you for letting me know.