Help understanding appendpipe

[chirsf]

Hi,

I didn't find anything about this while searching so here's my question.

I'm working on the proving a negative problem, adding appendpipe after a stats in order to display a result of 0 for each day for the period of time I need. I usually do this for a single row, however I need to have multiple rows for multiple days as output for stats or more importantly timechart.

I ran into a scenario I cannot explain and wanted to understand further. While testing I created this search:

| makeresults 
| eval value=0, category="test", _time=strftime(now(), "%H") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") ] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")]
| stats count by _time

The results of this output 256 results for a single date/time, and others follow with smaller amounts but not counts of 1.

If I change it to this:

| makeresults 
| eval value=0, category="test", _time=relative_time(now(), "-2d@d") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") 
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")
    | dedup value category _time]
| stats count by _time

Every row has a single count except for one, which makes sense given how this is written. I can move forward with this, but now I would like to know why this happens. 

[ITWhisperer · Accepted solution]

appendpipe is operating on each event in the pipeline, so the first appendpipe only has one event (the first you created with makeresults) to work with, and it appends a new event to the pipeline. The second appendpipe now has two events to work with, so it appends a new event for each event, making a total of 4. The third appendpipe doubles your events again, and so on.

[chirsf]

Thanks, this makes total sense. I don't know if my solution here is the correct one, I mean it works so in that vein it's correct. However I feel like it's.. a hack lol.

[ITWhisperer]

Your approach is probably more hacky than others I have seen 😀 - you could use append with makeresults (append at the end of the pipeline rather than after each event), you could use union with makeresults, you could use makecontinuous over the time field (although you would need more than one event so append/makeresults or something similar would still be required). There are many ways to skin that cat. 😀

[chirsf]

Thanks to mmcul on slack this is the answer I'm going with:

| append 
    [| gentimes start=-14 end=0 increment=1d 
    | eval _time=starttime, category="test", value=0
    | fields _time, category, value ]

[chirsf]

Yea I thought about using makecontinuous but I cannot guarantee even a single event will show up for the time range I'm looking for to use that, or I misunderstand how that works. Thanks for the leads on the other ideas i appreciate it.