Splunk Search

Help understanding appendpipe

chirsf
Explorer

Hi,

I didn't find anything about this while searching so here's my question.

I'm working on the proving a negative problem, adding appendpipe after a stats in order to display a result of 0 for each day for the period of time I need. I usually do this for a single row, however I need to have multiple rows for multiple days as output for stats or more importantly timechart.

I ran into a scenario I cannot explain and wanted to understand further. While testing I created this search:

 

 

 

| makeresults 
| eval value=0, category="test", _time=strftime(now(), "%H") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") ] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")]
| stats count by _time

 

 

 

The results of this output 256 results for a single date/time, and others follow with smaller amounts but not counts of 1.

If I change it to this:

 

 

 

| makeresults 
| eval value=0, category="test", _time=relative_time(now(), "-2d@d") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") 
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")
    | dedup value category _time]
| stats count by _time

 

 

 

Every row has a single count except for one, which makes sense given how this is written. I can move forward with this, but now I would like to know why this happens. 

 

 

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

appendpipe is operating on each event in the pipeline, so the first appendpipe only has one event (the first you created with makeresults) to work with, and it appends a new event to the pipeline. The second appendpipe now has two events to work with, so it appends a new event for each event, making a total of 4. The third appendpipe doubles your events again, and so on.

View solution in original post

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

appendpipe is operating on each event in the pipeline, so the first appendpipe only has one event (the first you created with makeresults) to work with, and it appends a new event to the pipeline. The second appendpipe now has two events to work with, so it appends a new event for each event, making a total of 4. The third appendpipe doubles your events again, and so on.

0 Karma

chirsf
Explorer

Thanks, this makes total sense. I don't know if my solution here is the correct one, I mean it works so in that vein it's correct. However I feel like it's.. a hack lol.

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Your approach is probably more hacky than others I have seen 😀 - you could use append with makeresults (append at the end of the pipeline rather than after each event), you could use union with makeresults, you could use makecontinuous over the time field (although you would need more than one event so append/makeresults or something similar would still be required). There are many ways to skin that cat. 😀

0 Karma

chirsf
Explorer

Thanks to mmcul on slack this is the answer I'm going with:

 

| append 
    [| gentimes start=-14 end=0 increment=1d 
    | eval _time=starttime, category="test", value=0
    | fields _time, category, value ]
0 Karma

chirsf
Explorer

Yea I thought about using makecontinuous but I cannot guarantee even a single event will show up for the time range I'm looking for to use that, or I misunderstand how that works. Thanks for the leads on the other ideas i appreciate it.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...