Splunk Search

Help understanding appendpipe

chirsf
Explorer

Hi,

I didn't find anything about this while searching so here's my question.

I'm working on the proving a negative problem, adding appendpipe after a stats in order to display a result of 0 for each day for the period of time I need. I usually do this for a single row, however I need to have multiple rows for multiple days as output for stats or more importantly timechart.

I ran into a scenario I cannot explain and wanted to understand further. While testing I created this search:

 

 

 

| makeresults 
| eval value=0, category="test", _time=strftime(now(), "%H") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") ] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")]
| stats count by _time

 

 

 

The results of this output 256 results for a single date/time, and others follow with smaller amounts but not counts of 1.

If I change it to this:

 

 

 

| makeresults 
| eval value=0, category="test", _time=relative_time(now(), "-2d@d") 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-1d@d") 
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-2d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-3d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-4d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-5d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-6d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-7d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-8d@d")
    | dedup value category _time] 
| appendpipe 
    [| eval value=0, category="test", _time=relative_time(now(), "-9d@d")
    | dedup value category _time]
| stats count by _time

 

 

 

Every row has a single count except for one, which makes sense given how this is written. I can move forward with this, but now I would like to know why this happens. 

 

 

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

appendpipe is operating on each event in the pipeline, so the first appendpipe only has one event (the first you created with makeresults) to work with, and it appends a new event to the pipeline. The second appendpipe now has two events to work with, so it appends a new event for each event, making a total of 4. The third appendpipe doubles your events again, and so on.

View solution in original post

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

appendpipe is operating on each event in the pipeline, so the first appendpipe only has one event (the first you created with makeresults) to work with, and it appends a new event to the pipeline. The second appendpipe now has two events to work with, so it appends a new event for each event, making a total of 4. The third appendpipe doubles your events again, and so on.

0 Karma

chirsf
Explorer

Thanks, this makes total sense. I don't know if my solution here is the correct one, I mean it works so in that vein it's correct. However I feel like it's.. a hack lol.

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Your approach is probably more hacky than others I have seen 😀 - you could use append with makeresults (append at the end of the pipeline rather than after each event), you could use union with makeresults, you could use makecontinuous over the time field (although you would need more than one event so append/makeresults or something similar would still be required). There are many ways to skin that cat. 😀

0 Karma

chirsf
Explorer

Thanks to mmcul on slack this is the answer I'm going with:

 

| append 
    [| gentimes start=-14 end=0 increment=1d 
    | eval _time=starttime, category="test", value=0
    | fields _time, category, value ]
0 Karma

chirsf
Explorer

Yea I thought about using makecontinuous but I cannot guarantee even a single event will show up for the time range I'm looking for to use that, or I misunderstand how that works. Thanks for the leads on the other ideas i appreciate it.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...