Splunk Search

Unable to use results of Streamstats

Jason
Motivator

I'm dealing with some web logs, and have generated statistics on how long a certain user stayed on a certain page by using the streamstats command below:

search ... | streamstats current=t global=f window=2 range(_time) as Dur by User | eval Duration=if(isnull(Dur), 0, if(Dur>1800, 0, Dur)) | stats count by _time, User, Page, Duration | fields - count

This shows Duration, the amount of time a particular User spent on a particular Page. (The eval ignores times over 30 minutes; they are assumed to be different web sessions).



Now I am trying to do more things with Duration, such as sum it up per page, or make a total amount of time all users spent on all pages. But I am running into the same problem - I can't seem to use the Duration field!

search ... | streamstats current=t global=f window=2 range(_time) as Dur by User | eval Duration=if(isnull(Dur), 0, if(Dur>1800, 0, Dur)) | stats count sum(Duration) by Page

Gives an error, saying Specified field(s) missing from results: Duration



And when I try to sum up all Durations using eventstats so I can make a percentage calculation later,

search ... | streamstats current=t global=f window=2 range(_time) as Dur by User | eval Duration=if(isnull(Dur), 0, if(Dur>1800, 0, Dur)) | eventstats sum(Duration) as AllDuration

The AllDuration field doesn't even show up. What is going wrong here? I thought streamstats (especially followed by an eval) would definitely create a usable field like any other.



Behavior seen on both 4.1.5/Linux64 and 4.1.5/Windows32.

Tags (3)
1 Solution

Jason
Motivator

Turns out it was weird because I was running stuff on a summary index that had been populated by sistats.

I thought you had to populate a summary index with sistats, but it turns out that's only if you plan to do the exact stats query when looking at the summary index. My workaround was to use the fields Duration, fields.. to kick out some prsrvd_* fields that were messing with the functionality of stats.

View solution in original post

Jason
Motivator

Turns out it was weird because I was running stuff on a summary index that had been populated by sistats.

I thought you had to populate a summary index with sistats, but it turns out that's only if you plan to do the exact stats query when looking at the summary index. My workaround was to use the fields Duration, fields.. to kick out some prsrvd_* fields that were messing with the functionality of stats.

Jason
Motivator

Yes, stats count by Duration, fields is the only thing that works. stats sum(Duration) by fields fails, as does | eventstats sum(Duration) as Total | stats count by Total, fields.

Lowell
Super Champion

Weird. I'm running 4.1.5 on Linux 32 bit, and I tried a similar search with no issues. This search worked fine: sourcetype=*ftpd* | streamstats current=t global=f window=2 range(_time) as Dur by pid | eval Duration=if(isnull(Dur), 0, if(Dur>1800, 0, Dur)) | stats count by Duration

0 Karma

Jason
Motivator

I tried adding [Duration] INDEXED_VALUE = false to my app's fields.conf, but this didn't work.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...