Why is the append command in my search showing incorrect results?

[HeinzWaescher]

Hi,

My search looks like this:

base search...
| timechart span=1d  dc(user_id) AS daily_customers
| timechart span=1mon avg(daily_customers) AS avg_daily_customers
| eval avg_daily_customers=round(avg_daily_customers,0)

| append  [base search...
| timechart span=1mon  dc(user_id) AS daily_customers]

The base search is about 14.000.000 events in both cases. The append command shows up incorrect results and I think this is caused by the maxtime and/or timeout . I tried to increase these parameters, but afterwards, nothing is appended.

| append  [base search... maxtime=1800 timeout=1800
| timechart span=1mon  dc(user_id) AS daily_customers]

What am I doing wrong with the append command?

BR

Heinz

[martin_mueller · Accepted solution]

Replace this

... | append [base search... maxtime=1800 timeout=1800 ...]

with this

... | append maxtime=1800 timeout=1800 [...]

http://docs.splunk.com/Documentation/Splunk/6.2.4/SearchReference/append

Additionally, I'd question any case that executes the same base search twice only to apply different stats.

[martin_mueller]

I'm not sure if there are limits on these limits, but if you need a six-hour append you're in serious need of some refactoring.

How 'bout something along these lines?

  base search | bin span=1d _time | stats count by _time user_id
| eventstats dc(user_id) as daily_customers by _time
| timechart span=1mon avg(daily_customers) AS avg_daily_customers dc(user_id) AS daily_customers
| eval avg_daily_customers=round(avg_daily_customers,0)

[HeinzWaescher]

I think in practice I would need an append of 15min, I was just interested in the possibilities of these options.
Generally I try to avoid eventstats for large searches, because the search performance was very low, when I used it in the past. But I will give it a try again

[HeinzWaescher]

This works fine, thanks!
Are there limits for these options or can it set them to 6 hours as well?
I used the same base search twice because I wanted to combine a new metric out of both results. This is the only way I was able to do this.

[woodcock]

Is it anything more than that you should be using AS monthly_customers instead?

The reason that you get nothing in your last example is because those are settings that go inside limits.conf or on the outiside, not inside your search; when they are inside your search, they are treated as fields and values and since your events do not have a field called maxtime (let alone one with a value of 1800), your search returns 0 events (which is correct).