topic Re: help on post process search with timechart in Splunk Search

Why doesn't my post process search work when using timechart command?

jip31 — Wed, 16 Nov 2022 07:21:48 GMT

hello

Why doesn't my post process search work when using timechart command?

<search id="cap"> <query> `index_mes` (sourcetype=web_request OR sourcetype=web:request) </query> <earliest>$date.earliest$</earliest> <latest>$date.latest$</latest> </search>

<row> <panel> <chart> <search base="cap"> <query> | timechart span=15m dc(sam) as cap</query>

Thanks

Re: help on post process search with timechart

gcusello — Tue, 15 Nov 2022 16:25:14 GMT

Hi @jip31,

if you don't use a streaming command (as stats o timechart) in the main search, you have to add, at the end of the main search, the fields command with the list of the fields that you have to use in the panel, in your case

index_mes` (sourcetype=web_request OR sourcetype=web:request) | fields _time sam

Ciao.

Giuseppe

Re: help on post process search with timechart

jip31 — Tue, 15 Nov 2022 16:55:32 GMT

sorry, there was a mistake in my example...

it's exactly what I have done but it doesnt works

here is my code

<search id="cap"> <query> `index_mesurescontinues` (sourcetype=web_request OR sourcetype=web:request) | fields web_url web_duration_ms sam web_domain site </query> <earliest>$date.earliest$</earliest> <latest>$date.latest$</latest> </search> <row> <panel> <chart> <search base="cap"> <query> | timechart span=1d avg(web_duration_ms) as web_duration_ms | eval "web_duration_ms"=round('web_duration_ms',0) | eventstats avg("web_duration_ms") as Moyenne | eval Moyenne=round(Moyenne,0) </query> </search>

Re: help on post process search with timechart

gcusello — Tue, 15 Nov 2022 17:01:23 GMT

Hi @jip31,

you missed _time that you use in timechart.

Ciao.

Giuseppe

Re: help on post process search with timechart

jip31 — Wed, 16 Nov 2022 06:18:35 GMT

It changes nothing.....

here is my code

<search id="cap"> <query> `index_mes` (sourcetype=web_request OR sourcetype=web:request) | fields web_url web_duration_ms sam web_domain _time </query> <earliest>$date.earliest$</earliest> <latest>$date.latest$</latest> </search> <row> <panel> <chart> <search base="cap"> <query> | timechart span=1d avg(web_duration_ms) as web_duration_ms </query>

Re: help on post process search with timechart

gcusello — Wed, 16 Nov 2022 07:15:03 GMT

Hi @jip31,

if you run the main search by itself, have you both the fields _time and web_duration_ms?

Check the filename of the second field.

Ciao.

Giuseppe

Re: help on post process search with timechart

jip31 — Wed, 16 Nov 2022 07:50:42 GMT

yes I have

Re: help on post process search with timechart

bowesmana — Wed, 16 Nov 2022 08:00:01 GMT

Your image shows _time and count - but your search is not calculating a field called count, so that image can't come from the search your are referring to

If you open your panel search using the magnifying glass icon in your timechart panel, what is the exact search and what are the results?

Re: help on post process search with timechart

jip31 — Wed, 16 Nov 2022 08:31:49 GMT

here is the search

`index_mes` (sourcetype=web_request OR sourcetype=web:request) | fields web_url web_duration_ms sam web_domain _time | timechart span=1d avg(web_duration_ms) as web_duration_ms

and the result

Re: help on post process search with timechart

bowesmana — Wed, 16 Nov 2022 22:55:35 GMT

OK, so what doesn't work here?

Re: help on post process search with timechart

jip31 — Thu, 17 Nov 2022 06:50:28 GMT

as I said, the post process search doesnt returns results even if I use _time in | fields....

for being more precise I have results just on the last day of the timepicker choice

for example, if I chose a time between the 25 oct and the 29 oct, I just can see the resulst between the 28 oct and the 29 oct even if events between the 25 oct and te 28 oct exists!

Re: help on post process search with timechart

bowesmana — Thu, 17 Nov 2022 07:26:42 GMT

Your previous screenshot showed results from November 9, 10, 11, 12 and 13, so it has _time otherwise it would never show that display

I take it you created that previous table by clicking the small magnifying glass in the timechart display

Your latest chart shows a valid timechart with results from 28 and 29 of October, so clearly it IS giving _time.

If you click on the magnifying glass for the chart you just showed, what is the table it generates?

It seems your problem is more an issue that is not displaying ALL the results you expect rather than NOT showing _time.

Can you confirm?

Re: help on post process search with timechart

jip31 — Thu, 17 Nov 2022 08:14:06 GMT

yes it seems

but why it is not displaying all results just when i use a post process search?

because if in run the search inline i have all the resulst

Re: help on post process search with timechart

bowesmana — Thu, 17 Nov 2022 08:49:10 GMT

Is that when you click the magnifying glass?

Temporarily change the visualisation from chart to table to see the results in the dashboard panel

Re: help on post process search with timechart

jip31 — Thu, 17 Nov 2022 12:16:14 GMT

yes it is

same problem with table viz

Re: help on post process search with timechart

bowesmana — Fri, 18 Nov 2022 01:50:07 GMT

Using base searches to collect data just by doing

index=x bla... | fields *

is a bad use of base searches. You are probably hitting the event limit of 500,000 I believe.

Using the type of base search you are doing will not necessarily improve performance with large data sets and can often slow down searches.

What are other types of post process searches you are doing with this base search?

You should always aim to do some kind of aggregation with a base search.

Re: help on post process search with timechart

jip31 — Fri, 18 Nov 2022 04:43:15 GMT

I have truncated my search for the example but it has other filters before the post process

And you are right i have more than 500000 events....

When i use stats instead timechart the post process search works fine...

And if i use the timechart inline, the chart is cotrectly so i am going to do this

Thanks

Re: help on post process search with timechart

bowesmana — Fri, 18 Nov 2022 06:22:57 GMT

One technique you can use for base searches where you want to keep data for many possible post processing variants is to make your base search something like

<search> | bin _time span=1d | stats count by _time a b c d e f g

which does work for a number of use case, for example, you can then still do a timechart after this base search with

| timechart span=1d sum(count) as count by a c e

so you can get creative with base searches, but if you are just doing this for performance, make sure you evaluate the performance of these searches, so you can see if it really is making a difference or not.