Optimize my search

[jyab6z]

This is my search:

Function="- Parts::GetPartSection =>" | rex "maingroupNo\>(?.+)\\(?.+)\\(?.+)\" | convert timeformat="%H:%M:%S" dur2sec(TimeInSec) | stats count first(TimeInSec) AS endTime last(TimeInSec) AS startTime by UserID DATE maingroup subgroup | appendcols [search Function="- Parts::GetPartSection =>" | rex "maingroupNo\>(?.+)\\(?.+)\\(?.+)\"  | stats list(secId) as secId_new by UserID DATE maingroup subgroup] | where count > 5 | eval Time_diff = abs(endTime - startTime) | eval avgTime_count = Time_diff/count | where avgTime_count < 20 | where maingroup=="03" AND subgroup=="26" | mvexpand secId_new | stats count by secId_new | rename secId_new as Avsnitt

THEN got this error when it runs over 1 year's data.

Any ideas?
Thanks in advance!

[richgalloway · Accepted solution]

The subsearch is trying to process too much data. Why do you even need it? You should be able to combine the two searches into a single search. See if this accomplishes the same task.

Function="- Parts::GetPartSection =>" | rex "maingroupNo\>(?.+)\\(?.+)\\(?.+)\" | convert timeformat="%H:%M:%S" dur2sec(TimeInSec) | stats count first(TimeInSec) AS endTime, last(TimeInSec) AS startTime, list(secId) as secId_new by UserID DATE maingroup subgroup | where count > 5 | eval Time_diff = abs(endTime - startTime) | eval avgTime_count = Time_diff/count | where avgTime_count < 20 | where maingroup=="03" AND subgroup=="26" | mvexpand secId_new | stats count by secId_new | rename secId_new as Avsnitt

[jyab6z]

Ohh, thank you!
It helps!!!