It is more likely that your performance issue is caused by the sort+streamstats rather than the lookup Here is an example that does not use sort or streamstats - it may or may not work in your data,...
See more...
It is more likely that your performance issue is caused by the sort+streamstats rather than the lookup Here is an example that does not use sort or streamstats - it may or may not work in your data, but the principle is to use stats. You can run this example and it will give you your results. The piece you would want is shown by the comment before the fields statement. | makeresults format=csv data="_time,DEV_ID,case_name,case_action
01:00,111,ping111.py,start
01:20,111,ping111.py,end
02:00,222,ping222.py,start
02:30,222,ping222.py,end
02:40,111,ping222.py,start
03:00,111,ping222.py,end"
| eval _time=strptime("2023-11-21 "._time.":00", "%F %T")
| append [
| makeresults format=csv data="_time,LOG_ID,Message_Name
01:10,01,event_a
02:50,02,event_a"
| eval _time=strptime("2023-11-21 "._time.":00", "%F %T")
| eval DEV_ID=111
]
``` So use your first two lines of your search and then the following```
| fields _time DEV_ID case_name case_action LOG_ID Message_Name
| eval t=if(isnull(LOG_ID),printf("%d##%s##%s", _time, case_action, case_name), null())
| eval lt=if(isnull(LOG_ID),null,printf("%d##%s##%s", _time, LOG_ID, Message_Name))
| fields - LOG_ID Message_Name case_*
| stats values(*) as * by DEV_ID
| where isnotnull(lt)
| mvexpand lt
| eval s=split(lt, "##")
| eval _time=mvindex(s, 0), LOG_ID=mvindex(s, 1), Message_Name=mvindex(s,2)
| rex field=t max_match=0 "(?<report_time>\d+)##(?<case_action>[^#]*)##(?<case_name>.*)"
| eval min_ix=-1
| eval c = 0
| foreach mode=multivalue report_time [ eval min_ix=if(_time > '<<ITEM>>', c, min_ix), c=c+1 ]
| eval case_name=if(min_ix>=0, mvindex(case_name, min_ix), "unknown")
| eval case_action=if(min_ix>=0, mvindex(case_action, min_ix), "unknown")
| fields - s lt t c min_ix report_time
| table _time Message_Name LOG_ID DEV_ID case_name