Hello! I'm having this issue of merging data from one sourcetype to another larger sourcetype. Example:
index=ecs_internal (sourcetype=ecs:encode parcel_id=* earliest=-30d@d latest=@d) OR (sourcetype=ecs:input barcode_id=* earliest=-60d@d latest=@d)
| eval parcel_id=if(isnotnull(parcel_id), parcel_id, barcode_id)
| stats latest(*) as *, sum(eval(if(sourcetype=ecs:encode, 1, 0))) as valid by parcel_id
| where valid >= 1
Aim: Display all events in ecs:encode, then lookup latest related information from ecs:input. Basically, only 30% of all events in ecs:input would have data relevant to ecs:encode, thus my search is extremely slow... Join command would not work as I'm having millions of events in both sourcetypes.
Many thanks in advance!
Isn't the phrase:
| where valid >=1
effectively saying don't show any stats that aren't in sourcetype=ecs:input?
Hi memarshall63, yes it works, but problem is the search takes too much time (currently it's taking me more than 1 hour!). Query above is just a simplified version, I have to do a lot of evals before the stats.