So far I created this Join index="index" "mysearchtext"
| rex field=message ", request_id: \\\"(?<request_id>[^\\\"]+)"
| fields _time request_id
| eval matchfield=request_id
| join matchfield [ search index="index"
| spath request.id
| rename request.id as id
| fields mynewfield
| eval matchfield=id
| table _time request_id mynewfield Basically I want to join 2 logs where request_id = id . The join is working as expected but as you expect is not efficient. I'd like to replace it with a more efficient search leveraging the fact that the events of the subsearch where I extract the field "mynewfield" are indexed for sure after some milliseconds the main search (where I extract the field request_id) Another useful info is that the logs that matches "mysearchtext" are way less than the logs in the subsearch Here a sample of the data {"AAA": "XXX","CCC":"DDD","message":{"request":{ "id": "MY_REQUEST_ID"} } }
{"AAA": "XXX","CCC":"DDD","message":"application logs in text format e.g. 2024/04/26 06:35:21 mysearchtext headers: [], client: clientip, server, host, request_id=\"MY_REQUEST_ID\" "} The first event contains the message field which is a json string --> we have thousands of this logs The second one are "alerts" and we have just a few of them, the format of the "message" field is plain text. Both contains the value MY_REQUEST_ID which is the field that I have to use to correlate both logs. The output should be a TABLE of ONLY the events with "mysearchtext" (the second event) with some additional fields coming from the second event. The events above are sorted by time (reverse order), the second event is happens just few milliseconds before the first one (basically the second one is just a log message of the same REST request of the first event. The first event is the REST request response sent to the customer)
... View more