I have 2 logs: an error log and a success log. When an item fails (error log), it is retried. I would like to filter out the items that succeed on retry and only display items that are still failing - In other words, items that only appear in the error log.
Sample search:
index=fail_log "Processing failed" | dedup event_id | transaction event_id | rex "\"itemId\":(?P<itemIdInt>\d+)" | fields itemIdInt | search NOT [search index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt ] | stats count
The reason for the substring is that in the success log, the items show up as ID- and in the fail log, they are just , so we need to just pull out the actual digits.
This seems to work for most things. We have 20,000+ successes and on the order of 10-20 failures. However, only 1 or 2 of those failures have not been successful, and a simple search of:
index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt | search itemIdInt=<itemId_from_fail_log>
will return results, indicating that itemId has succeeded on rerun.
Is there a more formal way of removing items from result A (failures) that appear in result B (successes)
... View more