I think I understand the essence of the challenge. Data analytics solution all depends on data characteristics. Can you describe data further? For example, the alternative field names, do they app...
See more...
I think I understand the essence of the challenge. Data analytics solution all depends on data characteristics. Can you describe data further? For example, the alternative field names, do they appear in the two different sources? In other words, is there a relationship like this? index=email source=/var/logs/esa_0.log index=cyber source=/varlogs/fe01.log sender, recipient, subject, ... suser, duser, msg, ... Such relationship can improve search by not using too many OR, which usually decreases efficiency. On the other hand, even if such relationships exist, if suser, duser, subject, ... do not always exist in the same event, your search will not satisfy all filters. As @PickleRick says, in that case you will have to sacrifice efficiency and fetch all events then filter. However, you have already clarified that except attachments, sender, recipient, subject, etc., always exist, so do suser, duser, msg, and so on. This means you can take advantage of those always-on fields. Now, to the bottom of the challenge. Yes, you can do that. But you need to change token strategy a little. For this, we will single out the token for attachments from the rest. Just to distinguish this token, I call it attachments_tok, and set up Name-Value pairs (Label-Value in Dashboard Studio parler) like these: Name Value Any * filename1 attachments = filename1 filename2 attachments = filename2 ... Once attachment_tok is set up, reorganize the search like this: (index=email source=/var/logs/esa_0.log ($attachments_tok$) sha256=$hash$
sender="$sender$" recipient="$recipient$" subject="$subject$" message-id="$email_id$" from-header="$reply_add$")
OR (index=cyber source=/varlogs/fe01.log suser="$sender$" duser="$recipient$" msg="$subject$" id="'<$email_id$>'" ReplyAddress="$reply_add$") Hope this helps.