Clarification on the search using OR

[Jananee_iNautix]

Hi,

There are logs coming from two sources (xxx.success, yyy.error) into one index.Fields are to be extracted from the events of these three sources. The information to be extracted are specific to each of the files. But one search query has to be framed for success and error events from the respective files. There are no overlapping information/values between two files. However search query should not be two queries rather one.

xxx.success :

SW:HandlefileTransfer
.
.
.
/SW:HandlefileTransfer

yyy.error :

.
SW:GblStatus
.
.
.SW:TextError occurred..../SW:Text

            .
           <SW:GblStatus>

.
.

Solution 1:

We framed the search query like

index =fxr |source = "xxx.success" OR source = "yyy.error" |(Extraction of fields using rex command)

If the search is to be performed this way, to get successful events the query will unnecessarily search in error events and to get error events the query will unnecessarily search in successful events thereby doing two unnecessary searches taking more time.

Can you tell an alternative way or suggest some other way to handle this scenario?

Solution 2:

Transformation of the logs after combining the logs from two sources into a single file and adding additional tag to the events so as to differentiate the sources from each other like

SW:HandlefileTransfer
.
.
.
..
/SW:HandlefileTransfer


SW:GblStatus
.
.
.SW:TextError occurred..../SW:Text

            .
           <SW:GblStatus>
            </Error>

As extra tag is added around the incoming data, we can eliminate the unnecessary search (as mentioned above) and thus reducing the time. Please advise.

[Ayn]

First of all your first search is incorrect, but I'm assuming it's pseudo-code?

Could you please explain what you mean by that it's performing "unnecessary" searches? If you want to find events from xxx.success and yyy.error there's obviously no way around that you'd have to search xxx.success and yyy.error for that. Where does the "unnecessary" part come in?

If you mean that it's somehow more resource-consuming to search for two different sources instead of just one, that isn't the case. With HUGE numbers of search terms this can be the case but we're talking about thousands of search terms before you might be seeing a performance impact.