How to find common values between two searches?

[shyam_v]

I have two queries from the same set of index and app names using different search terms from which I am extracting a set of fields as below:

Query1:

index=A cf_app_name=B  "search string 1"
| rex field=_raw "(?ms)Id: (?P<Id>[^,]+), service: (?P<service>[^,]+), serial: (?P<serial>[^,]+), Type: (?P<Type>[a-zA-Z-]+)"
| table serial Id Type service _time

Query 2:

index=A cf_app_name=B "search string 2"
| rex field=_raw "(?ms)serial\\W+(?P<serial>[^\\\\]+)\\W+\\w+\\W+(?P<Type>[^\\\\]+)\\W+\\w+\\W+\\w+\\W+(?P<Id>[a-zA-Z]+-\\d+-\\d+)\\W+\\w+\\W+(?P<gtw>[^\\\\]+)\\W+\\w+\\W+(?P<service>[^\\\\]+)"
| table serial Type Id service _time

My requirement is to list all the values in Query1 and then show a Y/N flag if there is a match in Query2 based on the field 'Id'. Tried join and append, but do not seem to be getting the right results, any suggestions will be appreciated.

[bowesmana]

Depending on the data, you could do something like

index=A cf_app_name=B ((search string 1) OR (search string 2)
| rex statement 1
| rex statement 2
| table serial Type Id service _time
| stats count values(*) as * by Id

| where count > 1
OR
| where condition_that_satisfies_the test_for_your_use_case

The where clause would be dependent on what your data looks like. For example if each of the data sets is from a different sourcetype, then you could do

...
| table serial Type Id service _time sourcetype
| stats count values(*) as * by Id
| where count>1 AND isnotnull(mvfind(sourcetype, "sourcetype_2"))

The two rex statements would appear not to conflict with each other and other than one rex will always fail for one of the data sets, it should work OK

Hope this helps

[PickleRick]

What I'd suggest is create two separate extractions (even better, but you won't do it on already indexed data, during ingestion cast the sourcetype to different "subtypes") and just work on statsing extracted field values.