Search for different logs lines from the same index and source types

[clementros]

Hi all,

I have created two different search as below :

index="nbtktfed44971" sourcetype="nbtktfed" "-I" "-c" "-l" "-f" 
| rex "\] (?<JobName>\w+)" 
| rex "-c (?<Channel>\w+)" 
| rex "-f (?<FilePath>.+)\s"

AND

index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
| rex "Readable Payload {\s*(?<ReadablePayload>[^}]+)[^{]+{ (?<Reason>[^}]+)[^{]+{(?<RejectionType>[^}]+)[^{]+{(?<CurrentLine>[^}]+)" 
| rex field=Reason "^(?<Reason_message>.*)." 
| rex field=Reason "Error at (?<Error_location>.*)" 
| rex field=Reason "Error in segment (?<Error_segment>.*)." 
| rex field=Reason "Error in group Group id: (?<Error_group>.*)." 
| rex field=Reason "Converted segments: (?<Converted_segments>.*)" 
| rex field=Reason "Cannot convert CSC from here==>(?<Error_CSC_location>.*)"

I want for example to display Reason message by job name. How can i do that ?

[markusspitzli · Accepted solution]

Hello

Just convert your rex commands into Field Extractions. You can do that either with the visual Tool by selecting an Event -> Event Actions -> Extract Fields. Or via the Navigation: Settings -> Fields -> Field extractions.

After the field have been extracted automatically you are able to use such a search:

index="nbtktfed44971" sourcetype="nbtktfed" (("-I" "-c" "-l" "-f") OR ("Malformed message"))

[clementros]

I found a way to do this.

index="nbtktfed44971" sourcetype="nbtktfed" -I -c -l -f 
| join type=outer 
    [ search index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
    | fields Reason] 
| fields JobName, Reason 
| table JobName, Reason

[clementros]

Thanks, i was able to extract fields from first search, but not the second.

In the second command i use rex on field "reason". When i use field extraction i'm not able to write a correct regex to extract all the fields...

[clementros]

Now i extracted all my fields using regex.
But i'm still not able to display a fields from one line with a field from another line.

For example, i have the following lines of logs:

timestamp [status] log log log log Name log log log
timestamp [status] log    : log
                   log    : log
                   infob  : valueb

And i want to create visualisations, table, report, statistic of Name by valueb. Is it possible ?

[markusspitzli]

Hey.

It should be possible by using the EXTRACT from props.conf

EXTRACT-<class> = [<regex>|<regex> in <src_field>]
* Use '<regex> in <src_field>' to match the regex against the values of a
specific field.  Otherwise it just matches against _raw (all raw event
data).

Example:

EXTRACT-reason1 =  ^(?<Reason_message>.*). in Reason