Reporting

Search for different logs lines from the same index and source types

clementros
Path Finder

Hi all,

I have created two different search as below :

index="nbtktfed44971" sourcetype="nbtktfed" "-I" "-c" "-l" "-f" 
| rex "\] (?<JobName>\w+)" 
| rex "-c (?<Channel>\w+)" 
| rex "-f (?<FilePath>.+)\s"

AND

index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
| rex "Readable Payload {\s*(?<ReadablePayload>[^}]+)[^{]+{ (?<Reason>[^}]+)[^{]+{(?<RejectionType>[^}]+)[^{]+{(?<CurrentLine>[^}]+)" 
| rex field=Reason "^(?<Reason_message>.*)." 
| rex field=Reason "Error at (?<Error_location>.*)" 
| rex field=Reason "Error in segment (?<Error_segment>.*)." 
| rex field=Reason "Error in group Group id: (?<Error_group>.*)." 
| rex field=Reason "Converted segments: (?<Converted_segments>.*)" 
| rex field=Reason "Cannot convert CSC from here==>(?<Error_CSC_location>.*)"

I want for example to display Reason message by job name. How can i do that ?

0 Karma
1 Solution

markusspitzli
Communicator

Hello

Just convert your rex commands into Field Extractions. You can do that either with the visual Tool by selecting an Event -> Event Actions -> Extract Fields. Or via the Navigation: Settings -> Fields -> Field extractions.

After the field have been extracted automatically you are able to use such a search:

index="nbtktfed44971" sourcetype="nbtktfed" (("-I" "-c" "-l" "-f") OR ("Malformed message"))

View solution in original post

markusspitzli
Communicator

Hello

Just convert your rex commands into Field Extractions. You can do that either with the visual Tool by selecting an Event -> Event Actions -> Extract Fields. Or via the Navigation: Settings -> Fields -> Field extractions.

After the field have been extracted automatically you are able to use such a search:

index="nbtktfed44971" sourcetype="nbtktfed" (("-I" "-c" "-l" "-f") OR ("Malformed message"))

View solution in original post

clementros
Path Finder

I found a way to do this.

index="nbtktfed44971" sourcetype="nbtktfed" -I -c -l -f 
| join type=outer 
    [ search index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
    | fields Reason] 
| fields JobName, Reason 
| table JobName, Reason
0 Karma

clementros
Path Finder

Thanks, i was able to extract fields from first search, but not the second.

In the second command i use rex on field "reason". When i use field extraction i'm not able to write a correct regex to extract all the fields...

0 Karma

clementros
Path Finder

Now i extracted all my fields using regex.
But i'm still not able to display a fields from one line with a field from another line.

For example, i have the following lines of logs:

timestamp [status] log log log log Name log log log
timestamp [status] log    : log
                   log    : log
                   infob  : valueb

And i want to create visualisations, table, report, statistic of Name by valueb. Is it possible ?

0 Karma

markusspitzli
Communicator

Hey.

It should be possible by using the EXTRACT from props.conf

EXTRACT-<class> = [<regex>|<regex> in <src_field>]
* Use '<regex> in <src_field>' to match the regex against the values of a
specific field.  Otherwise it just matches against _raw (all raw event
data).

Example:

EXTRACT-reason1 =  ^(?<Reason_message>.*). in Reason
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.