Reporting

Search for different logs lines from the same index and source types

clementros
Path Finder

Hi all,

I have created two different search as below :

index="nbtktfed44971" sourcetype="nbtktfed" "-I" "-c" "-l" "-f" 
| rex "\] (?<JobName>\w+)" 
| rex "-c (?<Channel>\w+)" 
| rex "-f (?<FilePath>.+)\s"

AND

index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
| rex "Readable Payload {\s*(?<ReadablePayload>[^}]+)[^{]+{ (?<Reason>[^}]+)[^{]+{(?<RejectionType>[^}]+)[^{]+{(?<CurrentLine>[^}]+)" 
| rex field=Reason "^(?<Reason_message>.*)." 
| rex field=Reason "Error at (?<Error_location>.*)" 
| rex field=Reason "Error in segment (?<Error_segment>.*)." 
| rex field=Reason "Error in group Group id: (?<Error_group>.*)." 
| rex field=Reason "Converted segments: (?<Converted_segments>.*)" 
| rex field=Reason "Cannot convert CSC from here==>(?<Error_CSC_location>.*)"

I want for example to display Reason message by job name. How can i do that ?

0 Karma
1 Solution

markusspitzli
Communicator

Hello

Just convert your rex commands into Field Extractions. You can do that either with the visual Tool by selecting an Event -> Event Actions -> Extract Fields. Or via the Navigation: Settings -> Fields -> Field extractions.

After the field have been extracted automatically you are able to use such a search:

index="nbtktfed44971" sourcetype="nbtktfed" (("-I" "-c" "-l" "-f") OR ("Malformed message"))

View solution in original post

markusspitzli
Communicator

Hello

Just convert your rex commands into Field Extractions. You can do that either with the visual Tool by selecting an Event -> Event Actions -> Extract Fields. Or via the Navigation: Settings -> Fields -> Field extractions.

After the field have been extracted automatically you are able to use such a search:

index="nbtktfed44971" sourcetype="nbtktfed" (("-I" "-c" "-l" "-f") OR ("Malformed message"))

clementros
Path Finder

I found a way to do this.

index="nbtktfed44971" sourcetype="nbtktfed" -I -c -l -f 
| join type=outer 
    [ search index="nbtktfed44971" sourcetype="nbtktfed" "Malformed message" 
    | fields Reason] 
| fields JobName, Reason 
| table JobName, Reason
0 Karma

clementros
Path Finder

Thanks, i was able to extract fields from first search, but not the second.

In the second command i use rex on field "reason". When i use field extraction i'm not able to write a correct regex to extract all the fields...

0 Karma

clementros
Path Finder

Now i extracted all my fields using regex.
But i'm still not able to display a fields from one line with a field from another line.

For example, i have the following lines of logs:

timestamp [status] log log log log Name log log log
timestamp [status] log    : log
                   log    : log
                   infob  : valueb

And i want to create visualisations, table, report, statistic of Name by valueb. Is it possible ?

0 Karma

markusspitzli
Communicator

Hey.

It should be possible by using the EXTRACT from props.conf

EXTRACT-<class> = [<regex>|<regex> in <src_field>]
* Use '<regex> in <src_field>' to match the regex against the values of a
specific field.  Otherwise it just matches against _raw (all raw event
data).

Example:

EXTRACT-reason1 =  ^(?<Reason_message>.*). in Reason
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...