Splunk Search

How to compare search result of logs from production server to QA search result logs?

raghavendrasred
New Member

I want compare the 2 search result error logs and show the result.
I want to compare based on "Error" log of 2 search result.

One search result from prod:

*NullPointerException*  index=prod | search log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | fieldformat FirstOccurence=strftime(FirstOccurence,"%m/%d/%y %H:%M:%S") | fieldformat LastOccurence =strftime(LastOccurence,"%m/%d/%y %H:%M:%S") | eval Error = static_msg | Table Error  count FirstOccurence LastOccurence | sort –count

Search result from QA:

*NullPointerException*  host=$host$ | search log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | fieldformat FirstOccurence=strftime(FirstOccurence,"%m/%d/%y %H:%M:%S") | fieldformat LastOccurence =strftime(LastOccurence,"%m/%d/%y %H:%M:%S") | eval Error = static_msg | Table Error  count FirstOccurence LastOccurence | sort –count

I tried diff like eval diff = prod - qa it's showing difference of count or number of entries displayed, but I am looking for actual log differences.

0 Karma
1 Solution

somesoni2
Revered Legend

Try something like this

| set diff [search *NullPointerException*  index=prod log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | convert ctime(*Occurrence) timeformat="%m/%d/%y %H:%M:%S" | rename  static_msg as Error| Table Error  count FirstOccurence LastOccurence] [search *NullPointerException* host=$host$ log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | convert ctime(*Occurrence) timeformat="%m/%d/%y %H:%M:%S" | rename  static_msg as Error| Table Error  count FirstOccurence LastOccurence ] | sort –count

View solution in original post

0 Karma

somesoni2
Revered Legend

Try something like this

| set diff [search *NullPointerException*  index=prod log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | convert ctime(*Occurrence) timeformat="%m/%d/%y %H:%M:%S" | rename  static_msg as Error| Table Error  count FirstOccurence LastOccurence] [search *NullPointerException* host=$host$ log_level="ERROR" OR log_severity="Error" OR status>399 | stats earliest(_time) as FirstOccurence latest(_time) as LastOccurence count by static_msg | convert ctime(*Occurrence) timeformat="%m/%d/%y %H:%M:%S" | rename  static_msg as Error| Table Error  count FirstOccurence LastOccurence ] | sort –count
0 Karma

raghavendrasred
New Member

it not worked.

I tried another way..

NullPointerException index=prod | search log_level="ERROR" OR log_severity="Error" OR status>399 | stats count by static_msg | eval prod = static_msg | appendcols [search NullPointerException host=$host$ | search log_level="ERROR" OR log_severity="Error" OR status>399 | stats count by static_msg | eval qa = static_msg ] | eval err=if(prod == qa, "OK", "Error")

it should work like vlookup, but it is not working..

0 Karma

raghavendrasred
New Member

Thanks. now i got the result in single Error field.
in result we have both the records of production and QA in single field as Error. The Error contains the 2 records with same message(like 2 entry with message "Failed to process ospp message."). Is it possible to highlight common records present in result.

0 Karma

raghavendrasred
New Member

Please some one answer the above request.

0 Karma
Get Updates on the Splunk Community!

Observability Unlocked: Kubernetes Monitoring with Splunk Observability Cloud

 Ready to master Kubernetes and cloud monitoring like the pros?Join Splunk’s Growth Engineering team for an ...

Update Your SOAR Apps for Python 3.13: What Community Developers Need to Know

To Community SOAR App Developers - we're reaching out with an important update regarding Python 3.9's ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...