Splunk Search

Is it possible for a search string to pick up instances where the last [say] 3 logs are identical?

Mick_OBrien
Path Finder

Hi All

We have a couple of jobs that occasionally loop around same code returning same message/log - is it possible for a search string to pick up instances where the last [say] 3 logs are identical?

Kinds regards

Mick

 

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

Assuming the field you are looking at is called message, you could try something like this

| streamstats count reset_on_change=t by message
| where count>2

View solution in original post

Mick_OBrien
Path Finder

Issue was 'sort' limited to 10,000 rows so replacing with 'sort 0' and I see what I need to see [no missing jobs]

0 Karma

Mick_OBrien
Path Finder

Hi All

Issue finally resolved.  This is what I was told - 'sort' command limit is 10,000 and issue was the rather large number of log being returned and the resolution was to replace 'sort' with 'sort 0' which returns all logs [and now I see everything I need to]

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Assuming the field you are looking at is called message, you could try something like this

| streamstats count reset_on_change=t by message
| where count>2

Mick_OBrien
Path Finder

My index has logs for multiple Robot jobs so I added a search before the suggested string....

index=ee_rpa_uipath_platform_* AND OrganizationUnitID IN ($folder$) | sort OrganizationUnitID, RobotName, _time, Message | streamstats count reset_on_change=true by Message | where count > 2 | table OrganizationUnitID, User, RobotName, ProcessName, MachineName, _time, Message | sort -_time

...but now what I am finding is that ONLY one Robot has its logs being displayed once search complete i.e. whilst search is ongoing other logs for other Robots are displayed in panel but then disappear once search finishes.  Any ideas on why these logs for other Robots are removed from search?

 

 

I put the suggested search string in my searh

0 Karma

yuanliu
SplunkTrust
SplunkTrust

Transaction may be your friend.

index=ee_rpa_uipath_platform_* AND OrganizationUnitID IN ($folder$)
```| sort OrganizationUnitID, RobotName, _time, Message```
| eval robotmessage = OrganizationUnitID . ":" . RobotName . ":" . Message
| transaction robotmessage maxevents=3
| where closed_txn=true AND eventcount > 2

About the commented-out sort: because your end goal will always be some kind of tables grouped by OrganizationUnitID and RobotName, there is no point to sort against these two early; if your events come in "naturally", most likely you do not need to sort by _time.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

So, if your filter criteria is also by org and robot name, then you can add those into the "BY" clause in the streamstats.

You may not need to use sort if you are also splitting by org+robot as the reset_on_change will reset only when org+robot+message changes.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

There are probably being removed by the where command i.e. consecutive messages are not the same, and you are left with occurrences which appear 3 or more times (as requested).

0 Karma
Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...