topic How to Identify a repeated set up messages [for looping process]? in Splunk Search

How to Identify a repeated set up messages [for looping process]?

Mick_OBrien — Fri, 22 Sep 2023 18:17:29 GMT

We have a job that occasionally loops around the same code spewing out same set of messages [2 different messages from same job] - is it possible to identify processes where the last 2 messages match the previous 2 messages...

message1

message2

message1 <-- starts repeating/looping here

message2

message1

message2

message1

message2

Any help appreciated.

Mick

Re: Identify a repeated set up messages [for looping process]

ITWhisperer — Thu, 21 Sep 2023 07:28:37 GMT

It is possible but it depends on your messages.

Are the messages from the process unique apart from when they are repeated?

Can you correlate messages from the same instance of the process without confusing them with messages from another instance of the process?

Are the loops any bigger or smaller than two messages?

What do you need to be kept in the report, e.g. all messages, just the process id?, just the time of the first duplicated message, just the fact that a process has looped?

Re: Identify a repeated set up messages [for looping process]

Mick_OBrien — Thu, 21 Sep 2023 07:43:37 GMT

The messages are valid but once starting to loop indicates issue with process - messages can be from different processes but I am only interested in messages repeating on same process.

Re: Identify a repeated set up messages [for looping process]

ITWhisperer — Thu, 21 Sep 2023 07:47:54 GMT

| eventstats count as repeats by process message | where repeats > 1

Re: Identify a repeated set up messages [for looping process]

Mick_OBrien — Thu, 21 Sep 2023 07:55:45 GMT

Sorry - but how does this pick up a set of messages on the same process repeating?

Re: Identify a repeated set up messages [for looping process]

ITWhisperer — Thu, 21 Sep 2023 08:11:19 GMT

| makeresults format=csv data="process,message A,message 0 B,message 0 A,message 1 B,message 1 A,message 2 B,message 2 A,message 1 B,message 3 A,message 2 A,message 1 A,message 2" | eventstats count as repeats by process message | where repeats > 1

As you can see, only messages that are repeated are shown

Re: Identify a repeated set of messages [for looping process]

Mick_OBrien — Fri, 22 Sep 2023 16:33:50 GMT

I ran from search prompt bar but nothing was returned for result set - is there a specific way to use 'makeresults' syntax?

Re: Identify a repeated set of messages [for looping process]

ITWhisperer — Fri, 22 Sep 2023 18:38:10 GMT

Which version of Splunk are you using (the makeresults command changed in version 9).

The makeresults is only to create some example data to show you that the commands work.

Re: Identify a repeated set of messages [for looping process]

Mick_OBrien — Sun, 24 Sep 2023 16:05:04 GMT

Splunk Enterprise

Version:8.2.7.1

Re: Identify a repeated set of messages [for looping process]

Mick_OBrien — Sun, 24 Sep 2023 16:10:05 GMT

See below - no output from search string...

Re: Identify a repeated set of messages [for looping process]

ITWhisperer — Sun, 24 Sep 2023 16:19:31 GMT

Try this pre-9 syntax

| makeresults | eval _raw="process,message A,message 0 B,message 0 A,message 1 B,message 1 A,message 2 B,message 2 A,message 1 B,message 3 A,message 2 A,message 1 A,message 2" | multikv forceheader=1 | table process,message | eventstats count as repeats by process message | where repeats > 1

Re: Identify a repeated set of messages [for looping process]

PickleRick — Sun, 24 Sep 2023 16:28:19 GMT

Ha! Good to know about the makeresults. I didn't know that.

Re: Identify a repeated set of messages [for looping process]

Mick_OBrien — Sun, 24 Sep 2023 17:24:35 GMT

Thanks - the pre-9 syntax works but multiple instances of the same repeated log are displayed....

Is there a way to limit to one set of logs?