Splunk Search

finding repetition in array of logs

Path Finder

I have a field in my logs that contains an array of string elements. Is there a way to detect for repeating strings and parse it differently within the search query for a dashboard?

array = ["string1", "string2","string1"] regardless of however many repetition

into

array = "extra(string1), string2"

0 Karma
1 Solution

Esteemed Legend

Like this:

| makeresults 
| eval array="[\"string1\", \"string2\", \"string1\"]
[\"string1\", \"string2\", \"string3\"]
[\"string2\", \"string2\", \"string1\"]"
| makemv delim="
" array
| mvexpand array

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| fields array
| rex field=array mode=sed "s/\"//g s/^\[// s/\]$//"
| eval array=split(array, ",")
| rex field=array mode=sed "s/^\s+//"
| streamstats count AS _serial
| mvexpand array
| stats count BY array _serial
| eval array=if((count=1), array, "Extra(" . array . ")")
| stats values(array) AS array BY _serial

Then maybe you would like to add:

| nomv array
| eval array = "[" . array . "]"

View solution in original post

0 Karma

Esteemed Legend

Like this:

| makeresults 
| eval array="[\"string1\", \"string2\", \"string1\"]
[\"string1\", \"string2\", \"string3\"]
[\"string2\", \"string2\", \"string1\"]"
| makemv delim="
" array
| mvexpand array

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| fields array
| rex field=array mode=sed "s/\"//g s/^\[// s/\]$//"
| eval array=split(array, ",")
| rex field=array mode=sed "s/^\s+//"
| streamstats count AS _serial
| mvexpand array
| stats count BY array _serial
| eval array=if((count=1), array, "Extra(" . array . ")")
| stats values(array) AS array BY _serial

Then maybe you would like to add:

| nomv array
| eval array = "[" . array . "]"

View solution in original post

0 Karma

Path Finder

I am trying to add another stats line at the end of line 19, but seems to return 0 results. stats count by requests, array , do you have any suggestion on how to use this as well as using stats within the same query?

0 Karma

Esteemed Legend

The fields array command drops all fields other than array so you need to change line 11 to fields array requests and you also need to change line 19 to stats values(array) AS array first(requests) AS requests BY _serial.

0 Karma

Path Finder

stats values(array) AS array first(requests) AS request by _serial with that, my request field is still empty, I think the by _serial possibly?

0 Karma

Esteemed Legend

OK, like this:

| makeresults 
| eval array="[\"string1\", \"string2\", \"string1\"]
[\"string1\", \"string2\", \"string3\"]
[\"string2\", \"string2\", \"string1\"]"
| makemv delim="
" array
| mvexpand array
| streamstats count AS results
| eval results=results+10

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| fields array results
| rex field=array mode=sed "s/\"//g s/^\[// s/\]$//"
| eval array=split(array, ",")
| rex field=array mode=sed "s/^\s+//"
| streamstats count AS _serial
| mvexpand array
| stats count first(results) AS results BY array _serial
| eval array=if((count=1), array, "Extra(" . array . ")")
| stats values(array) AS array first(results) AS results BY _serial

Path Finder

Hello Woodcock, sorry but to follow up, what happens if I want to remove the repetitions without splitting it, so for example Extra(string1 + string2) for array = ["string1", "string2","string1"]

0 Karma

Legend

Hi exocore123,
are you speaking about a multi values field that you want to put in different events?
if this is your need you should try makemv and mvexpand commands, something like this:

your_search
| makemv multi_values_field
| mvexpand multi_values_field
| ...

Bye.
Giuseppe

0 Karma

Path Finder

It is not really multi-value? More of just a different representation

0 Karma

Legend

Hi exocore123,
sorry but I don't understand your situation, could you share a sample of your logs and the expected output?
Bye.
Giuseppe

0 Karma

Path Finder

I dont care for the number of repetitions, I just want to know if there are repetition of the string within that field called array, then on my tabular dashboard, I want to display it as extra(...) , (rest of the strings). For instance with the example above, say the array within logs are displayed as array=["string1", "string2", "string1"], I want to be able to parse it as "Extra(string1), string2" on my dashboard. Array can have repetitive values, or different values across the board, assume we do not know array's length.

0 Karma

Esteemed Legend

Do you have a field called array right now? Does it have 1 value, 2 values, or 3 values for your example?
What EXACTLY would you like to do with array?

0 Karma

Champion

I think that it is possible to search by string search if you want to search. What exactly do you want to do?
Do you want to know the number of repetitions?

0 Karma