finding repetition in array of logs

[exocore123]

I have a field in my logs that contains an array of string elements. Is there a way to detect for repeating strings and parse it differently within the search query for a dashboard?

array = ["string1", "string2","string1"] regardless of however many repetition

into

array = "extra(string1), string2"

[woodcock · Accepted solution]

Like this:

| makeresults 
| eval array="[\"string1\", \"string2\", \"string1\"]
[\"string1\", \"string2\", \"string3\"]
[\"string2\", \"string2\", \"string1\"]"
| makemv delim="
" array
| mvexpand array

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| fields array
| rex field=array mode=sed "s/\"//g s/^\[// s/\]$//"
| eval array=split(array, ",")
| rex field=array mode=sed "s/^\s+//"
| streamstats count AS _serial
| mvexpand array
| stats count BY array _serial
| eval array=if((count=1), array, "Extra(" . array . ")")
| stats values(array) AS array BY _serial

Then maybe you would like to add:

| nomv array
| eval array = "[" . array . "]"

[exocore123]

I am trying to add another stats line at the end of line 19, but seems to return 0 results. stats count by requests, array , do you have any suggestion on how to use this as well as using stats within the same query?

[woodcock]

The fields array command drops all fields other than array so you need to change line 11 to fields array requests and you also need to change line 19 to stats values(array) AS array first(requests) AS requests BY _serial.

[exocore123]

stats values(array) AS array first(requests) AS request by _serial with that, my request field is still empty, I think the by _serial possibly?

[woodcock]

OK, like this:

| makeresults 
| eval array="[\"string1\", \"string2\", \"string1\"]
[\"string1\", \"string2\", \"string3\"]
[\"string2\", \"string2\", \"string1\"]"
| makemv delim="
" array
| mvexpand array
| streamstats count AS results
| eval results=results+10

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| fields array results
| rex field=array mode=sed "s/\"//g s/^\[// s/\]$//"
| eval array=split(array, ",")
| rex field=array mode=sed "s/^\s+//"
| streamstats count AS _serial
| mvexpand array
| stats count first(results) AS results BY array _serial
| eval array=if((count=1), array, "Extra(" . array . ")")
| stats values(array) AS array first(results) AS results BY _serial

[exocore123]

Hello Woodcock, sorry but to follow up, what happens if I want to remove the repetitions without splitting it, so for example Extra(string1 + string2) for array = ["string1", "string2","string1"]

[gcusello]

Hi exocore123,
are you speaking about a multi values field that you want to put in different events?
if this is your need you should try makemv and mvexpand commands, something like this:

your_search
| makemv multi_values_field
| mvexpand multi_values_field
| ...

Bye.
Giuseppe

[exocore123]

It is not really multi-value? More of just a different representation

[gcusello]

Hi exocore123,
sorry but I don't understand your situation, could you share a sample of your logs and the expected output?
Bye.
Giuseppe

[exocore123]

I dont care for the number of repetitions, I just want to know if there are repetition of the string within that field called array, then on my tabular dashboard, I want to display it as extra(...) , (rest of the strings). For instance with the example above, say the array within logs are displayed as array=["string1", "string2", "string1"], I want to be able to parse it as "Extra(string1), string2" on my dashboard. Array can have repetitive values, or different values across the board, assume we do not know array's length.

[woodcock]

Do you have a field called array right now? Does it have 1 value, 2 values, or 3 values for your example?
What EXACTLY would you like to do with array?

[HiroshiSatoh]

I think that it is possible to search by string search if you want to search. What exactly do you want to do?
Do you want to know the number of repetitions?