Splunk Search

multiple occurences of one field - from rex to field-extraction

Marco204
Engager

Hi there,

so I have a search that results contains multiple occurences of one field.

My current solution is using rex together with max_match=0 in order to get this:

 

index="dev_logs" pod::apollo* some.url.com/api statusCode | rex field=_raw max_match=0 "\"statusCode\":(?<ApolloStatusCode>\d+)"

 

 Well, right now I want an alert for the case that status is neither 200 nor 204.

So I played around with this:

 

| search 200 OR search 204

| search NOT 200 AND search NOT 204

| search NOT [search 200 OR search 204]

 

To be honest neither works 🤔

 

Right now I think that the sub-search is the problem, and a solution could be to use field-extraction.

So I used the field extraction wizard and changed the generated regex to this afterwards:

 

"statusCode":(?<ApolloStatusCode>\d+)

 

But this only returns the first occurence - but I need them all.

 

With field transformation I didn't make any progress, and editing some conf files are out of scope...

 

Thanks for any help,

Marco

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try this search for non-200 result codes.

index="dev_logs" pod::apollo* some.url.com/api statusCode 
| rex field=_raw max_match=0 "\"statusCode\":(?<ApolloStatusCode>\d+)"
| where isnotnull(mvfind(ApolloStatusCode, "[3-9]\d{2}")
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

Marco204
Engager

Thanks a lot for your answer. In between I was able to combine field transformation and field extraction, so right now I have the variable named MARCO. In a search over the last 30 day there occur 3 values: 200, 204 and 400. But using your where-clause returns no results at all. And the 2nd param of the mvfind function is a regex, so it has to be quoted.

Here some results:

| where isnull(mvfind(MARCO,"204"))

return the 200 and 400.

| where isnull(mvfind(MARCO,"400"))

return the 200 and 204.

| where isnull(mvfind(MARCO,"200"))

return 0 events. 🙄

So now I need to explain the results a normal search without where-clause.

  • 200,204,200
  • 200
  • 200,204
  • 200,400
  • 200,200,204

So somehow there is always a 200, and removing those results in 0 events.

Do you know another way of filtering?

Currently I want the (200,400) and maybe in the future I will get something like (200,500) or (200,404) which I also want...

Best regards,
Marco

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this search for non-200 result codes.

index="dev_logs" pod::apollo* some.url.com/api statusCode 
| rex field=_raw max_match=0 "\"statusCode\":(?<ApolloStatusCode>\d+)"
| where isnotnull(mvfind(ApolloStatusCode, "[3-9]\d{2}")
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Use the field name in your search.

index="dev_logs" pod::apollo* some.url.com/api statusCode 
| rex field=_raw max_match=0 "\"statusCode\":(?<ApolloStatusCode>\d+)"
| search NOT (ApolloStatusCode=200 OR ApolloStatusCode=204)

That's the normal case, but you're using max_match=0, which produces a multi-value field and that's not so normal.

index="dev_logs" pod::apollo* some.url.com/api statusCode 
| rex field=_raw max_match=0 "\"statusCode\":(?<ApolloStatusCode>\d+)"
| where (isnull(mvfind(ApolloStatusCode, 200)) AND isnull(mvfind( ApolloStatusCode,204)))
---
If this reply helps you, an upvote would be appreciated.
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!