Error when searching log text that contains a line break

[jymmitch]

Sample text from a log that I'm searching:

"store license for Store 123456
2022-03-27 02:01:59,649 [XNIO-2 task-3] ERROR"

I'm trying to search for, and return, a store number that's associated with a particular error. The following search successfully returns the store number (and count):

index=* host="log*" "store license for" | rex field=_raw "Store\s(?P<storenumber>.*)" | stats count by storenumber

But when I try to search for the storenumber along with error string that follows it, I get "no results found." Here's the search i'm trying:

index=* host="log*" "store license for" | rex field=_raw "Store\s(?P<storenumber>.*)[\r\n]+2022\-03\-27\s02:01:59,649\s\[XNIO-2\stask-3\]\sERROR" | stats count by storenumber

Splunk doesn't seem to like the newline character. I've tried \n and [r\n\] and others, but all with the same incorrect results.

[jymmitch · Accepted solution]

I'll give it a try today, thanks!

[mayurr98]

Hi there are two ways to do this.

1st way :

put the specific error in the main search and you will find the all the storenumber counts with that error.

index=* host="log*" "store license for" "Error" | rex field=_raw "Store\s(?P<storenumber>.*)" | stats count by storenumber

2nd way:

Extract the error from the raw data and display/filter in the statistics.

index=* host="log*" "store license for" | rex field=_raw "Store\s*(?P<storenumber>\d+)\n*.*ERROR(?<Error>.*)" | stats count by storenumber Error

let me know if this helps!

[jymmitch]

When I try something like that where all the data is on one line in the logs, I get results. When the data is on separate lines in the logs, I get no results in my search.

[jymmitch]

That "should" have worked, but didn't. I still get "no results found." For some reason, it doesn't seem to recognize (or acknowledge) the \n.

[jymmitch]

I think a slightly tweaked version of this will work.

[jymmitch]

I think a slightly tweaked version of this will work.

[jymmitch]

Here are two examples from logs...

store license for Store 123456
2022-03-27 02:01:59,649 [XNIO-2 task-3] ERROR hostnane is null

store license for Store 234567
2022-03-27 00:02:22,566 [XNIO-2 task-7] INFO com.

I want to find only store numbers that are followed by the error text 

[jymmitch]

Not sure, but the line break in the log seems to be messing me up:

"store license for Store 123456
2022-03-27 02:01:59,649 [XNIO-2 task-3] ERROR"

[jymmitch]

I'm trying to find every occurrence of the store number in the logs that is followed by a specific error text. That "ERROR" in my sample is just the first word in the error string. There are other occurrences of that store number in the logs, but I want to find only those that are followed by the specific error text. I know that I'll also have to deal with the date/time stamp, but for now I'm just trying to figure out to write the search query to find that hardcoded value.

[somesoni2]

You can put the specific error message in your base search as filter

index=* host="log*" "store license for" "<your hard coded error message>" | rex field=_raw "Store\s(?P<storenumber>.*)" | stats count by storenumber

Example:

index=* host="log*" "store license for" "ERROR hostnane is null" | rex field=_raw "Store\s(?P<storenumber>.*)" | stats count by storenumber

[somesoni2]

Are you trying to  find count by storenumber and error,  like this?

index=* host="log*" "store license for" | rex field=_raw "Store\s(?P<storenumber>.*)" | rex "\](?<Error>.+)"| stats count by storenumber Error