Splunk Search

Why is my Search not returning results?

jymmitch
Path Finder

Here's the text string from the log I'm searching:

store license for Store 123456
2022-04-07 19:17:44,360 ERROR path not found

 

Here's my splunk search:

index=* host="storelog*" "store license for " |rex field=_raw "Store\s123456\n\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s(?P<errortext>.*)path" | stats count by errortext

 

Why am I getting the following when I search?

No results found.

Labels (2)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

It sounds kinda strange since the single/multiline behaviour is repeatable. And while matching regex to a string Splunk doesn't care whether the event contains timestamp, social securiyy number, your shoe size or whatever other data you can have. It's just text - if it fits, it fits.

OK. It's interesting because sometimes indeed (as in my example) Splunk treats regexes as single line, sometimes as multiline.

Anyway, your initial data sample together with your regex works for me.

PickleRick_0-1649406153881.png

BTW, your "no results found" might mean that no events matched your search, not that the rex command extracted the field wrongly or not at all.

See the "Inspect Job" and see if you got anything from the initial search.

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

I won't give you a solution because I tend to avoid such events 😉 but the main problem is most probably the fact that your event is multiline and by default regex matches only over single line.

Try for yourself:

| makeresults 
| eval a="abcd
fghi"
| rex field=a "(?<b>.*)"

You will only get the first line as a results.

You have to fiddle with the single/multiline regex modifiers to make your rex work across the newline. Might work with the (?s) modifier at the beginning of your expression.

0 Karma

jymmitch
Path Finder

I have other searches that work where the regex matches over two lines. The only difference between those and this one is the fact that this one includes a datestamp  (the others don't).

0 Karma

PickleRick
SplunkTrust
SplunkTrust

It sounds kinda strange since the single/multiline behaviour is repeatable. And while matching regex to a string Splunk doesn't care whether the event contains timestamp, social securiyy number, your shoe size or whatever other data you can have. It's just text - if it fits, it fits.

OK. It's interesting because sometimes indeed (as in my example) Splunk treats regexes as single line, sometimes as multiline.

Anyway, your initial data sample together with your regex works for me.

PickleRick_0-1649406153881.png

BTW, your "no results found" might mean that no events matched your search, not that the rex command extracted the field wrongly or not at all.

See the "Inspect Job" and see if you got anything from the initial search.

jymmitch
Path Finder

That worked!

0 Karma

newrose
Explorer

Is the  errortext  field created by the rex command being populated correctly? One possible explanation could be that the regex is not matching the text in the events.

You can execute the search again without the stats command and see if the  errortext  field is present for the desired events.

0 Karma

jymmitch
Path Finder

I took off the stats command and reran the search, but the errortext field is NOT in the events.

0 Karma

mayurr98
Super Champion

could you please share more sample events as I do not see any error in your search.

I have tried in this run anywhere search

 

 

| makeresults 
| eval _raw="store license for Store 123456
2022-04-07 19:17:44,360 ERROR path not found" 
| rex field=_raw "Store\s123456\n\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s(?P<errortext>.*)path" 
| stats count by errortext

 

From the regex I could see that you are searching  for "Store 123456", please add that in the main search instead of in regex.

index=* host="storelog*" "store license for Store 123456"

Also is it a multiline event? that means is timestamp is on new line in raw logs or its just one line?

 

0 Karma

newrose
Explorer

The regex will not work considering the sample events you provided.

If the event starts with the timestamp and is single lined maybe you should try this regex:

| rex field=_raw "\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s(?P<errortext>.*)path"

 

0 Karma

jymmitch
Path Finder

There is a timestamp preceding the text in my main search, but there is also a timestamp that follows it. I'm only searching from the main search text forward:

raw log text:
store license for Store 123456
2022-04-07 19:17:44,360 ERROR path not found

 

index=* host="storelog*" "store license for " |rex field=_raw "Store\s123456\n\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s(?P<errortext>.*)path" | stats count by errortext

Based on my search, I would think splunk should recognize the "store license for " text, then recognize the single whitespace, then recognize the "Store\s123456" text in the regex, then recognize the newline (\n), then recognize the date/timestamp, and finally set errortext field to ERROR.

0 Karma

newrose
Explorer

If I correctly understood your logic, it isn't exactly how Splunk is interpreting your query, although the selected events will probably be the same.

Your base search,

index=* host="storelog*" "store license for "

will extract all events which have the "store license for " string, including the single whitespace.

Then it will run the rex over all the selected commands, and the regex will try the match starting from the beginning of the event, not from where you stopped from the previous command.

To overcome the newline issue, check if it is possible given your dataset to run something like this:

index=* host="storelog*" "store license for Store 123456" 
| rex field=_raw "\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s(?P<errortext>.*)path"
| stats count by errortext

 The "Store 123456" was moved to the main search, and the regex will try to match starting from the timestamp.

Also you should probably look over this data input parameters, as the raw events doesn't look to have the right boundaries from what you showed here.

0 Karma

jymmitch
Path Finder

I added "Store 123456" to main search and removed it from regex, but still get "no results found."

0 Karma

jymmitch
Path Finder

Timestamp is on a new line

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...