Splunk Search

Missing Results / Random Results on searches after updating from 8.1.3 to 8.2.9

FGAnders
Explorer

Hi,

 

Has anyone an idea what could be the reason why before an update was able to run a query and get correct results but after updating to  8.2.9 getting random results. The data is in the event data, I can find the specific data if I specify for one specific but if I run the query I can get 1 result, 15 results, 42 results so on. Running the same query within the same timeframe. 

We have over 1500 Indexes and it seems to only be issue with one specific index. It does seem odd that when running it the data is there if I use specific user=123 instead of using user=* but then I would only get results for user 123. I tried even adding user=123 OR user=* does not change anything just random results.

Could it be something that needs to be cleared or something?

Has anyone seen this before?

 

index=ABC operation=Paymentcompleted PAYMENT_METHOD=* user=* firstName=* lastName=* jurisdiction=UK AMOUNT=* country=GB | dedup user
| eval NameofPayer = FIRST_NAME." ".LAST_NAME
| eval NameofCust = firstName." ".lastName
| eval NameofCust=upper(NameofCust)
| eval NameofPayer=upper(NameofPayer)
| where NOT match(NameofPayer,NameofCust)
| stats list(NameofPayer) as NameofPayer, list(NameofCust) as NameofCust by user | fieldformat Time = strftime(Time, "%Y-%m-%d %H:%M:%S")

 


Running Stats list, values or tables does not make a difference to the random results.

FGAnders_0-1689155667249.pngFGAnders_1-1689155705023.pngFGAnders_2-1689155821193.png

While this should be over 140 results.

Thank you in advanced,

Labels (3)
Tags (1)
0 Karma

meetmshah
Contributor

I suspect "dedup" is the culprit, can you please remove the same and fit "stats dc()" or "stats count" or so and validate the results?

PickleRick
SplunkTrust
SplunkTrust

Yeah, if I see "dedup" and "random" in the same sentence, that's the first thing I'd suspect.

But if it's not the case - it would be interesting to see job logs.

0 Karma

FGAnders
Explorer

Hi,

 

Thank you for the suggestion.
I already tried without dedup and this does not change anything regrading random results. This was working without any issues before update was done. 

I did find an older post where there seems to have been similar issues. 

https://community.splunk.com/t5/Getting-Data-In/Missing-data-Splunk-is-showing-random-gaps-in-the-in...

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...