Particular sourcetype not appearing in search

[JyPl4wNYu7GV1uL]

CentOS 7.7.1908, Splunk  v9.1.0.2

I want to get an example event for each sourcetype on each host (excluding one host).  This is my search:

index=prefix_* host!="NotThisHost" sourcetype=*
| sort - _time
| dedup 1 host, sourcetype
| table host, sourcetype, _time, _raw

I run this for the period "yesterday."

This all works, but I'm not getting a result for a particular sourcetype which has "myString" in the sourcetype name.

If I rerun the exact same search with:

index=prefix_* host!="NotThisHost" sourcetype=*myString*
| sort - _time
| dedup 1 host, sourcetype
| table host, sourcetype, _time, _raw

I do get the expected results for the sourcetype with "myString" in the sourcetype name.  So, the only change is to zero in the sourcetype with the "myString" in the sourcetype name.  I'm stumped.  I'm an admin user, so I have all the permissions.  I had another admin user try the same 2 searches and they saw the same behavior where the sourcetype with "myString" in the sourcetype name was not in the first search, but appeared in the second.

Not much help on the web with the Google AI or without.  One suggestion was to see if your sourcetype had a hidden trailing space (it does not).

I've tried with and without the "sort - _time"; no change.  I've tried with various integers for dedup; no change.

It's a pretty simple search, so I'm stumped.

Anybody got any bright ideas?

This feels like a bug to me.  There'd be no way to recreate enough for a splunk issue ticket; and, I wouldn't bother anyway unless I could reproduce on 10.0.1 (when it comes out).

[JyPl4wNYu7GV1uL]

@livehybrid Boom!  Learn something new every day.

[JyPl4wNYu7GV1uL]

Ok, solved it with this search:

index=prefix_* host!="NotThisHost"
| dedup 1 host, sourcetype sortby - _time
| table host, sourcetype, source, _raw

So, for some reason piping the "sort" to the "dedup" broke the search.  I found this cuz the search dropdown suggested using "dedup ... sortby ..." over "sort | dedup."  

So my question is "WTF??!!"  The interwebs AI claims the "dedup ... sortby ..." is more efficient than "sort | dedup."  Perhaps the search dropdown should warn "Don't do that, I'm going to break your search results."

So this is certainly a bug, so be forewarned.

I'd give myself karma if I could. haha

[livehybrid · Accepted solution]

Hi @JyPl4wNYu7GV1uL 

Your sort command introduced a limit of 10,000 events which could potentially have meant that you had no data for the sourcetype  containing "myString"

Removing the sort means that your results are not limited to 10,000 events.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

Ok

1. Don't believe what LLMs hallucinate.

2. I don't know what dropdown you are talking about. The dropdown in the search dialog doesn't suggest anything; it just tells you what arguments a command accepts.

3. By default results are sorted in reverse chronological order so unless you reordered your results sorting - _time does completely nothing (ok, to be absolutely clear, I'm not sure about stability of sorting).

4. If you don't get results for a particular combination of fields that looks like a bug. Unless you're hitting some limits but I suppose you shouldn't have that many host/sourcetype pairs to dedup.