Issue with Searching by Sourcetypes in Splunk

[sigma]

Hi all,
I'm collecting iLO logs in Splunk and have set up configurations on a Heavy Forwarder (HF). Logs are correctly indexed in the ilo index with sourcetypes ilo_log and ilo_error, but I'm facing an issue with search results.

When I run index=ilo | stats count by sourcetype, it correctly shows the count for ilo_log and ilo_error. Also, index=ilo | spath | table _raw sourcetype confirms logs are indexed with the correct sourcetype.

However, when I search directly with index=ilo sourcetype=ilo_log, index=ilo sourcetype=ilo_error, or even index=ilo ilo_log, I get zero results. Strangely, sourcetype!=ilo_error returns all ilo_log events and the same for ilo_error.

props.conf:

[source::udp:5000] 
TRANSFORMS-set_sourcetype = set_ilo_error
SHOULD_LINEMERGE = false 
LINE_BREAKER = ([\r\n]+)

 transforms.conf:

[set_ilo_error] 
REGEX=(failure|failed) 
DEST_KEY=MetaData:Sourcetype 
FORMAT=sourcetype::ilo_error
WRITE_META = true

[bowesmana]

Is sourcetype ending up a multivalue field or does it contain odd characters?

index=ilo
| head 1
| eval st_count=mvcount(sourcetype), st_len=len(sourcetype)
| eval tmp_sourcetype=":".sourcetype.":"
| table sourcetype tmp_sourcetype st_count st_len

[sigma]

@bowesmana  I don't think so:

[isoutamo]

Is there any possibility that there are field named sourcetype in your event?

[sigma]

Unfortunately No!
And after weeks yet I don't know what the problem is!

[PickleRick]

Don't you have some search-time field defined overriding the original one?

What does the search log say (especially the LISPY part) when you search for a specific sourcetype?