This happens in one of newly installed 10.0.1 instances. The only data ingested is tutorialdata.zip from Splunk Tutorial. The tutorial exercises on fields like action, productId, etc. But this instance gives none of them. After some long diagnosis, I found the culprit: The search command doesn't perform default key-value extraction in raw events. Even if I force an extract command in search like this,
sourcetype=access_combined_wcookie
| extract kvdelim== pairdelim=&those fields are still not extracted:
Only when I force the same pairdelim + kvdelim directly on url_query like this
sourcetype=access_combined_wcookie
| rename _raw as temp
| rename uri_query as _raw
| extract kvdelim== pairdelim=&will this instance extract those fields:
How can I find the root cause?
Additional data points:
How have you ingested the data?
Using the exact sequence as detailed in Upload the tutorial data. As mentioned, the same exercise was done in a few other instances including newly installed instances with no problem. Today, I created another 10.0.1 instance on the same OS with nearly identical configurations, no such problem.
This is a strange one and given that it sems to work in every other environment, it sounds like there is nothing wrong with the process you used. Is it possible for you to compare you config files on a file by file basis between a working environment and non-working environment to determine what the significant difference might be?