Events for simple query index=os sourcetype=cpu are not breaking for users without admin role.
All other user without admin role
For user with admin role
What could be the reason? Any suggestions please
I think there is a problem on KV_MODE settings on your search head for non-admin users. It should be "multi".
You may have another props.conf that has [cpu] stanza inside which overwrites the Splunk_TA_nix app props.
Could you please run the below command on your search head and see if this exists? If yes you will see on which config file you have a second cpu stanza that has wrong KV_MODE config. You may see KV_MODE setting which is different than multi.
/opt/splunk/bin/splunk btool props list cpu --debug
You are probably using this one https://splunkbase.splunk.com/app/833/ ?
Have you installed it as described on instructions? And is this issue only with this one source type or other too?
Basically if this has installed as expected and all those events are collected after that those should be show as exactly same way independent of user/role. When installation and indexing has done right those should be indexed as events in splunk.
Can you do a new query with exactly same time period like earliest="mm/dd/yyyy:HH:MM:SS" latest="mm/dd:yyyy:HH+1:MM:SS" and check if those are still differing? Check also if there is any difference between hosts where those are collected.
KO <=> Knowledge Object
And it would indeed look like a permission problem but I don't recall any search-time settings affecting event breaking. The events are separated at ingest time. Are you sure nothing changed on your sources' side or in ingest settings? (You show events from two different days)
@PickleRick No nothing has changed. (You show events from two different days) >>It is different timezone .
For Admin role the events are breaking properly.Permission is given for all the users in local.meta
It's interesting though because it's not that easy to split an event in search-time. There must be some indeed some KO affecting your search but it's hard to say which one without listing them all and manually verifying.
Remember that instead of clicking through the UI you can list various KO types with REST calls. Then you can check the permissions fields (you'll definitely want to limit list of fields returned from REST because there are typically up to several hundred fields).
In one of our Single Instance (test) We faced the same issue. Then we found that in metadata permission was not given for all the users. After updating the metadata it started working fine for all the users.
Now the same app we placed it in our production distributed environment but it is still not working ,the events are not breaking for non admin users.
It was suggested that we place splunk ta-nix in all our instances (Indexers,Forwarders,DS).we tried that as well.
What else can we try to break the events for non admin users as well?
Version of app is 8.3.0
@isoutamo @ITWhisperer @scelikok @gcusello @thambisetty ++++ any suggestions pls