Getting an intermittent error searching against a kvstore with json using inputlookup and lookup- How to fix?

[donelliot]

on splunk cloud 8.2.2202.2

issuing the command as follows I get an error one times out of four - 

| inputlookup append=t ethos_vulnaction_generic

Last 30 minutes
 
Error in 'inputlookup' command: External lookup table 'inputlookup' returned error code 0. Results might be incorrect.
The search job has failed due to an error. You may be able view the job in the Job Inspector.

| inputlookup append=t ethos_vulnaction_generic

restarted splunk - no luck

Not sure how to decipher job inspector - but this inconsistency - sometimes it work sometimes it doesn't is strange.

kvstore was populated with json, and lookup; does have a filter in it - NOT asset_specific = "true"

I tried removing the filter seeing if this impacted the results but I still get an error about one time in four.. 

if i do a rest query of the kvstore in json it looks  healthy to me... besides if I take this filter out I still get stability issues
        "asset_specific": true,

A cut down example of the json used to populate the record. I do refer explicitly to the field in the lookup as details.plugin_id which the lookup command seems to like... a snippet of json

{
"action_description": "zulu specific",
"asset_specific": true,
"details": {
"plugin_id": [
"153989"
]
}
}

[donelliot]

Ok found a bit more...

56 _userContext= nobody
08-19-2022 03:15:09.120 ERROR HttpClientRequest [13376 phase_1] - Caught exception while parsing HTTP reply: Unparsable gzip header in HTTP response
08-19-2022 03:15:09.120 ERROR KVServiceClient [13376 phase_1] - KVServiceClient transaction failed after 0 retries. uri = <blah>
08-19-2022 03:15:09.120 ERROR SSCInputLookup [13376 phase_1] - Failed to call KVServiceClient for Input Lookup:
08-19-2022 03:15:09.120 ERROR SearchOperator:inputcsv [13376 phase_1] - Error in 'inputlookup' command: External lookup table 'inputlookup' returned error code 0. Results might be incorrect.