Splunk Search

Getting an intermittent error searching against a kvstore with json using inputlookup and lookup- How to fix?

donelliot
Path Finder

on splunk cloud 8.2.2202.2

issuing the command as follows I get an error one times out of four - 

 


| inputlookup append=t ethos_vulnaction_generic

Last 30 minutes
 
Error in 'inputlookup' command: External lookup table 'inputlookup' returned error code 0. Results might be incorrect.
The search job has failed due to an error. You may be able view the job in the Job Inspector.


| inputlookup append=t ethos_vulnaction_generic

restarted splunk - no luck

Not sure how to decipher job inspector - but this inconsistency - sometimes it work sometimes it doesn't is strange.

kvstore was populated with json, and lookup; does have a filter in it - NOT asset_specific = "true"

I tried removing the filter seeing if this impacted the results but I still get an error about one time in four.. 

if i do a rest query of the kvstore in json it looks  healthy to me... besides if I take this filter out I still get stability issues
        "asset_specific": true,

A cut down example of the json used to populate the record. I do refer explicitly to the field in the lookup as details.plugin_id which the lookup command seems to like... a snippet of json

{
"action_description": "zulu specific",
"asset_specific": true,
"details": {
"plugin_id": [
"153989"
]
}
}

Labels (1)
Tags (2)
0 Karma

donelliot
Path Finder

Ok found a bit more...

56 _userContext= nobody
08-19-2022 03:15:09.120 ERROR HttpClientRequest [13376 phase_1] - Caught exception while parsing HTTP reply: Unparsable gzip header in HTTP response
08-19-2022 03:15:09.120 ERROR KVServiceClient [13376 phase_1] - KVServiceClient transaction failed after 0 retries. uri = <blah>
08-19-2022 03:15:09.120 ERROR SSCInputLookup [13376 phase_1] - Failed to call KVServiceClient for Input Lookup:
08-19-2022 03:15:09.120 ERROR SearchOperator:inputcsv [13376 phase_1] - Error in 'inputlookup' command: External lookup table 'inputlookup' returned error code 0. Results might be incorrect.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...