topic Splunk python SDK and REST string search query issues. in Splunk Dev

Splunk python SDK and REST string search query issues.

cresposh — Wed, 07 Oct 2020 00:22:11 GMT

Hi everyone,

In advance, thanks for reading and responding.

I have an existing issue when using (with python) Splunk SDK and Rest to perform a search.

In this case i am attempting to query for details given a SMTP message ID (the query parameter) .

Additionally, to note that about 75% of the queries work as expected and return the data while the rest will indicate that there are no results. (and of course, i can confirm that there is data through the gui)

As an example, the search query will return results for parameter (1) (2) and (3) but not for parameter (4) (5)

92037848562344152638461b32.1739vb98635.290-9302924841.1701506175.7300a656@mail00.cat66.vvvv.net
AM7P191MB0581C4397B54F7DA07DD3DAF840D0@AM7P191MB0581.EURP191.PROD.OUTLOOK.COM
WHGD892HSG6328EA0C84C32E79576307E810D0@VXBSGHD82978GS.US9978WS.PRUDD.OUTLOOK.COM
PHJKUYU4758WHD74393JHEHE7387648Y3B0CC40D0@DSE334WS01MB4950.DEVDEV.predd.exchange.com
MU98SAHKJ8E87495023B503385D6E36513B0CC40D0@TEUYS899WK93UE3.DROID.svrti.resound.com

I am really confused about the reason behind this issue.

Re: Splunk python SDK and REST string search query issues.

ITWhisperer — Wed, 07 Oct 2020 07:33:13 GMT

What is the exact query you are using and some sample raw log events (anonymised of course) would be helpful too?

Re: Splunk python SDK and REST string search query issues.

cresposh — Wed, 07 Oct 2020 15:52:36 GMT

Hi ITWhisperer,

I did not add any of the code just because it is the one i use as a template to complete other queries successfully, so i would (maybe mistakenly) think that it is not necessarily the code.

Still here is part of it and thanks for your input.

my_creds_here {} servercontent = requests.post(str(baseurl) + '/services/auth/login', headers = {}, data = data, verify = False) sessionkey = minidom.parseString(servercontent.content).getElementsByTagName('sessionKey')[0].childNodes[ 0].nodeValue searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id' searchjob = requests.post(str(baseurl) + '/services/search/jobs', headers = {'Authorization': 'Splunk %s' % sessionkey}, data = {'search': searchquery}, verify = False) # [1] sid = minidom.parseString(searchjob.content).getElementsByTagName('sid')[0].childNodes[0].nodeValue servicessearchstatusstr = '/services/search/jobs/%s/' % sid isnotdone = True while isnotdone: searchstatus = requests.get(baseurl + servicessearchstatusstr, auth = (username, password), verify = False).content.decode('utf-8') isdonestatus = re.compile('isDone">(0|1)') isdonestatus = isdonestatus.search(searchstatus).groups()[0] if (isdonestatus == '1'): isnotdone = False print("====>search status: %s <====" % isdonestatus) # Get the search results if isdonestatus == '1': services_search_results_str = '/services/search/jobs/%s/results?output_mode=json&count=0' % sid searchresults = requests.get(baseurl + services_search_results_str, auth = (username, password), verify = False) #print response

This is the response for queries in which no results are given.

b'{"preview":false,"init_offset":0,"post_process_count":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"results":[]}'

Response where it responds with results

b'{"preview":false,"init_offset":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"fields":[{"MY FIELDS HERE"],"results":[{"MY RESULTS HERE "}], "highlighted":{}}'

Re: Splunk python SDK and REST string search query issues.

ITWhisperer — Wed, 07 Oct 2020 15:47:51 GMT

Maybe this line should be closing the double quotes around message_id?

searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id'

Re: Splunk python SDK and REST string search query issues.

cresposh — Wed, 07 Oct 2020 15:52:03 GMT

Hi,

It is, i had removed it by accident when posting the code.

Re: Splunk python SDK and REST string search query issues.

cresposh — Wed, 07 Oct 2020 23:43:54 GMT

RESOLVED:

While looking for API call logs to see if i could find any clue, i found that the search head the call is goes through is missing configuration for the parsing of fields from a few sources and including the messageID field that I am querying for.

I have not had this issue before but were under the (wrong) impression that the parsing of fields and data was synchronized automatically.