Splunk python SDK and REST string search query issues.

[cresposh]

Hi everyone, 

In advance, thanks for reading and responding. 

I have an existing issue when using (with python) Splunk SDK and Rest to perform a search. 

In this case i am attempting to query for details given a SMTP message ID (the query parameter) . 

Additionally, to note that about 75% of the queries work as expected and return the data while the rest will indicate that there are no results. (and of course, i can confirm that there is data through the gui) 

As an example, the search query will return results for parameter (1) (2) and (3) but not for parameter (4) (5) 

1. 92037848562344152638461b32.1739vb98635.290-9302924841.1701506175.7300a656@mail00.cat66.vvvv.net  
2. AM7P191MB0581C4397B54F7DA07DD3DAF840D0@AM7P191MB0581.EURP191.PROD.OUTLOOK.COM 
3. WHGD892HSG6328EA0C84C32E79576307E810D0@VXBSGHD82978GS.US9978WS.PRUDD.OUTLOOK.COM
4. PHJKUYU4758WHD74393JHEHE7387648Y3B0CC40D0@DSE334WS01MB4950.DEVDEV.predd.exchange.com 
5. MU98SAHKJ8E87495023B503385D6E36513B0CC40D0@TEUYS899WK93UE3.DROID.svrti.resound.com

I am really confused about the reason behind this issue. 

[cresposh · Accepted solution]

RESOLVED: 

While looking for API call logs to see if i could find any clue, i found that the search head the call is goes through is missing configuration for the parsing of fields from a few sources and including the messageID field that I am querying for. 

I have not had this issue before but were under the (wrong) impression that the parsing of fields and data was synchronized automatically.  

[ITWhisperer]

What is the exact query you are using and some sample raw log events (anonymised of course) would be helpful too?

[cresposh]

Hi ITWhisperer, 

I did not add any of the code just because it is the one i use as a template to complete other queries successfully, so i would (maybe mistakenly) think that it is not necessarily the code.

Still here is part of it and thanks for your input. 

my_creds_here {}

servercontent = requests.post(str(baseurl) + '/services/auth/login', headers = {}, data = data, verify = False)
sessionkey = minidom.parseString(servercontent.content).getElementsByTagName('sessionKey')[0].childNodes[
0].nodeValue


searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id'

searchjob = requests.post(str(baseurl) + '/services/search/jobs',
headers = {'Authorization': 'Splunk %s' % sessionkey},
data = {'search': searchquery}, verify = False) # [1]
sid = minidom.parseString(searchjob.content).getElementsByTagName('sid')[0].childNodes[0].nodeValue



servicessearchstatusstr = '/services/search/jobs/%s/' % sid
isnotdone = True
while isnotdone:
searchstatus = requests.get(baseurl + servicessearchstatusstr, auth = (username, password),
verify = False).content.decode('utf-8')
isdonestatus = re.compile('isDone">(0|1)')
isdonestatus = isdonestatus.search(searchstatus).groups()[0]
if (isdonestatus == '1'):
isnotdone = False
print("====>search status: %s <====" % isdonestatus)

# Get the search results
if isdonestatus == '1':
services_search_results_str = '/services/search/jobs/%s/results?output_mode=json&count=0' % sid
searchresults = requests.get(baseurl + services_search_results_str, auth = (username, password),
verify = False)

#print response

This is the response for queries in which no results are given. 

• b'{"preview":false,"init_offset":0,"post_process_count":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"results":[]}'

Response where it responds with results

• b'{"preview":false,"init_offset":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"fields":[{"MY FIELDS HERE"],"results":[{"MY RESULTS HERE "}], "highlighted":{}}'

[ITWhisperer]

Maybe this line should be closing the double quotes around message_id?

searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id'

[cresposh]

Hi, 

It is, i had removed it by accident when posting the code.