Developing for Splunk Enterprise

Splunk python SDK and REST string search query issues.

cresposh
Explorer

Hi everyone, 

In advance, thanks for reading and responding. 

I have an existing issue when using (with python) Splunk SDK and Rest to perform a search. 

In this case i am attempting to query for details given a SMTP message ID (the query parameter) . 

Additionally, to note that about 75% of the queries work as expected and return the data while the rest will indicate that there are no results. (and of course, i can confirm that there is data through the gui) 

 

As an example, the search query will return results for parameter (1) (2) and (3) but not for parameter (4) (5) 

  1. 92037848562344152638461b32.1739vb98635.290-9302924841.1701506175.7300a656@mail00.cat66.vvvv.net  
  2. AM7P191MB0581C4397B54F7DA07DD3DAF840D0@AM7P191MB0581.EURP191.PROD.OUTLOOK.COM 
  3. WHGD892HSG6328EA0C84C32E79576307E810D0@VXBSGHD82978GS.US9978WS.PRUDD.OUTLOOK.COM
  4. PHJKUYU4758WHD74393JHEHE7387648Y3B0CC40D0@DSE334WS01MB4950.DEVDEV.predd.exchange.com 
  5. MU98SAHKJ8E87495023B503385D6E36513B0CC40D0@TEUYS899WK93UE3.DROID.svrti.resound.com

I am really confused about the reason behind this issue. 

Labels (3)
Tags (3)
0 Karma
1 Solution

cresposh
Explorer

RESOLVED: 

While looking for API call logs to see if i could find any clue, i found that the search head the call is goes through is missing configuration for the parsing of fields from a few sources and including the messageID field that I am querying for. 

 

I have not had this issue before but were under the (wrong) impression that the parsing of fields and data was synchronized automatically.  

View solution in original post

0 Karma

cresposh
Explorer

RESOLVED: 

While looking for API call logs to see if i could find any clue, i found that the search head the call is goes through is missing configuration for the parsing of fields from a few sources and including the messageID field that I am querying for. 

 

I have not had this issue before but were under the (wrong) impression that the parsing of fields and data was synchronized automatically.  

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

What is the exact query you are using and some sample raw log events (anonymised of course) would be helpful too?

0 Karma

cresposh
Explorer

Hi ITWhisperer, 

I did not add any of the code just because it is the one i use as a template to complete other queries successfully, so i would (maybe mistakenly) think that it is not necessarily the code.

Still here is part of it and thanks for your input. 

 

 

my_creds_here {}

servercontent = requests.post(str(baseurl) + '/services/auth/login', headers = {}, data = data, verify = False)
sessionkey = minidom.parseString(servercontent.content).getElementsByTagName('sessionKey')[0].childNodes[
0].nodeValue


searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id'

searchjob = requests.post(str(baseurl) + '/services/search/jobs',
headers = {'Authorization': 'Splunk %s' % sessionkey},
data = {'search': searchquery}, verify = False) # [1]
sid = minidom.parseString(searchjob.content).getElementsByTagName('sid')[0].childNodes[0].nodeValue



servicessearchstatusstr = '/services/search/jobs/%s/' % sid
isnotdone = True
while isnotdone:
searchstatus = requests.get(baseurl + servicessearchstatusstr, auth = (username, password),
verify = False).content.decode('utf-8')
isdonestatus = re.compile('isDone">(0|1)')
isdonestatus = isdonestatus.search(searchstatus).groups()[0]
if (isdonestatus == '1'):
isnotdone = False
print("====>search status: %s <====" % isdonestatus)

# Get the search results
if isdonestatus == '1':
services_search_results_str = '/services/search/jobs/%s/results?output_mode=json&count=0' % sid
searchresults = requests.get(baseurl + services_search_results_str, auth = (username, password),
verify = False)

#print response

 

This is the response for queries in which no results are given. 

  • b'{"preview":false,"init_offset":0,"post_process_count":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"results":[]}'

Response where it responds with results

  • b'{"preview":false,"init_offset":0,"messages":[{"type":"INFO","text":"Your timerange was substituted based on your search string"}],"fields":[{"MY FIELDS HERE"],"results":[{"MY RESULTS HERE "}], "highlighted":{}}'

 

 

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Maybe this line should be closing the double quotes around message_id?

searchquery = 'index=myindex smtp-message-id="' + str(message_id) + '" earliest=-72 | fields smtp-message-id'

 

0 Karma

cresposh
Explorer

Hi, 

It is, i had removed it by accident when posting the code. 

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>