I’m working on a Splunk search that needs to perform a lookup against a CSV file. The challenge is that some of the fields in the lookup table contain empty values, meaning an exact match doesn’t work. Here’s a simplified version of my search:   index="main" eventType="departure" | table _time commonField fieldA fieldB fieldC fieldD fieldE | lookup reference_data.csv commonField fieldA fieldB fieldC fieldD fieldE OUTPUTNEW offset   The lookup file reference_data.csv contains the fields: commonField , fieldA, fieldB, fieldC, fieldD, fieldE, lookupValue. Sometimes, fieldB, fieldC, or other fields in the lookup table are empty. fieldA always has a value, sometimes the same, but the value of the offset field changes based on the values of the other fields. If a lookup row has an empty value for fieldB, I still want it to match based on the available fields. What I've Tried: Using lookup normally, but it requires all fields to match exactly, which fails when lookup fields are empty. Creating multiple lookup commands for different field combinations, but this isn’t scalable. Desired Outcome: If commonField  matches, but fieldB is empty in the lookup file, I still want the lookup to return lookupValue. The lookup should prioritize rows with the most matching fields but still work even if some fields are missing.   Is there a way to perform a lookup in Splunk that allows matches even when some lookup fields are empty? ... View more

Hi, I`m trying to make an API request from my local machine to our Splunk Cloud instance, without much success. Checked the Firewall logs and I can`t see any blocked/denied traffic. Using: -  curl 7.29.0 - nss-3.90 Error received:   * Host myDomain.splunkcloud.com:8089 was resolved. * IPv6: (none) * IPv4: xx.xx.xx.xxx * Trying xx.xx.xx.xxx:8089... * Connected to myDomain.splunkcloud.com (xx.xx.xx.xxx) port 8089 * schannel: disabled automatic use of client certificate * ALPN: curl offers http/1.1 * Recv failure: Connection was reset * schannel: failed to receive handshake, SSL/TLS connection failed * closing connection #0 curl: (35) Recv failure: Connection was reset     ... View more

Hi Everyone, The issue with the code below appears to be with the values of the {report_id} variable not being passed correctly to the download_report function, in particular this line:       url = f"https://example_url/{report_id}/download"       If I hardcode the url with a valid token, instead of the {report_id} variable,  the report gets downloaded, as expected. Any help would be much appreciated ! Full code below:       import requests def collect_events(helper, ew): """ Main function to authenticate, generate report ID, and download the report. """ username = helper.get_arg('username') password = helper.get_arg('password') auth_url = "https://example_url/auth" headers = { 'Content-Type': 'application/x-www-form-urlencoded', } data = { 'username': username, 'password': password, 'token': 'true', 'permissions': 'true', } try: # Step 1: Authenticate to get the JWT token auth_response = requests.post(auth_url, headers=headers, data=data) if auth_response.status_code == 201: jwt_token = auth_response.text.strip() # Extract and clean the token if jwt_token: # Log and create an event for the JWT token event = helper.new_event( data=f"JWT Token: {jwt_token}" ) ew.write_event(event) # Step 2: Generate the report ID report_id = generate_report_id(jwt_token, helper) if report_id: # Log and create an event for the report ID event = helper.new_event( data=f"Report ID: {report_id}" ) ew.write_event(event) # Step 3: Download the report file_path = download_report(jwt_token, report_id, helper) if file_path: helper.log_info(f"Report successfully downloaded to: {file_path}") else: raise ValueError("Failed to download the report.") else: raise ValueError("Failed to generate report ID.") else: raise ValueError("JWT token not found in response.") else: raise ValueError(f"Failed to get JWT: {auth_response.status_code}, {auth_response.text}") except Exception as e: helper.log_error(f"Error in collect_events: {e}") def generate_report_id(jwt_token, helper): url = "https://example_url" headers = { "accept": "application/json", "Authorization": f"Bearer {jwt_token}" } params = { "havingQuery": "isSecurity: true", "platform": "Windows" } try: response = requests.get(url, headers=headers, params=params) if response.status_code in (200, 201): report_data = response.json() report_id = report_data.get('reportId') if report_id: return report_id else: raise ValueError("Report ID not found in response.") else: raise ValueError(f"Failed to generate report ID: {response.status_code}, {response.text}") except Exception as e: helper.log_error(f"Error while generating report ID: {e}") raise ValueError(f"Error while generating report ID: {e}") def download_report(jwt_token, report_id, helper): """ Downloads the report using the JWT token and report ID. """ url = f"https://example_url/{report_id}/download" headers = { "accept": "application/json", "Authorization": f"Bearer {jwt_token}", } try: # Make the request to download the report response = helper.send_http_request(url, method="GET", headers=headers, verify=True) if response.status_code in (200, 201): # Save the report content to a file sanitized_report_id = "".join(c if c.isalnum() else "_" for c in report_id) file_path = f"C:\\Program Files\\Splunk\\etc\\apps\\splunk_app_addon-builder\\local\\temp\\{sanitized_report_id}.csv.gz" with open(file_path, "wb") as file: file.write(response.content) helper.log_info(f"Report downloaded successfully to: {file_path}") return file_path else: raise ValueError(f"Failed to download report: {response.status_code}, {response.text}") except Exception as e: helper.log_error(f"Error while downloading report: {e}") raise ValueError(f"Error while downloading report: {e}")         ... View more

Can`t seem to get my head round this one - I`ve got a table and would like the users to be able to click on a row and to add a Summary comment, but there`s a bug in the code. The comments get submitted BEFORE I click on the Submit button, which doesn`t seems to work anyway.     <form version="1.1" theme="light" script="TA-images_and-_files:tokenlinks.js"> <label>Report</label> <search> <query>| makeresults|eval Date=strftime(_time,"%d/%m/%Y")|fields - _time</query> <done> <set token="defaut_time">$result.Date$</set> </done> </search> <fieldset submitButton="false"> <input type="dropdown" token="date_tok" searchWhenChanged="true"> <label>Date:</label> <fieldForLabel>Date</fieldForLabel> <fieldForValue>Date</fieldForValue> <search> <query>| makeresults | timechart span=1d count | sort - _time | eval Date=strftime(_time, "%d/%m/%Y"), earliest=relative_time(_time, "@d") | table Date, earliest | head 7 | sort - earliest</query> <earliest>-7d@h</earliest> <latest>now</latest> </search> <default>$defaut_time$</default> </input> <input type="dropdown" token="shift_tok" searchWhenChanged="true"> <label>Shift:</label> <choice value="Day">Day</choice> <choice value="Night">Night</choice> <default>Day</default> <initialValue>Day</initialValue> </input> </fieldset> <row> <panel id="input_panel" depends="$show_input$"> <input type="text" token="Summary"> <label>Summary</label> </input> <input type="text" token="Date"> <label>Date</label> </input> <input type="text" token="Time"> <label>Time</label> </input> <input type="text" token="Shift"> <label>Shift</label> </input> <html> <div> <button type="button" id="buttonId" class="btn btn-primary">Submit</button> <button style="margin-left:10px;" class="btn" data-token-json="{&quot;show_input&quot;: null}">Cancel</button> </div> </html> </panel> </row> <row depends="$hideMe$"> <panel> <table> <search> <done> <unset token="form.Summary"></unset> <unset token="form.Date"></unset> <unset token="form.Time"></unset> <unset token="form.Shift"></unset> <unset token="show_input"></unset> </done> <query>| inputlookup handover_timeline_comments.csv | append [ | makeresults | eval "Summary" = "$form.Summary$", Shift="$form.Shift$", Date="$form.Date$", Time="$form.Time$" ] | outputlookup handover_timeline_comments.csv</query> <earliest>$earliest$</earliest> <latest>$latest$</latest> <refresh>30</refresh> <refreshType>delay</refreshType> </search> <option name="drilldown">none</option> <option name="refresh.display">progressbar</option> </table> </panel> </row> <row> <panel> <table> <search> <query>| makeresults count=24 | eval Date= "$date_tok$", Shift="$shift_tok$" | streamstats count as Time | eval Time=if(Time&lt;10, "0".Time.":00", Time.":00") | eval Time=case( Shift == "Night" AND Time &gt;= "19:00", Time, Shift == "Day" AND Time &gt;= "07:00" AND Time &lt;= "18:00", Time, 1==1, null ) | where isnotnull(Time) | append [ | makeresults count=24 | streamstats count as Time | eval Time=if(Time&lt;10, "0".Time.":00", Time.":00") | table Time | eval Date= "$date_tok$", Shift="$shift_tok$" | eval Time=case( Shift == "Night" AND Time &lt;= "06:00", Time, 1==1, null ) | where isnotnull(Time) ] | eval Summary="" | fields - _time | lookup handover_timeline_comments.csv Date Shift Time OUTPUT Summary | eventstats last(Summary) as Summary by Date Shift Time</query> <earliest>-24h@h</earliest> <latest>now</latest> <refresh>10s</refresh> </search> <option name="count">12</option> <option name="drilldown">cell</option> <option name="refresh.display">progressbar</option> <drilldown> <set token="form.Date">$row.Date$</set> <set token="form.Shift">$row.Shift$</set> <set token="form.Time">$row.Time$</set> <set token="show_input">true</set> </drilldown> </table> </panel> </row> </form>     .js:   requirejs([ '../app/simple_xml_examples/libs/jquery-3.6.0-umd-min', '../app/simple_xml_examples/libs/underscore-1.6.0-umd-min', 'util/console', 'splunkjs/mvc', 'splunkjs/mvc/simplexml/ready!' ], function($, _, console, mvc) { function setToken(name, value) { console.log('Setting Token %o=%o', name, value); var defaultTokenModel = mvc.Components.get('default'); if (defaultTokenModel) { defaultTokenModel.set(name, value); } var submittedTokenModel = mvc.Components.get('submitted'); if (submittedTokenModel) { submittedTokenModel.set(name, value); } } $('.dashboard-body').on('click', '[data-set-token],[data-unset-token],[data-token-json]', function(e) { e.preventDefault(); var target = $(e.currentTarget); var setTokenName = target.data('set-token'); if (setTokenName) { setToken(setTokenName, target.data('value')); } var unsetTokenName = target.data('unset-token'); if (unsetTokenName) { setToken(unsetTokenName, undefined); } var tokenJson = target.data('token-json'); if (tokenJson) { try { if (_.isObject(tokenJson)) { _(tokenJson).each(function(value, key) { if (value === null) { // Unset the token setToken(key, undefined); } else { setToken(key, value); } }); } } catch (e) { console.warn('Cannot parse token JSON: ', e); } } }); });     ... View more

Hi Everyone, Has anyone managed to successfully use the "Akamai Prolexic DNS GTM and SIEM API (Unofficial)"  app ? I keep getting this error when testing the Prolexic API data input:         Traceback (most recent call last): File "C:\Program Files\Splunk\etc\apps\akamai-api-integration\bin\akamai_api_integration\aob_py3\urllib3\connection.py", line 175, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "C:\Program Files\Splunk\etc\apps\akamai-api-integration\bin\akamai_api_integration\aob_py3\urllib3\util\connection.py", line 72, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "C:\Program Files\Splunk\Python-3.7\lib\socket.py", line 752, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 11001] getaddrinfo failed         The official Akamai SIEM app was not designed to ingest the Prolexic API so unfortunately is of no use to me. Many thanks. ... View more

Hi Everyone, We`ve created a new TA to get data in from an API - this was done on the HF and the data is being sent to our Cloud instance, however the field values are getting duplicated. Tried changing the INDEXED_EXTRACTIONS and KV_MODE settings on the HV as explained by many others without success. In Cloud there wasn`t a source type for this data feed, so we`ve created one manually and set INDEXED_EXTRACTIONS = none and KV_MODE = json however this made no change.  I`ve also added a stanza in local.meta on the HF as suggested by others as follows: export = system. Here`s a snap of the sourcetype stanza on the HF. As you can see INDEXED_EXTRACTIONS  and KV_MODE  are both set to false, but I`ve tried pretty much every combination possible - which suggests to me the issue is in the Cloud.   ADD_EXTRA_TIME_FIELDS = True ANNOTATE_PUNCT = True AUTO_KV_JSON = false BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE_DATE = CHARSET = UTF-8 DATETIME_CONFIG = CURRENT DEPTH_LIMIT = 1000 DETERMINE_TIMESTAMP_DATE_WITH_SYSTEM_TIME = false HEADER_MODE = INDEXED_EXTRACTIONS = none KV_MODE = none LB_CHUNK_BREAKER_TRUNCATE = 2000000 LEARN_MODEL = true LEARN_SOURCETYPE = true LINE_BREAKER = ([\r\n]+) LINE_BREAKER_LOOKBEHIND = 100 MATCH_LIMIT = 100000 MAX_DAYS_AGO = 2000 MAX_DAYS_HENCE = 2 MAX_DIFF_SECS_AGO = 3600 MAX_DIFF_SECS_HENCE = 604800 MAX_EVENTS = 256 MAX_TIMESTAMP_LOOKAHEAD = 128 MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = MUST_NOT_BREAK_BEFORE = NO_BINARY_CHECK = true SEGMENTATION = indexing SEGMENTATION-all = full SEGMENTATION-inner = inner SEGMENTATION-outer = outer SEGMENTATION-raw = none SEGMENTATION-standard = standard SHOULD_LINEMERGE = 0 TIME_FORMAT = TRANSFORMS = TRUNCATE = 10000 category = Structured detect_trailing_nulls = false disabled = false maxDist = 100 priority = pulldown_type = 1 sourcetype = termFrequencyWeightedDist = false    Any help would be greatly appreciated. ... View more

Hi, Once a month we receive a file via email that we manually upload to Splunk as a lookup CSV file.  The current process is to delete the old file and to upload the new one, keeping the same file name. The existing reports use this file without any issues. There is now a requirement to compare the current file with the previous version and highlight if any values have been added or removed (the columns stay the same). Initially I wanted to use the "inputlookup" and "collect" commands to output the data into an index and then build a search to compare the data based on the ingest time, effectively comparing the 2 files. However, I`m getting the following error: "The lookup table 'test.csv' requires a .csv or KV store lookup definition." The file actually exists and it`s located in "/opt/splunk/etc/apps/test_app/lookups/test.csv" The lookup definition also exists: "test_LD" I suspect this is caused by the size of the lookup file (approx. 36 MB) and wanted to ask for suggestions or workarounds ? Many thanks. ... View more