All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

In PAN tarffic, where we are trying to exclude events with "block-untst-rule" string. PAN traffic logs are configured to receive in syslog server. I tried to use filter in syslog-ng config as shown b... See more...
In PAN tarffic, where we are trying to exclude events with "block-untst-rule" string. PAN traffic logs are configured to receive in syslog server. I tried to use filter in syslog-ng config as shown below.  filter exclogs { not match("block\\-untst\\-rule"); }; log {source(s_syslog_pa_cloud); filter {exclogs}; destination(d_syslog_pa_cloud); }; Other way we tried is by using props.conf and transforms.conf props.conf [pan:traffic] TRANSFORMS-set = setnull transforms.conf [setnull] REGEX = block\\-untst\\-rule DEST_KEY = queue FORMAT = nullQueue Both ways we are not able to exclude the events from ingesting. Please do assist.
I am struggling to get the MS O365 reporting app working with OAuth. Here's the error in ta_ms_o365_reporting_ms_o365_message_trace_oauth.log: 2022-10-21 16:24:38,096 INFO pid=91299 tid=MainThread ... See more...
I am struggling to get the MS O365 reporting app working with OAuth. Here's the error in ta_ms_o365_reporting_ms_o365_message_trace_oauth.log: 2022-10-21 16:24:38,096 INFO pid=91299 tid=MainThread file=splunk_rest_client.py:_request_handler:99 | Use HTTP connection pooling 2022-10-21 16:24:38,097 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer (body: {}) 2022-10-21 16:24:38,100 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_new_conn:975 | Starting new HTTPS connection (1): 127.0.0.1:8089 2022-10-21 16:24:38,122 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 5562 2022-10-21 16:24:38,123 DEBUG pid=91299 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.025611 2022-10-21 16:24:38,124 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/ (body: {'count': -1, 'offset': 0, 'search': 'TA_MS_O365_Reporting_checkpointer'}) 2022-10-21 16:24:38,136 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/?count=-1&offset=0&search=TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 4714 2022-10-21 16:24:38,137 DEBUG pid=91299 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.013834 2022-10-21 16:24:38,141 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth (body: {}) 2022-10-21 16:24:38,149 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth HTTP/1.1" 404 140 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Start date: 2022-10-16 16:24:38.151205, End date: 2022-10-16 17:24:38.151205 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ message trace URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2022-10-16T16:24:38.151205Z' and EndDate eq datetime'2022-10-16T17:24:38.151205Z' 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Getting proxy server. 2022-10-21 16:24:38,151 INFO pid=91299 tid=MainThread file=setup_util.py:log_info:142 | Proxy is not enabled! 2022-10-21 16:24:38,154 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_new_conn:975 | Starting new HTTPS connection (1): login.windows.net:443 2022-10-21 16:24:38,284 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://login.windows.net:443 "POST /[TENANT ID]/oauth2/token HTTP/1.1" 400 747 2022-10-21 16:24:38,290 ERROR pid=91299 tid=MainThread file=base_modinput.py:log_error:316 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/lib/splunktaucclib/modinput_wrapper/base_modinput.py", line 140, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 362, in collect_events get_events_continuous(helper, ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 98, in get_events_continuous if 'value' in message_response: TypeError: argument of type 'NoneType' is not iterable I've highlighted the requests that got a HTTP repsonse code of 200 in blue, and the 404 and 400 errors in red. If I use curl to access the URL with the 404 error I get: <?xml version="1.0" encoding="UTF-8"?> <response> <messages> <msg type="ERROR">Could not find object.</msg> </messages> </response> It looks like this is trying to access a "storage" directory inside /opt/splunk/etc/apps/TA-MS_O365_Reporting, which doesn't exist Looking at splunkd_access.log, when running curl, I get: 127.0.0.1 - splunk-system-user [21/Oct/2022:16:53:37.043 +0100] "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth HTTP/1.1" 404 140 "-" "curl" - - - 5ms What I've tried and checked: Splunk is running as the splunk user I've checked and reset the filesystem permissions for /opt/splunk I've deleted and recreated the inputs and the accounts I've deleted the app and re-installed, both through the CLI and Splunk Web Things that are less likely to be related, but I've done anyway: I've created new credentials in Azure for my enterprise app I've checked the permissions in Azure and confirmed that the permissions have been correctly granted The enterprise app in Azure is in the Global Reader and Exchange Administrator roles I'm out of ideas so any help is gratefully received! I'm giessing that resolving the 404 may resolve the 400 to. Splunk v. 9.0.1 Splunk Add-on for Microsoft Office 365 Reporting Web Service v. 2.0.1 Ubuntu 20.04.5 LTS on VMware 7 20 CPUs, 24GB RAM Thank you!        
Hi All, i need information shared on Splunkd uses what port ranges in Python 3.7
Hello, Quick question. How do I change the default number of lines to return in search? Is there a setting in limits.conf?   index=_audit action=search AND search!=*_internal* AND search!=*_au... See more...
Hello, Quick question. How do I change the default number of lines to return in search? Is there a setting in limits.conf?   index=_audit action=search AND search!=*_internal* AND search!=*_audit* AND user="user1"   The linecount max is 128. My larger search values return with ...(truncated)', autojo... in the _raw. Thanks in advance and God bless.
Hello all, I am trying to solve an issue with your addon for Splunk. Our client has an indice with name *:security-audit-*. Since we have a distributed environment we have heavy forwarders in cloud ... See more...
Hello all, I am trying to solve an issue with your addon for Splunk. Our client has an indice with name *:security-audit-*. Since we have a distributed environment we have heavy forwarders in cloud and indexers and searcheads in the on prem environment. The URL is resolvable via nslookup and the endpoint is giving us a correct response if we try to connect to it via our username and password. As you can see its transforming the : character into %3. Now, I am not sure if this is an actual issue, but our on-prem team is not receiving any data. I also tried using *security-audit-* which solved the errors but still no data has been recieved. [elasticsearch_json://srvadm] ca_certs_path = /opt/splunk/etc/auth/VWAG date_field_name = @timestamp elasticsearch_indice = *:security-audit-* elasticsearch_instance_url = https://redacted:9243 greater_or_equal = {{ ansible_date_time.date }} index = vw_de_aws_mlaas_apps interval = 300 lower_or_equal = now secret = {{ es_password }} use_ssl = 1 user = siem_readonly verify_certs = 0   root@fpea:/var/snap/amazon-ssm-agent/6312# cat /opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log | grep security-audit 2022-10-21 11:52:38,943 WARNING pid=811606 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.032s] 2022-10-21 11:57:40,619 WARNING pid=814893 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.095s] 2022-10-21 12:02:41,357 WARNING pid=818040 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.091s] 2022-10-21 12:07:39,512 WARNING pid=820807 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.066s] 2022-10-21 12:12:46,422 WARNING pid=823706 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.036s]  
Hi, I have a list of hosts/devices say from HostA to HostZ (PS: its not  a lookup file) I want to find out which host among the list show up in the particular index, say Index=IndexA. Could som... See more...
Hi, I have a list of hosts/devices say from HostA to HostZ (PS: its not  a lookup file) I want to find out which host among the list show up in the particular index, say Index=IndexA. Could someone help me on this?
I am trying to group range of decimal number: Range between 10.0.0 and10.15 =Medium 10.16 -11=High 11.1-11.5=critical  for example: Severity 10.15.4 12.6 12.6.0 10.15.7 10.15.7 10.15.7 ... See more...
I am trying to group range of decimal number: Range between 10.0.0 and10.15 =Medium 10.16 -11=High 11.1-11.5=critical  for example: Severity 10.15.4 12.6 12.6.0 10.15.7 10.15.7 10.15.7 12.6 12.6 10.15.7 12.5.1 12.6 12.6.0 10.15.7 12.6 12.6.0 11.0 12.5.1 11.0 12.6 12.6.0 11.0.1 12.3.1 12.6 12.6.0 11.2 12.6 12.6.0 11.2.3 11.6.3 12.2.1 12.6 12.6.0 11.2.3 12.6 12.6.0 11.4 12.5.1 11.4 12.5.1 12.5.1 12.6 12.6 11.4 12.6 12.6.0 11.5.1 12.6 12.6.0 11.5.2 11.6.2 12.6 12.6.0 11.5.2 11.7 11.7.0 11.5.2 12.2.1 12.5.1 11.5.2 12.2.1 12.6 12.6.0 11.5.2 12.6 12.6.0 11.6 11.6.0 11.6 11.6.2 12.2.1 12.6 12.6.0 11.6 11.6.2 12.3.1
I have three graphs that show results based on a global time range. However, if I have no results (no errors) the third graph is not displayed. I just want to display an empty graph with the ... See more...
I have three graphs that show results based on a global time range. However, if I have no results (no errors) the third graph is not displayed. I just want to display an empty graph with the same date ranges as the other graphs without displaying artificially inserted results that could be intepreted as errors. None of the many posts regarding this issue solved my problem as far as I understand it. Thanks in advance.
Hi, I have the following SPL working fine when I have a starting event and ending event in my logs. If I have a starting event but no ending event I get no results and would like to show at least s... See more...
Hi, I have the following SPL working fine when I have a starting event and ending event in my logs. If I have a starting event but no ending event I get no results and would like to show at least starting event info but am having some troubles. Is there a way to fake out the ending event to be the starting event just to get some data ? Or are there any other approaches that might make this work ? index=anIndex sourcetype=aSourceType (aString1 AND "START of script") OR (aString2 AND "COMPLETED") earliest=@d latest=now | rex "(?<event_name>(START of script)|(COMPLETED OK))" | eval event_name=CASE(event_name="START of script", "script_start", event_name="COMPLETED OK", "script_complete") | eval event_time=strftime(_time, "%Y-%m-%d %H:%M:%S") | eval {event_name}_time=_time | transaction host job_name startswith=(event_name="script_start") endswith=(event_name="script_complete") | eval aTime1= _time - (strptime(strftime(_time,"%Y-%m-%dT%H:%M:%S.%3N")." CDT","%Y-%m-%dT%H:%M:%S.%N%Z") - _time) | eval eventStartTimeCDT=strftime(aTime1, "%H:%M:%S %p") | eval endTime = _time + duration | eval eventEndTimeCDT=strftime(endTime, "%H:%M:%S %p") | eval dayNumber = strftime(endTime, "%w") | eval "Start / End Job's"="aString1 / aString2" | eval "Host Name"=if (host="aHostName1", "aHostName1", "aHostName2") | eval "Duration"=tostring(duration, "duration") | eval "Day" = strftime(endTime, "%a. %b. %e, %Y") | eval "Start Time"=eventStartTimeCDT | eval "End Time"=eventEndTimeCDT | eval "Due By Time" = if (dayNumber == 0, "02:00 PM", "07:00 AM") | table "Host Name", "Day", "Start / End Job's", "Start Time", "End Time", "Due By Time", "Duration"  
Hi everyone, I am following guide to create new custom REST endpoint, but I have problem on debug my code in local. How I can debug python code local. Currently, I am using Splunk Docker and moun... See more...
Hi everyone, I am following guide to create new custom REST endpoint, but I have problem on debug my code in local. How I can debug python code local. Currently, I am using Splunk Docker and mount local app to Splunk container. After every time changed code, I must refresh debug on Splunk. It is very taken time. Does everyone have other way to debug python code on Splunk. Thank.
To provide further from yesterday's SPL query. I am facing huge events in multivalues. I want to break in a single event. How can I achieve it. My current events are look like as below.  
Hello everyone! What is the best way to remove dots from domain in field? for example | eval field = lower(mvindex(split(field, "."), 0)) removes just 1 dot, and what if 2+ dots in domain?
I have configured HTTP inputs by creating HEC token in heavy forwarder. I see duplicate events every time I test sending data via these HEC token. I have validated that source does not have duplica... See more...
I have configured HTTP inputs by creating HEC token in heavy forwarder. I see duplicate events every time I test sending data via these HEC token. I have validated that source does not have duplicate events. Even if I send a test event using curl command, it appears twice. curl -k https://<endpoint>:8088/services/collector/event -H "Authorization: Splunk <HEC token>” -d '{"event": "This is a test event"}' Please help find out why are there duplicate events and how can I fix it.
Hi, I have a base search and post process searches on a dashboard that need to be split by source, but it doesn't appear like splitting by source works. The only thing shared is the index, and some ... See more...
Hi, I have a base search and post process searches on a dashboard that need to be split by source, but it doesn't appear like splitting by source works. The only thing shared is the index, and some fields but depending on the source I need to evaluate the fields differently. For instance: Base search: index=test_logs | fields A   Two post process searches: | search source=sourceA . (evaluate field A certain way because it's from source A)   | search source=sourceB . (evaluate field A a different way as it's from source B)   The problem is that when I do this nothing will load. I've found the only way to get this to work is to put the source in the base search but then I wouldn't be able to do my evaluations properly.
I know you can delete KVStore via the command line : https://dev.splunk.com/enterprise/docs/developapps/manageknowledge/kvstore/usetherestapitomanagekv/  Is it possible to delete from the splunk se... See more...
I know you can delete KVStore via the command line : https://dev.splunk.com/enterprise/docs/developapps/manageknowledge/kvstore/usetherestapitomanagekv/  Is it possible to delete from the splunk search line? I tried using the following but it doesn't work.   | rest /servicesNS/nobody/search/storage/collections/data/my_collection     I also tried installing the Webtools TA app to get access to being able to send external api requests but it says I'm unauthorized:   | curl method=get user="" password="" uri=https://host:8089/servicesNS/nobody/search/storage/collections/data/my_collection   The Webtools TA app uses python requests and I'm able to get a 200 when I make the request from a python script.
I have added the following to my dashboard and I am still getting this error (A custom JavaScript error caused an issue loading your dashboard). What should be my next steps? <dashboard version="1.... See more...
I have added the following to my dashboard and I am still getting this error (A custom JavaScript error caused an issue loading your dashboard). What should be my next steps? <dashboard version="1.1" script="myCustomJS.js"> or  <form version="1.1" script="myCustomJS.js">   Manage dashboards that need jQuery updates - Splunk Documentation
Is it possible to restrict a role to run a certain search or only be able to run saved searches? Ie a user can only run index=index | stats count by field? OR | savedsearch search1
Hello, Can anyone help me with the dashboards code for splunk app for windows? When i looked it on splunkbase, i see this app has been archived. please share if anyone is still using it.     ... See more...
Hello, Can anyone help me with the dashboards code for splunk app for windows? When i looked it on splunkbase, i see this app has been archived. please share if anyone is still using it.     Thanks,
I have logs like shown below: 2022-03-09T13:22:45.345-01:00 [app_driver_group_stream_api-1] | INFO s.p.k.o.external.thread 345 - [applicationid=String, offset=100, CADM=String,  IPOD=String,  Uniq... See more...
I have logs like shown below: 2022-03-09T13:22:45.345-01:00 [app_driver_group_stream_api-1] | INFO s.p.k.o.external.thread 345 - [applicationid=String, offset=100, CADM=String,  IPOD=String,  UniqueStringId=String, IMPT=000000-0000-000000-400-00000, applicationname=wwms-processor, Msgid=String, appId=app_group, EndToEndID=String, timestamp=17789552323] - app_thread [app_schema-677ghh89hhjjj-appThread-2] Processed 12 total count, run 0 quotations, and completed 0 total apps past the  last update 2022-03-09T13:22:45.345-01:00 [app_driver_group_stream_api-1] | INFO s.p.k.o.external.thread 345 - [applicationid=String, offset=100, CADM=String,  IPOD=String,  UniqueStringId=String, IMPT=000000-0000-000000-400-00000, applicationname=wwms-processor, Msgid=String, app=app_group_payment, EndToEndID=String, timestamp=17789552323] - app_thread [app_schema-677ghh89hhjjj-appThread-2] Processed 12 total count, run 0 quotations, and completed 0 total apps past the  last update 2022-03-10T12:10:45.345-01:00 [app_driver_group_stream_api-1] | INFO s.p.k.o.external.thread 345 - [applicationid=String, offset=100, CADM=String,  IPOD=String,  UniqueStringId=String, IMPT=000000-0000-000000-400-00000, applicationname=wwms-processor, Msgid=String, app=app_group_payment, EndToEndID=String, timestamp=17789552323] - app_thread [app_schema-677ghh89hhjjj-appThread-2] Processed 12 total count, run 0 quotations, and completed 0 total apps past the  last update 2022-03-15T10:44:45.345-01:00 [app_driver_group_stream_api-1] | INFO s.p.k.o.external.thread 345 - [applicationid=String, offset=100, CADM=String,  IPOD=String,  UniqueStringId=String, IMPT=000000-0000-000000-400-00000, applicationname=wwms-processor, Msgid=String, app=app_group_payment, EndToEndID=String, timestamp=17789552323] - app_thread [app_schema-677ghh89hhjjj-appThread-2] Processed 12 total count, run 0 quotations, and completed 0 total apps past the  last update From the above logs i want to get the  min, max, avg, p95 and p99 response_time by app i am not sure how to calculate the response time from the above logs by app.
On a Splunk custom rest API endpoint, I need to get the body of http POST request on the executed python script handling this endpoint. the full rest.py handler script:   # rest.py from server im... See more...
On a Splunk custom rest API endpoint, I need to get the body of http POST request on the executed python script handling this endpoint. the full rest.py handler script:   # rest.py from server import serverless_request from pathlib import Path from splunk.persistconn.application import PersistentServerConnectionApplication import json class App(PersistentServerConnectionApplication): def __init__(self, _command_line, _command_arg): log('init connection', _command_line, _command_arg) super(PersistentServerConnectionApplication, self).__init__() # Handle a syncronous from splunkd. def handle(self, in_string): """ Called for a simple synchronous request. in_string: request data passed in @rtype: string or dict @return: String to return in response. If a dict was passed in, it will automatically be JSON encoded before being returned. """ log(self) log(dir(self)) request = json.loads(in_string.decode()) log("request info", request) log('now proccessing request, hopefully at would be executed by flask') path_info = request['path_info'] if "path_info" in request else '/' method = request['method'] log("request", request) log('sending flask', {"path_info": path_info, method: "method"}) response = serverless_request(path_info, method) payload = response.data if type(payload) is bytes: payload = payload.decode() log('return payload from flask', payload) return {'payload': payload, 'status': 200} def handleStream(self, handle, in_string): """ For future use """ raise NotImplementedError( "PersistentServerConnectionApplication.handleStream") def done(self): """ Virtual method which can be optionally overridden to receive a callback after the request completes. """ pass   when sending a POST request over the custom endpoint with the body    {"isTimeSeriesCollection":true,"collectionName":"333","timeField":"_time","metaField":""}    I would expect the only argument 'in_string' passed to the handler function of `App.handle` to contain information about the body request, but the logs show that the value does not contain any of it:   request info {'output_mode': 'xml', 'output_mode_explicit': False, 'server': {'rest_uri': 'https://127.0.0.1:8089', 'hostname': 'ELIAVS-PC', 'servername': 'Eliavs-PC', 'guid': 'CD4B2374-0104-42C8-A069-F0115A5035DE'}, 'restmap': {'name': 'script:backend', 'conf': {'handler': 'application.App', 'match': '/backend', 'script': 'rest.py', 'scripttype': 'persist'}}, 'path_info': 'new_collection/tsdb', 'query': [], 'connection': {'src_ip': '127.0.0.1', 'ssl': False, 'listening_port': 12211}, 'session': {'user': 'eliav2', 'authtoken': 'ICvMPKZyW3OiN1FV5WE^3^YGOdqGvkpRax7DNB_C6pzoWS53mhj9yEYJH_UwrsJZEK4MH3gUAQh_DNiv0BNOsf4JkVJcjBh5yL1ni1n7LURwQ8a8c6vGvB__qfuTCcs_UIanwMQVmF'}, 'rest_path': '/backend/new_collection/tsdb', 'lang': 'en-US', 'method': 'POST', 'ns': {'app': 'darkeagle'}, 'form': []}     so how can I access the body of the json request? I followed https://dev.splunk.com/enterprise/docs/devtools/customrestendpoints/customrestscript and various other sources to get to this point, the docs are lacking basic information.