As in previous posts I am talking about using variables or tokens in the Contributing Events part of enterprise security.   1. When I use $host$ it substitutes the actual Splunk Host instead of the host returned from the correlation search.  2. Can someone provide Splunk documentation links for creating or using Variables in the Drilldown search or contributing events search? 3. I am requesting and have requested in Splunk Ideas "TO MAKE THE **bleep** DRILL DOWN SEARCH WINDOW A MULTI-LINED TEXT BOX" since Splunk Enterprise Security version 6.0 and none of the GUI issues were not addressed!  HELP ME OBI-WAN!!! 

[Filter: smut] lugoon's post body matched "damn", board "security-splunk-enterprise-security". Post Subject: More Enterprise Security Correlation Search Variable Substitution for Contributing Events Post Body: As in previous posts I am talking about using variables or tokens in the Contributing Events part of enterprise security.   1. When I use $host$ it substitutes the actual Splunk Host instead of the host returned from the correlation search.  2. Can someone provide Splunk documentation links for creating or using Variables in the Drilldown search or contributing events search? 3. I am requesting and have requested in Splunk Ideas "TO MAKE THE DAMN DRILL DOWN SEARCH WINDOW A MULTI-LINED TEXT BOX" since Splunk Enterprise Security version 6.0 and none of the GUI issues were not addressed!  HELP ME OBI-WAN!!!  Body text "DAMN" matched filter pattern "damn". Post by User[id=199398,login=lugoon] was rejected for the following end-user facing error(s): This board requires at least one label for each message.

I am still getting information from all of the servers that have the universal forwarders on them and verified the service is running, but am still getting "missing forwarders" alert setup from initial setup search. Not sure what is going on. Also, looking at splunk logs for errors, found on each server the powershell script was failing inside: C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\bin\powershell\dns-zoneinfo.ps1 Get-WMIObject : Invalid namespace "root\MicrosoftDNS" At C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\bin\powershell\dns-zoneinfo.ps1:75 char:10 + $Zones = Get-WMIObject -Computer $ServerName -Namespace "root\Microso ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [Get-WmiObject], ManagementException + FullyQualifiedErrorId : GetWMIManagementException,Microsoft.PowerShell.Commands.GetWmiObjectCommand

  Microsoft Office 365 Reporting Web Service works fine with an "Index Once" config where Start date/time & End date/time are defined. Set this to Continuously Monitor and it appears to fail...  This connector is defaults with empty start or end date/time fields 2022-10-21 13:36:54,969 INFO pid=15262 tid=MainThread file=splunk_rest_client.py:_request_handler:99 | Use HTTP connection pooling 2022-10-21 13:36:54,970 DEBUG pid=15262 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer (body: {}) 2022-10-21 13:36:54,971 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_new_conn:941 | Starting new HTTPS connection (1): 127.0.0.1:8089 2022-10-21 13:36:54,973 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_make_request:442 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 5564 2022-10-21 13:36:54,974 DEBUG pid=15262 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/ (body: {'count': -1, 'offset': 0, 'search': 'TA_MS_O365_Reporting_checkpointer'}) 2022-10-21 13:36:54,974 DEBUG pid=15262 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.003694 2022-10-21 13:36:54,976 DEBUG pid=15262 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.002273 2022-10-21 13:36:54,976 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_make_request:442 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/?count=-1&offset=0&search=TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 4716 2022-10-21 13:36:54,978 DEBUG pid=15262 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/O365_Message_Trace_obj_checkpoint_oauth (body: {}) 2022-10-21 13:36:54,979 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_make_request:442 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/O365_Message_Trace_obj_checkpoint_oauth HTTP/1.1" 404 140 2022-10-21 13:36:54,980 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_new_conn:941 | Starting new HTTPS connection (1): login.windows.net:443 2022-10-21 13:36:54,980 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Proxy is enabled: web:8080 2022-10-21 13:36:54,980 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Getting proxy server. 2022-10-21 13:36:54,980 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ message trace URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2022-10-16T13:36:54.979985Z' and EndDate eq datetime'2022-10-16T14:36:54.979985Z' 2022-10-21 13:36:54,980 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Start date: 2022-10-16 13:36:54.979985, End date: 2022-10-16 14:36:54.979985 2022-10-21 13:36:55,142 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_make_request:442 | https://login.windows.net:443 "POST /2445612c-659f-4f0e-a8b2-51087c624102/oauth2/token HTTP/1.1" 200 1815 2022-10-21 13:36:55,144 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Proxy is enabled: web:8080 2022-10-21 13:36:55,144 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Getting proxy server. 2022-10-21 13:36:55,145 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_new_conn:941 | Starting new HTTPS connection (1): reports.office365.com:443 2022-10-21 13:36:59,928 DEBUG pid=15262 tid=MainThread file=connectionpool.py:_make_request:442 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-10-16T13:36:54.979985Z'%20and%20EndDate%20eq%20datetime'2022-10-16T14:36:54.979985Z' HTTP/1.1" 200 216 2022-10-21 13:36:59,930 DEBUG pid=15262 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ max date before getting message: 2022-10-16 13:36:54.979985 I changed the Start date/time 2022-10-19 00:00:00 2 full days ago, so I don't bump against the 7 day boundary. 2022-10-21 13:40:31,102 DEBUG pid=15810 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ message trace URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2022-10-19T00:00:00Z' and EndDate eq datetime'2022-10-19T01:00:00Z' 2022-10-21 13:40:31,102 DEBUG pid=15810 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Start date: 2022-10-19 00:00:00, End date: 2022-10-19 01:00:00 2022-10-21 13:40:31,103 DEBUG pid=15810 tid=MainThread file=connectionpool.py:_new_conn:941 | Starting new HTTPS connection (1): login.windows.net:443 2022-10-21 13:40:31,339 DEBUG pid=15810 tid=MainThread file=connectionpool.py:_make_request:442 | https://login.windows.net:443 "POST /2445612c-659f-4f0e-a8b2-51087c624102/oauth2/token HTTP/1.1" 200 1815 2022-10-21 13:40:31,341 DEBUG pid=15810 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Proxy is enabled: web:8080 2022-10-21 13:40:31,341 DEBUG pid=15810 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Getting proxy server. 2022-10-21 13:40:31,342 DEBUG pid=15810 tid=MainThread file=connectionpool.py:_new_conn:941 | Starting new HTTPS connection (1): reports.office365.com:443 2022-10-21 13:40:34,302 DEBUG pid=15810 tid=MainThread file=connectionpool.py:_make_request:442 | https://reports.office365.com:443 "GET /ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-10-19T00:00:00Z'%20and%20EndDate%20eq%20datetime'2022-10-19T01:00:00Z' HTTP/1.1" 200 122 2022-10-21 13:40:34,303 DEBUG pid=15810 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ max date before getting message: 2022-10-19 00:00:00 I've not been able to determine what the comment "_Splunk_ max date before getting message: <2022-10-19 00:00:00>" The lookup TA_MS_O365_Reporting_checkpointer shows a row with _key <nameofinput>_once_checkpoint_oauth which looks to be from when I did the Index Once. Would some who's running Continuously Monitor please take a look into lookup TA_MS_O365_Reporting_checkpointer & let me know what _key name & state columns indicate for where _key = *_checkpoint_* Of course, if someone has experienced the same & figured this out, I'd appreciate any words of wisdom.  

I have two independent/unrelated queries (same index, though) , and I want to create a timechart where there are two bars in each time bucket, one for each of the two queries. Is this possible? Thanks! Jonathan

Our application logs for each method: when it begins, when it ends, and the thread it is on.  We are wanting to visualize how long each method takes.  The logging structure is as follows:  [Thread #] Timestamp Begin/End MethodName Example:  [Thread-13569 (ActiveMQ-client-global-threads)] Fri Oct 21 14:29:00 EDT 2022 Begin purgeHistory(Connection, String) [Thread-13569 (ActiveMQ-client-global-threads)] Fri Oct 21 14:29:00 EDT 2022 End purgeHistory(Connection, String) So we need a way to match every method "Begin" with every method "End" that's on the same thread, to be able to calculate and display how long each method took to execute. Is there any way we could get some help tackling this query? 

Hello there! I've been ingesting data from Azure Storage Explorer via the Splunk Add-On for Microsoft Cloud Services app, however, I now wish to ingest data from an Azure Event Hub.  I know I can either create an input in the same app or use the Microsoft Azure Add-on for Splunk app.  Is there a way to specify which partition to collect data from?  Furthermore, is there a way to send data to different indexes and sourcetypes from 1 Event Hub? I'm working on Splunk Cloud, so I currently don't have access to config files. Thanks in advance!

    const splunkjs = require('splunk-sdk'); const service = new splunkjs.Service({ host: "xxxxxx", port: xxxx, username: "administrator", password: "xxxxxx", scheme: "https", version: "default" } ); const init = new splunkjs.Service service.login(function(err, success) { if (err) { throw err; } else { console.log('Logged in successfully'); } } );         Hey,  I was trying to use the SDK to login remotely however the server keeps returning an error message as shown below. I even tried using postman on the local splunk server but it gives the same error as well.      { response: { headers: {}, statusCode: 600 }, status: 600, data: undefined, error: Error: write EPROTO 22680:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:c:\ws\deps\openssl\openssl\ssl\record\ssl3_record.c:332: at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) { errno: -4046, code: 'EPROTO', syscall: 'write' } }        However on the local server I am able to login just fine with the code shown below however I really don't want to do development on the splunk server directly as its quite a pain     const service = new splunkjs.Service({ username: "xxx", password: "xxx" } );      

In PAN tarffic, where we are trying to exclude events with "block-untst-rule" string. PAN traffic logs are configured to receive in syslog server. I tried to use filter in syslog-ng config as shown below.  filter exclogs { not match("block\\-untst\\-rule"); }; log {source(s_syslog_pa_cloud); filter {exclogs}; destination(d_syslog_pa_cloud); }; Other way we tried is by using props.conf and transforms.conf props.conf [pan:traffic] TRANSFORMS-set = setnull transforms.conf [setnull] REGEX = block\\-untst\\-rule DEST_KEY = queue FORMAT = nullQueue Both ways we are not able to exclude the events from ingesting. Please do assist.

I am struggling to get the MS O365 reporting app working with OAuth. Here's the error in ta_ms_o365_reporting_ms_o365_message_trace_oauth.log: 2022-10-21 16:24:38,096 INFO pid=91299 tid=MainThread file=splunk_rest_client.py:_request_handler:99 | Use HTTP connection pooling 2022-10-21 16:24:38,097 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer (body: {}) 2022-10-21 16:24:38,100 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_new_conn:975 | Starting new HTTPS connection (1): 127.0.0.1:8089 2022-10-21 16:24:38,122 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 5562 2022-10-21 16:24:38,123 DEBUG pid=91299 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.025611 2022-10-21 16:24:38,124 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/ (body: {'count': -1, 'offset': 0, 'search': 'TA_MS_O365_Reporting_checkpointer'}) 2022-10-21 16:24:38,136 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/config/?count=-1&offset=0&search=TA_MS_O365_Reporting_checkpointer HTTP/1.1" 200 4714 2022-10-21 16:24:38,137 DEBUG pid=91299 tid=MainThread file=binding.py:new_f:74 | Operation took 0:00:00.013834 2022-10-21 16:24:38,141 DEBUG pid=91299 tid=MainThread file=binding.py:get:695 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth (body: {}) 2022-10-21 16:24:38,149 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://127.0.0.1:8089 "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth HTTP/1.1" 404 140 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Start date: 2022-10-16 16:24:38.151205, End date: 2022-10-16 17:24:38.151205 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ message trace URL: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2022-10-16T16:24:38.151205Z' and EndDate eq datetime'2022-10-16T17:24:38.151205Z' 2022-10-21 16:24:38,151 DEBUG pid=91299 tid=MainThread file=base_modinput.py:log_debug:298 | _Splunk_ Getting proxy server. 2022-10-21 16:24:38,151 INFO pid=91299 tid=MainThread file=setup_util.py:log_info:142 | Proxy is not enabled! 2022-10-21 16:24:38,154 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_new_conn:975 | Starting new HTTPS connection (1): login.windows.net:443 2022-10-21 16:24:38,284 DEBUG pid=91299 tid=MainThread file=connectionpool.py:_make_request:461 | https://login.windows.net:443 "POST /[TENANT ID]/oauth2/token HTTP/1.1" 400 747 2022-10-21 16:24:38,290 ERROR pid=91299 tid=MainThread file=base_modinput.py:log_error:316 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/lib/splunktaucclib/modinput_wrapper/base_modinput.py", line 140, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 362, in collect_events get_events_continuous(helper, ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 98, in get_events_continuous if 'value' in message_response: TypeError: argument of type 'NoneType' is not iterable I've highlighted the requests that got a HTTP repsonse code of 200 in blue, and the 404 and 400 errors in red. If I use curl to access the URL with the 404 error I get: <?xml version="1.0" encoding="UTF-8"?> <response> <messages> <msg type="ERROR">Could not find object.</msg> </messages> </response> It looks like this is trying to access a "storage" directory inside /opt/splunk/etc/apps/TA-MS_O365_Reporting, which doesn't exist Looking at splunkd_access.log, when running curl, I get: 127.0.0.1 - splunk-system-user [21/Oct/2022:16:53:37.043 +0100] "GET /servicesNS/nobody/TA-MS_O365_Reporting/storage/collections/data/TA_MS_O365_Reporting_checkpointer/Testing_obj_checkpoint_oauth HTTP/1.1" 404 140 "-" "curl" - - - 5ms What I've tried and checked: Splunk is running as the splunk user I've checked and reset the filesystem permissions for /opt/splunk I've deleted and recreated the inputs and the accounts I've deleted the app and re-installed, both through the CLI and Splunk Web Things that are less likely to be related, but I've done anyway: I've created new credentials in Azure for my enterprise app I've checked the permissions in Azure and confirmed that the permissions have been correctly granted The enterprise app in Azure is in the Global Reader and Exchange Administrator roles I'm out of ideas so any help is gratefully received! I'm giessing that resolving the 404 may resolve the 400 to. Splunk v. 9.0.1 Splunk Add-on for Microsoft Office 365 Reporting Web Service v. 2.0.1 Ubuntu 20.04.5 LTS on VMware 7 20 CPUs, 24GB RAM Thank you!        

Hello, Quick question. How do I change the default number of lines to return in search? Is there a setting in limits.conf?   index=_audit action=search AND search!=*_internal* AND search!=*_audit* AND user="user1"   The linecount max is 128. My larger search values return with ...(truncated)', autojo... in the _raw. Thanks in advance and God bless.

Hello all, I am trying to solve an issue with your addon for Splunk. Our client has an indice with name *:security-audit-*. Since we have a distributed environment we have heavy forwarders in cloud and indexers and searcheads in the on prem environment. The URL is resolvable via nslookup and the endpoint is giving us a correct response if we try to connect to it via our username and password. As you can see its transforming the : character into %3. Now, I am not sure if this is an actual issue, but our on-prem team is not receiving any data. I also tried using *security-audit-* which solved the errors but still no data has been recieved. [elasticsearch_json://srvadm] ca_certs_path = /opt/splunk/etc/auth/VWAG date_field_name = @timestamp elasticsearch_indice = *:security-audit-* elasticsearch_instance_url = https://redacted:9243 greater_or_equal = {{ ansible_date_time.date }} index = vw_de_aws_mlaas_apps interval = 300 lower_or_equal = now secret = {{ es_password }} use_ssl = 1 user = siem_readonly verify_certs = 0   root@fpea:/var/snap/amazon-ssm-agent/6312# cat /opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log | grep security-audit 2022-10-21 11:52:38,943 WARNING pid=811606 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.032s] 2022-10-21 11:57:40,619 WARNING pid=814893 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.095s] 2022-10-21 12:02:41,357 WARNING pid=818040 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.091s] 2022-10-21 12:07:39,512 WARNING pid=820807 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.066s] 2022-10-21 12:12:46,422 WARNING pid=823706 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.036s]  

Hi, I have a list of hosts/devices say from HostA to HostZ (PS: its not  a lookup file) I want to find out which host among the list show up in the particular index, say Index=IndexA. Could someone help me on this?

I am trying to group range of decimal number: Range between 10.0.0 and10.15 =Medium 10.16 -11=High 11.1-11.5=critical  for example: Severity 10.15.4 12.6 12.6.0 10.15.7 10.15.7 10.15.7 12.6 12.6 10.15.7 12.5.1 12.6 12.6.0 10.15.7 12.6 12.6.0 11.0 12.5.1 11.0 12.6 12.6.0 11.0.1 12.3.1 12.6 12.6.0 11.2 12.6 12.6.0 11.2.3 11.6.3 12.2.1 12.6 12.6.0 11.2.3 12.6 12.6.0 11.4 12.5.1 11.4 12.5.1 12.5.1 12.6 12.6 11.4 12.6 12.6.0 11.5.1 12.6 12.6.0 11.5.2 11.6.2 12.6 12.6.0 11.5.2 11.7 11.7.0 11.5.2 12.2.1 12.5.1 11.5.2 12.2.1 12.6 12.6.0 11.5.2 12.6 12.6.0 11.6 11.6.0 11.6 11.6.2 12.2.1 12.6 12.6.0 11.6 11.6.2 12.3.1

I have three graphs that show results based on a global time range. However, if I have no results (no errors) the third graph is not displayed. I just want to display an empty graph with the same date ranges as the other graphs without displaying artificially inserted results that could be intepreted as errors. None of the many posts regarding this issue solved my problem as far as I understand it. Thanks in advance.