Hi all, Maybe a dummy question, do I need to setup Universal Forwarder on Splunk server to monitor and index data? (so it's like the server is forwarding data to itself) I tested setup an app in etc/apps/ with below config but it doesn't work. inputs.conf   [batch:///opt/splunk/temp/test_forward/*] move_policy = sinkhole disabled = 0 index = test sourcetype = test crcSalt = test _TCP_ROUTING = test   outputs.conf   [indexAndForward] index = false [tcpout] indexAndForward = false maxQueueSize = 200MB [tcpout:test] server = <server IP>:9997   Thanks ... View more

Hi all, I'm having issue with this add-on "Microsoft Teams Add-on for Splunk". I currently use the add-on to get Teams Call Details Record downloaded to Splunk. For small call, this run well without issue. However, for big meeting when there are 200+ participants, call details record will be divided into several pages, and the URL to download next page is via "@odata.nextLink". But this add-on doesn't seem to download next page, it stops after downloading the first page and therefore not all participants' details are downloaded to Splunk. Via powershell, i can confirm that Graph API did return the "@odata.nextLink" (which was missing before, they only fixed it recently). Look into python code,  the add-on uses function "get_item" to fetch the data, and this function doesn't care about nextLink at all, which explains why I encountered the issue. Below is the code of this get_item function   def get_item(helper, access_token, url): headers = {} headers["Authorization"] = "Bearer %s" % access_token headers["Content-type"] = "application/json" proxies = get_proxy(helper, "requests") try: r = requests.get(url, headers=headers, proxies=proxies) r.raise_for_status() response_json = None response_json = json.loads(r.content) item = response_json except Exception as e: raise e return item     I found another function within the same library that does look into nextLink data, which is get_items   def get_items(helper, access_token, url, items=[]): headers = {} headers["Authorization"] = "Bearer %s" % access_token headers["Content-type"] = "application/json" proxies = get_proxy(helper, "requests") try: r = requests.get(url, headers=headers, proxies=proxies) if r.status_code != 200: return items r.raise_for_status() response_json = None response_json = json.loads(r.content) items += response_json['value'] if '@odata.nextLink' in response_json: nextLink = response_json['@odata.nextLink'] # This should never happen, but just in case... if not is_https(nextLink): raise ValueError("nextLink scheme is not HTTPS. nextLink URL: %s" % nextLink) helper.log_debug("_Splunk_ nextLink URL (@odata.nextLink): %s" % nextLink) get_items(helper, access_token, nextLink, items) except Exception as e: raise e return items     So, what I did was changing the code from   call_record = azutils.get_item(helper, access_token, url)     to   call_record = azutils.get_items(helper, access_token, url)   but it doesn't work. Anyone know how to get around this?   Thanks a lot ... View more

I downloaded and installed Teams Add-on for Splunk and it worked for a while, until we encountered a lot of 404 error like below ERROR pid=14248 tid=MainThread file=base_modinput.py:log_error:309 | Error getting callRecord data: 404 Client Error: Not Found for url: https://graph.microsoft.com/v1.0/communications/callRecords/<call ID>?$expand=sessions($expand=segments)   I found out that the callID was removed from Teams CDR for some reason,  therefore when Splunk tried to download the CDR, it returned error 404, which is understandable. However Teams Add-on will not remove the Call ID from webhook directory for this scenario. The call ID will remain there forever and Splunk will keep on trying again and again to download the CDR and failed. This results in a huge amount of call IDs that never get cleaned up and massive number of error messages in the log. Further more, i found out that if there were too many call ID files exist in the wehbook directory (~60K), the Add-on will encountered error "401 Unauthorized to download the CDR" and stopped working soon afterward. After restarting Splunk, the Add-on worked again and then stopped the  moment it hit 401 error again. I manually created a script to manage the load of webhook folder, so this is OK for now but it would be preferable that the Add-on has load management feature by itself. Hopefully the author of this Add-on will add this error handling soon, but meanwhile if anyone knows how to get around this 404 issue please kindly share. Thanks a lot! ... View more

Hi all, I have this json file like below:   rootfield: [[-] {[-] field 1: A field 2: [[-] value1 value2 ] } {[-] field 1: B field 2: [[-] value1 value2 value3 value4 ] } {[-] field 1: C field 2: [[-] value1 value3 value4 ] }   I want to extract the result to look like this A_value1,value2 B_value1,value2,value3,value4 C_value1,value3,value4 i know combination of spath and mvexpand will work but I want to avoid mvexpand because it's eating memory and our json file is huge. Would be nice to be able to get the data without mvexpand. Thanks   ... View more

Hi all, When I was trying to setup a new connection in DBConnect 3.2, i got this error message java.sql.SQLException: Error while saving schema to file: java.io.FileNotFoundException  anyone knows how to fix it? Thanks ... View more

Hi all, I have problem getting Splunk to connect to MongoDB. Below are what i have done so far: 1. Download the driver from https://raw.githubusercontent.com/michaelloliveira/traccar-mongodb/master/lib/mongodb_unityjdbc_full... and copy it to drivers folder of dbconnect app 2. copy db_connection_types.conf from default to local folder, and add following line [mongo] displayName= MongoDB jdbcDriverClass = mongodb.jdbc.MongoDriver serviceClass = com.splunk.dbx2.DefaultDBX2JDBC jdbcUrlFormat = jdbc:mongo://<host>:<port>/<database> #jdbcUrlSSLFormat = jdbc:mongo://<host>:<port>/<database> useConnectionPool = false port = 27017 testQuery = SELECT 1 [mongo2] displayName= MongoDB2 serviceClass = com.splunk.dbx2.DefaultDBX2JDBC jdbcUrlFormat = jdbc:mongodb://<host>:<port>/<database> #jdbcUrlSSLFormat = jdbc:mongodb://<host>:<port>/<database> jdbcDriverClass = mongodb.jdbc.MongoDriver port = 27017 3. Create identity for the connection. This account has readWrite permission on the database and tested working. 4. Restart Splunk. I can see driver can be detected for both MongoDB connection types 5. Create connection that use either MongoDB or MongoDB2. Both failed Error for MongoDB: not authorized for query on Test._schema Error for MongoDB2: No suitable driver found for jdbc:mongodb://si0vmc3134.de.bosch.com:30000/Test   Anyone know what's wrong here? Thanks ... View more

Hi all, I'm trying to pull data from Azure Log Analytics workspace to Splunk. I have installed the add-on Microsoft Log Analytics Add-on (https://splunkbase.splunk.com/app/4127/) . When I checked the log, this is what I see 2020-06-19 11:23:36,446 INFO pid=85670 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1 2020-06-19 11:23:37,263 INFO pid=85670 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1 2020-06-19 11:23:38,694 INFO pid=85670 tid=MainThread file=splunk_rest_client.py:_request_handler:100 | Use HTTP connection pooling 2020-06-19 11:23:38,694 DEBUG pid=85670 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-ms-loganalytics/s torage/collections/config/TA_ms_loganalytics_checkpointer (body: {}) 2020-06-19 11:23:38,695 INFO pid=85670 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1 2020-06-19 11:23:38,699 DEBUG pid=85670 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-ms-loganalytics/storage/collecti ons/config/TA_ms_loganalytics_checkpointer HTTP/1.1" 200 5632 2020-06-19 11:23:38,699 DEBUG pid=85670 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.005300 2020-06-19 11:23:38,700 DEBUG pid=85670 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-ms-loganalytics/s torage/collections/config/ (body: {'offset': 0, 'count': -1, 'search': 'TA_ms_loganalytics_checkpointer'}) 2020-06-19 11:23:38,702 DEBUG pid=85670 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-ms-loganalytics/storage/collecti ons/config/?offset=0&count=-1&search=TA_ms_loganalytics_checkpointer HTTP/1.1" 200 4830 2020-06-19 11:23:38,702 DEBUG pid=85670 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.002460 2020-06-19 11:23:38,704 DEBUG pid=85670 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-ms-loganalytics/s torage/collections/data/TA_ms_loganalytics_checkpointer/AzureLogAnalytic (body: {}) 2020-06-19 11:23:38,706 DEBUG pid=85670 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-ms-loganalytics/storage/collecti ons/data/TA_ms_loganalytics_checkpointer/AzureLogAnalytic HTTP/1.1" 404 140 2020-06-19 11:23:38,708 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - Authority:Performing instance discovery : https://login.microsoftonline.com/0ae51e19-07c8-4e4b-bb6d-648ee58410f4 2020-06-19 11:23:38,708 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - Authority:Performing static instance di scovery 2020-06-19 11:23:38,708 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - Authority:Authority validated via stati c instance discovery 2020-06-19 11:23:38,709 INFO pid=85670 tid=MainThread file=log.py:info:103 | 16e5bba7-a023-431f-9813-396e814eabc9 - TokenRequest:Getting token with client cr edentials. 2020-06-19 11:23:38,709 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - TokenRequest:No user_id passed for cach e query 2020-06-19 11:23:38,709 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - OAuth2Client:finding with query: {"_clientId": "dce1fe27-225d-4615-bcee-d22ff8071a0f"} 2020-06-19 11:23:38,709 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - OAuth2Client:Looking for potential cache entries: 2020-06-19 11:23:38,709 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - OAuth2Client:{"_clientId": "dce1fe27-225d-4615-bcee-d22ff8071a0f"} 2020-06-19 11:23:38,709 DEBUG pid=85670 tid=MainThread file=log.py:debug:108 | 16e5bba7-a023-431f-9813-396e814eabc9 - OAuth2Client:Found 0 potential entries. 2020-06-19 11:23:38,713 DEBUG pid=85670 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): login.microsoftonline.com 2020-06-19 11:23:38,716 INFO pid=85670 tid=MainThread file=log.py:info:103 | 16e5bba7-a023-431f-9813-396e814eabc9 - OAuth2Client:Get Token request failed 2020-06-19 11:23:38,718 ERROR pid=85670 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/modinput_wrapper/base_modinput.py", line 127, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/log_analytics.py", line 96, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/input_module_log_analytics.py", line 49, in collect_events token_response = context.acquire_token_with_client_credentials('https://api.loganalytics.us/', application_id, application_key) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/authentication_context.py", line 160, in acquire_token_with_client_credentials return self._acquire_token(token_func) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/authentication_context.py", line 109, in _acquire_token return token_func(self) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/authentication_context.py", line 158, in token_func return token_request.get_token_with_client_credentials(client_secret) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/token_request.py", line 316, in get_token_with_client_credentials token = self._oauth_get_token(oauth_parameters) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/token_request.py", line 113, in _oauth_get_token return client.get_token(oauth_parameters) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/adal/oauth2_client.py", line 262, in get_token verify=self._call_context.get('verify_ssl', None)) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/requests/api.py", line 110, in post return request('post', url, data=data, json=json, **kwargs) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/requests/api.py", line 56, in request return session.request(method=method, url=url, **kwargs) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/requests/sessions.py", line 488, in request resp = self.send(prep, **send_kwargs) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/opt/splunk/etc/apps/TA-ms-loganalytics/bin/ta_ms_loganalytics/requests/adapters.py", line 487, in send raise ConnectionError(e, request=request) ConnectionError: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /0ae51e19-07c8-4e4b-bb6d-648ee58410f4/oauth2/token?api-version=1.0 (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f6e3a7aaa10>: Failed to establish a new connection: [Errno -2] Name or service not known',)) Anyone has any idea how to solve this issue? Thanks   ... View more