About Gabriel

Gabriel · ‎09-14-2023

Hi everyone I got a question regarding the configuration of the app Microsoft Teams Add-on for Splunk. When I configure a Webhook, a TeamsSubscription, and a CallRecord according to this guide, MS Teams data flow into my Splunk instance. Just like the guide suggests, I use ngrok since the server my Splunk instance is running on is not accessible via HTTPS. Ngrok is fine for testing, but I want to switch it out for my actual proxy server. I tried several different settings, but there is no more data coming in. Given that data came in for as long as I used ngrok, all settings related to Azure (Tenant ID, Client ID, Client Secret) must be correct. The issue lies somewhere in the proxy server settings. Can anyone share some insights on how to configure the MS Teams Add-on as well as proxy server settings? Here is my current setup. Webhook - Name: Webhook - Interval: 30 - Index: ms_teams - Port: 4444 Subscription - Name: Subscription - Interval: 86400 - Index: ms_teams - Global Account: MSAzure - Tenant ID: mytenantidfromazure - Environment: Public - Webhook URL: myproxy.server.com <------- splunkinstanceserver.com:4444 or myproxy.server.com? - Endpoint: v1.0 CallRecord - Name: CallRecord - Interval: 30 - Index: ms_teams - Global Account: MSAzure - Tenant ID: mytenantidfromazure - Environment: Public - Endpoint: v1.0 - Max Batch Site: 5000 Proxy - Enable: checked - Host: myproxyserver.com - Port: 4444 <--------- Is this meant to be the port of my webhook or where my proxy takes https requests? - Username: userformyproxyserver - PW: userpwformyproxyserver splunkd.log ***Paths are shortened for readability. .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: WARNING:root:Run function: get_password failed: Traceback (most recent call last): .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: File ".../TA_MS_Teams/bin/ta_ms_teams/aob_py3/solnlib/utils.py", line 148, in wrapper .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: return func(*args, **kwargs) .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: File ".../TA_MS_Teams/bin/ta_ms_teams/aob_py3/solnlib/credentials.py", line 128, in get_password .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: "Failed to get password of realm=%s, user=%s." % (self._realm, user) .../TA_MS_Teams/bin/TA_MS_Teams_rh_settings.py persistent}: solnlib.credentials.CredentialNotExistException: Failed to get password of realm=__REST_CREDENTIAL__#TA_MS_Teams#configs/conf-ta_ms_teams_settings, user=proxy.

Gabriel · ‎08-10-2023

Hi Rick 😃 I edited the indexes.conf file by hand on the server, then restarted splunk. Running the command you suggested also shows that frozenTimePeriodInSecs is equivalent to what btool reports and is active in the runtime: runtime config

Gabriel · ‎08-10-2023

Hi everyone I got a question on the frozenTimePeriodInSecs parameter. Here are my settings inside the indexes.conf file: /opt/splunk/etc/system/local/indexes.conf [_internal] frozenTimePeriodInSecs = 864000 # Data retention set to 10 days. maxTotalDataSizeMB = 750 [_audit] frozenTimePeriodInSecs = 864000 # Data retention set to 10 days. maxTotalDataSizeMB = 750 What I would expect is, that buckets in _internal and _audit where all events are older than 10 days get deleted. However, this is not the case. Anyone knows why? On the other hand, maxTotalDataSizeMB does work as expected. I have checked a couple places for hints why frozenTimePeriodInSecs does not work. The results of those checks are further down below as screenshots. - buckets: Whether there are buckets that contain only events older than 10 days. - btools: Whether the settings are actually taken into account. - monitoring console: Whether the settings are actually taken into account. - _internal logs: Check whether there are freeze events occuring. They only appear for maxTotalDataSizeMB. _audit Buckets _audit btool output monitoring console 1 monitoring console 2 freeze events

Gabriel · ‎04-04-2023

Your solution worked for me perfectly. Thanks!

Gabriel · ‎03-16-2023

I do realize this is an old post, however, I had the same issue of slight fluctuations in search results. During the course of examining this issue, I stumbled upon this yet unanswered question. In my search query, I use the perc() function. Its documentation says the following: The perc and upperperc functions give approximate values for the integer percentile requested. The approximation algorithm that is used, which is based on dynamic compression of a radix tree, provides a strict bound of the actual value for any percentile. This means, sometimes it might be perc50.3, in the next run perc49.6, etc. In my case, this was the cause for the fluctuations I observed. Once I swapped it for a function like avg(), search results were steady.

Gabriel · ‎03-15-2023

I testet it yesterday and it turns out our production environment suppresses dashboards and reports that make use of the mapping command. I had to use subsearches after all. Thanks again for pointing me in the right direction, I am sure the mapping command will come in handy, at some point, even if I can't use it this time 🤓

Gabriel · ‎03-14-2023

I just tried the mapping command. It did precisely what I needed. Thanks a lot, you saved me heaps of time! 💪

Gabriel · ‎03-14-2023

Hi everyone I got the following sample search that yields the table below. index=server | stats avg(response_time) by server_name | sort + avg(response_time) | streamstats count as rank | head 3 rank server_name avg(response_time) new_performance_metric 1 best.server 300 2 second.best.server 350 3 third.best.server 400 Once I know the top servers, I want to calculate additional new_performance_metric for each of the three servers. Does anyone know how this can be done? Note: - I can't use foreach since the metric I want to calculate involves streaming commands. Foreach does not support that. - I think I can't use a subsearch since it is executed first where the top servers are not known yet. - I can't precompute the new_performance_metric for all servers and then use something like a lookup since this is computationally too expensive. My guess is that the solution involves a macro but I couldn't figure it out yet. Many thanks in advance.

Gabriel · ‎03-09-2023

Thanks for the hint! Tracking the files (by means of du) would then let me understand what makes up disk usage.

Gabriel · ‎03-07-2023

Hello everyone Is there a way to determine what occupies disk storage? The following SPL yields a line graph that shows disk utilization per host and each of its drives. index=winperf_prod sourcetype=*Perfmon* storage_free_percent="*" | eval storage_used_percent=round(100-storage_free_percent,2) | eval host_dev=printf("%s:%s\\",host,instance) | timechart max(storage_used_percent) by host_dev Now I want to determine what the drop in one of the drives cause on February 12. Ideally I get a list of all processes/files/etc. that occupy disk storage. Help is appreciated 😃

Gabriel · ‎12-04-2022

Hi @pdrieger_splunk I even tested that as well but must have done something wrong in the process. Thanks for your quick and detailed help, greatly appreciated! 🎉

Gabriel · ‎12-02-2022

Hi @pdrieger_splunk First off, thanks for your reply 😃 1. Yes, I used pythons pickle library (import pickle). I appended pictures of the codes. 2. After I load the model, it seems to be exactly the same as before saving it. Also checkout the picktures appended. 3. Thanks for the heads up. I use DSDL for practicing purposes and kmeans due to it being relatively simple and unsupervised. 4. Am I right to assume reporting an issue here https://github.com/splunk/splunk-mltk-container-docker would be correct?

Gabriel · ‎12-01-2022

Hello everyone I am using the DSDL app: https://splunkbase.splunk.com/app/4607 The model I use is sklearn's kmeans: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html My goal is to cluster a dataset with kmeans and then assign new observations to center points form my kmeans model. I built the model and everything compiled just fine. I then wanted to check my model for possible logical errors. To this end, I use the same dataset in the | fit and | apply command: | inputlookup wineventlog.csv | fit MLTKContainer algo=Pipeline_V3 k=5 fe_* into app:pipeline_v3 | inputlookup wineventlog.csv | apply pipeline_v3 To my understanding, this should yield the exact same results. However, it does not. | fit creates 5 clusters and assigns observations to each cluster. | apply on the other hand only assigns observations to 3 out of the 5 clusters. Does anyone have a precise idea what goes on behind the scenes with | fit and | apply? I already checked the documentation that outlines what they both do: https://docs.splunk.com/Documentation/MLApp/5.3.3/User/Understandfitandapply I thoroughly checked whether I have null values, non-numeric fields that might get converted, etc. but I could not figure out why fit & apply wouldn't yield the same result. Following the code I use for each respective command. def fit(model, df, param): # Number of clusters k = int(param["options"]["params"]["k"]) # Fit kmeans model kmeans = KMeans(n_clusters=k, random_state=0).fit(df) model["kmeans"] = kmeans return model def apply(model, df, param): # Assign new observations to kmeans centers predictions = model["kmeans"].predict(df) return predictions

Posts	14
Solutions	0
Karma Given	13
Karma Received	1
Member Since	‎11-29-2022

Online Status	Offline
Date Last Visited	a month ago

Microsoft Teams Add-on for Splunk

Why frozenTimePeriodInSecs does not take effect?

How to add column values based on subsearch?

Disk Usage: How to show memory consuming files?

DSDL - Kmeans yields different results for fit and...

Microsoft Teams Add-on for Splunk

Re: frozenTimePeriodInSecs does not take effect

Why frozenTimePeriodInSecs does not take effect?

Re: Prevent any processing until Submit button was...

Re: Same search returns different results each tim...

Re: Add column values based on subsearch

Re: Add column values based on subsearch

How to add column values based on subsearch?

Re: Disk Usage: How to show memory consuming files...

Disk Usage: How to show memory consuming files?

Re: DSDL - Kmeans yields different results for fit...

Re: DSDL - Kmeans yields different results for fit...

DSDL - Kmeans yields different results for fit and...