Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
All Posts
Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- Find Answers
- :
- All Posts
All Posts
02-17-2024
06:37 AM
I have already send a support request to MS Office365. Hope that they can give a better or more detailed reason then "SendAsDenied; ticket@eremote.nl not allowed to send as Splunk_eRemote@uBDC01;" ...
See more...
I have already send a support request to MS Office365. Hope that they can give a better or more detailed reason then "SendAsDenied; ticket@eremote.nl not allowed to send as Splunk_eRemote@uBDC01;" In the mean time I will follow also my Splunk support-case route... Will post my findings here, later....
02-17-2024
06:32 AM
The sendemail.py is apparently trying to send but the server is rejecting the email. It's easiest to check at the denying party's logs why it's happening. If really nothing has changed recently on ei...
See more...
The sendemail.py is apparently trying to send but the server is rejecting the email. It's easiest to check at the denying party's logs why it's happening. If really nothing has changed recently on either side (are you absolutely sure there was no change in policies in the mail environment?), maybe it's simply a case of over-quota recipient mailbox.
02-17-2024
06:26 AM
Ok, firstly, we seem to be mixing limits. 50000 is the default limit for subsearch used by join command. The limit for subsearch is 10000 results. But as I understand the wording from the limits.con...
See more...
Ok, firstly, we seem to be mixing limits. 50000 is the default limit for subsearch used by join command. The limit for subsearch is 10000 results. But as I understand the wording from the limits.conf spec, it applies to the number of results returned by the search, not the initial events processed by the first part of the pipeline. I'll have to test it.
02-17-2024
04:51 AM
Currently I am feeding Splunk Zeek logs (formerly known as bro) via the monitor command. Some of the logs in the Zeek index are being parsed correctly. Other logs, however, are still appearing as raw...
See more...
Currently I am feeding Splunk Zeek logs (formerly known as bro) via the monitor command. Some of the logs in the Zeek index are being parsed correctly. Other logs, however, are still appearing as raw text. I remember in the past there was a certain link in the settings where I could specify how to extract each field in the event what to call the field and what data belonged to it. I also remember being able to test the specific settings I was applying via a log of the same index/source type. Any help interpreting what I am trying to communicate or guidance as to finding that specific page I am looking for is very much appreciated.
Labels
- Labels:
-
heavy forwarder
-
transforms.conf
02-17-2024
04:47 AM
The limit is to do with the events not the result i.e. the number of events returned by the first part of the subsearch (before first pipe), so, as you have already stated, you had more than 50k even...
See more...
The limit is to do with the events not the result i.e. the number of events returned by the first part of the subsearch (before first pipe), so, as you have already stated, you had more than 50k events to get your 250+ results. You need to reframe this initial part of the subsearch so that fewer than 50k events are found.
02-17-2024
04:30 AM
Yes it does when using:
| makeresults
| sendemail to="aaa@ccc", from="bbb@eee.nl", subject="Testing Email from Splunk / from: uBDC01", use_ssl=false, use_tls=true
or as form in a Dashboard we...
See more...
Yes it does when using:
| makeresults
| sendemail to="aaa@ccc", from="bbb@eee.nl", subject="Testing Email from Splunk / from: uBDC01", use_ssl=false, use_tls=true
or as form in a Dashboard we use to create a ticket for our customer. Only alerting has stop working recently? BTW: we use OFFICE365 so this should be occuring elsewhere too, do not you think?. And no errors-essaging is showing in Splunk (as user or as admin) when this happens. So you suggest to look in the O365 log of the exchange environment?
02-17-2024
04:26 AM
This is a message returned by the other end of the smtp transaction so it can contain anything. Next thing I'd check the smtp server's logs for the reason for rejection because it's the mail server t...
See more...
This is a message returned by the other end of the smtp transaction so it can contain anything. Next thing I'd check the smtp server's logs for the reason for rejection because it's the mail server that's not accepting messages from you.
02-17-2024
04:20 AM
It seems, according to https://docs.splunk.com/observability/en/gdi/opentelemetry/components/smartagent-receiver.html#smartagent-receiver that collectd monitors are indeed not supported on Windows.
02-17-2024
04:07 AM
Hi PickleRick, Thanks, 1) Yes, I have tried many thing, also by using the opposite condition: alert_condition = search mincount < 7 2) In all the logs I have checked (including sendemail.py, ...
See more...
Hi PickleRick, Thanks, 1) Yes, I have tried many thing, also by using the opposite condition: alert_condition = search mincount < 7 2) In all the logs I have checked (including sendemail.py, which does not log anything in my opinion) and searched, it is/was successful, status: success, not skipped or otherwise found to be delayed or ??? 3) example spl: index=_* AND alert_actions="email" | stats count by status Output: 42 Success over last 24h 02-17-2024 12:12:17.264 +0100 INFO SavedSplunker - savedsearch_id="nobody;search;Alert_trigger_1v1", search_type="scheduled", search_streaming=0, user="admin", app="search", savedsearch_name="Alert_trigger_1v1", priority=default, status=success, digest_mode=0, durable_cursor=0, scheduled_time=1708168200, window_time=0, dispatch_time=1708168333, run_time=0.142, result_count=1, alert_actions="email", sid="scheduler__admin__search__RMD5ea1ed26b5154d33f_at_1708168200_47", suppressed=0, fired=1, skipped=0, action_time_ms=3528, thread_id="AlertNotifierWorker-0", message="", workload_pool="" So no error message to be found sofar?? What do I miss here. Although I also searched in python.log and I found part of many messages: 2024-02-17 12:32:17,265 +0100 ERROR sendemail:572 - (554, b'5.2.252 SendAsDenied; ticket@eremote.nl not allowed to send as Splunk_eRemote@uBDC01; STOREDRV.Submission.Exception:SendAsDeniedException.MapiExceptionSendAsDenied; Failed to process message due to a permanent exception with message [BeginDiagnosticData]Cannot submit message. ....>> Splunk_eRemote@uBDC01; Splunk_eRemote is a textfield in SMPT-settings. Python code seems to append "@uBDC01" to it (is the hostname of my testserver ???) I can not make any sense out of it (same setting used in past 6 year?!) on our production server Again: Sendemail is working fine, in dashboard-search (form) and when using it for testing by running manual SPL code in searches Any thoughts? regards AshleyP
02-17-2024
03:37 AM
Hi, I had followed below documentation. https://docs.splunk.com/observability/en/gdi/monitors-hosts/apache-tomcat.html There was mentioned Kubernetes and Linux in the installation steps. Not able...
See more...
Hi, I had followed below documentation. https://docs.splunk.com/observability/en/gdi/monitors-hosts/apache-tomcat.html There was mentioned Kubernetes and Linux in the installation steps. Not able to monitor tomcat via OTEL collector. I am getting below error in event logs. Regards, Eshwar
02-17-2024
03:17 AM
Checking which document? And what is not supported on Windows? OTEL collector? Or monitoring tomcat with it?
02-17-2024
03:15 AM
See https://docs.splunk.com/Documentation/Splunk/latest/Search/Changetheformatofsubsearchresults The format command is implicitly added by Splunk at the end of the subsearch if it's not explicitly p...
See more...
See https://docs.splunk.com/Documentation/Splunk/latest/Search/Changetheformatofsubsearchresults The format command is implicitly added by Splunk at the end of the subsearch if it's not explicitly put there by you (you might want to explicitly override the format in which Splunk returns the data from the subsearch; but usually it's ok as it is). So if you want to see what the data from your subsearch will be rendered as when returned from the subsearch, you can use the format command to see it as a resulting string value. So if the string containings the resulting set of conditions is OK (and works properly when literally copy-pasted to your original search), you must be hitting some limit when running the search as subsearch (It's apparently not the result count limit if you're getting only 250 or so rows so it's probably the time limit for the subsearch).
02-17-2024
03:10 AM
What I would start with when debugging such problems would be narrowing down the source of the problem. If you have an alert based on a search which obviously always matches (which is not obvious bec...
See more...
What I would start with when debugging such problems would be narrowing down the source of the problem. If you have an alert based on a search which obviously always matches (which is not obvious because you wrote that it should trigger when the value is less than 7 and your .conf dump shows a condition for higher than 7), I'd start by verifying: 1. If the search is indeed run at all? (or maybe your environment has some problems scheduling the search and skips the searches or delays them significantly) 2. If the search is indeed being run, is the alert action being dispatched. 3. If the action is being dispatched - do you see any errors from the sendemail.py in _internal.
02-17-2024
02:47 AM
Hi Splunk experts, We have some apache tomcat web servers which are installed on windows so we want to monitor those servers via OTEL collector but while checking the document it says the configurat...
See more...
Hi Splunk experts, We have some apache tomcat web servers which are installed on windows so we want to monitor those servers via OTEL collector but while checking the document it says the configuration only support on Kubernetes and Linux. So, is there a way that we can monitor windows apache tomcat servers? Please suggest! Thank in advance. Regards, Eshwar
02-17-2024
01:38 AM
This sort of chart is not possible because the y-axis has to be a number and unfortunately cannot be formatted as a time string.
02-17-2024
01:29 AM
Created a supportticket: Sendemail does not work if selected and set in the Alert config. But Sendemail function is working OK!? Business Impact: Can not respond on any "System_down/System_off...
See more...
Created a supportticket: Sendemail does not work if selected and set in the Alert config. But Sendemail function is working OK!? Business Impact: Can not respond on any "System_down/System_offline" situation . Happens not very often but very critical to respond to. Product Version : 9.2.0.1 / I assume that it might not work since Splunk Enterprise v9.1.2 either (not sure) Area: Search/Index - Splunk Enterprise Deployment Type: On Prem / Small instance with only indexer, kV Search-head active OS: Windows 2019 server When did you first notice the issue? Somewhere 1/26/2024 (noticed a system_down situation on dashboard but was not notified by email) Did you make any changes recently?: I upgrade last week to v9.2.0.1 on our test server. Later I found that our production server (v9.1.2) has the same issue Steps Reproduce: You can created an Alert_Trigger_Test: (zie code below) | makeresults | eval ATT=4 | stats max(ATT) as mincount Then test it every 5 minutes (cron-schedule) by: search mincount < 7 ======================Alert Code : in savedsearch.conf (.../search/local) ===================== [Alert_trigger_1v1] action.email = 1 action.email.cc = <your@email_address_2> action.email.include.search = 1 action.email.inline = 1 action.email.priority = 2 action.email.sendresults = 1 action.email.to = <your@email_address_1> action.email.useNSSubject = 1 action.lookup = 0 action.lookup.append = 1 action.lookup.filename = alerttrigger.csv alert.digest_mode = 0 alert.expires = 1h alert.suppress = 0 alert.track = 1 alert_condition = search mincount >7 allow_skew = 5m counttype = custom cron_schedule = */5 * * * * dispatch.earliest_time = -5m dispatch.latest_time = now display.general.type = statistics display.page.search.mode = verbose display.page.search.tab = statistics enableSched = 1 quantity = 0 relation = greater than request.ui_dispatch_app = search request.ui_dispatch_view = search search = | makeresults \ | eval ATT=3\ | stats max(ATT) as mincount =======================END Code =================== Is there any one else suffering from the same issues? regards AshleyP
Labels
- Labels:
-
alert action
-
alert condition
-
email
02-17-2024
12:59 AM
Hi @PickleRick, The one you gave worked and I got all the request ids Can you please explain how format worked out?
02-17-2024
12:49 AM
This is a long thread but some of the answers you need are there, two key ones are these: If your subsearch is using more than 50k events, your results will be compromised Rename the request_id fie...
See more...
This is a long thread but some of the answers you need are there, two key ones are these: If your subsearch is using more than 50k events, your results will be compromised Rename the request_id field as search search sourcetype="my_source" "failed request, request id=" | rex “failed request, request id==(?<search>[\w-]+)" | | stats count by search | fields search
02-17-2024
12:42 AM
My mistake - I missed out the time bin part - try this over the previous 30 minutes for 15 minute groups - you may need to align your time period so that you only get 2 15 minute bins index=my_index...
See more...
My mistake - I missed out the time bin part - try this over the previous 30 minutes for 15 minute groups - you may need to align your time period so that you only get 2 15 minute bins index=my_index source="/var/log/nginx/access.log"
| bin _time span=15m
| stats avg(request_time) as Average_Request_Time by _time
| streamstats count as weight
| eval alert=if(Average_Request_Time>1,weight,0)
| stats sum(alert) as alert
| where alert==1
02-17-2024
12:08 AM
Hello @gcusello
Thank you so much for your quick response. Here what I did.
| inputlookup account_audit.csv
| eval t=strftime(relative_time(now(),"-30d"), "%m/%d/%y" %H:%M:%S)
|eval updatedate...
See more...
Hello @gcusello
Thank you so much for your quick response. Here what I did.
| inputlookup account_audit.csv
| eval t=strftime(relative_time(now(),"-30d"), "%m/%d/%y" %H:%M:%S)
|eval updatedate=strptime(UPDATE_DATE, "%m/%d/%y" %H:%M:%S)
|eval comparetdate =strtime(t, "%m/%d/%y" %H:%M:%S)
|where updatedate >comparedate
|table account_id Name Org_Code UPDATE_DATE
But I am not getting result as expected. It's coming like as follow (duplicate account id comes under same event as a group)
account_id Name Org_Code UPDATE_DATE
121 test Y 01/24/2024 04:52:10
121
121
123 test2 A 01/30/2024 12:50:10
123
126 test3 B 02/01/2024 11:12:02
126
Total events :3
How can I remove duplicate account Id, I tried with dedup, but not working.
- Tags:
- Lookup timestamp
