I'm useing alert manager in splunk alert action  with email action together.   But some time ,only the email can got the alert  notification, i check in _internal index, found some err log   8/6/218:10:02.402 AM | 08-06-2021 08:10:02.402 +0800 ERROR sendmodalert - action=alert_manager STDERR - UnicodeEncodeError: 'latin-1' codec can't encode characters in position 171-177: Body ('文件完整性告警') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.host = bj-vm-sec-searchhead-splunk-188index = _internalsourcetype = splunkdsplunk_server = bj-vm-sec-searchhead-splunk-188 8/6/218:10:02.319 AM | 2021-08-06 08:10:02,319 INFO pid="86180" logger="alert_manager_suppression_helper" message="Checking for matching suppression rules for alert=/etc/passwd文件完整性告警" (SuppressionHelper.py:66)host = bj-vm-sec-searchhead-splunk-188index = _internalmessage = Checking for matching suppression rules for alert=/etc/passwd文件完整性告警sourcetype = alert_manager_suppression_helper-too_smallsplunk_server = bj-vm-sec-searchhead-splunk-188 8/6/218:10:02.248 AM | 2021-08-06 08:10:02,248 INFO pid="86180" logger="alert_manager" message="Found job for alert '/etc/passwd文件完整性告警' with title 'HIDS passwd file monitorning'. Context is 'HIDS_all' with 1 results." (alert_manager.py:566)host = bj-vm-sec-searchhead-splunk-188index = _internalmessage = Found job for alert '/etc/passwd文件完整性告警' with title 'HIDS passwd file monitorning'. Context is 'HIDS_all' with 1 results.sourcetype = alert_manager-too_smallsplunk_server = bj-vm-sec-searchhead-splunk-188 8/6/218:10:01.733 AM | 08-06-2021 08:10:01.733 +0800 INFO sendmodalert - Invoking modular alert action=alert_manager for search="/etc/passwd文件完整性告警" sid="scheduler__splunk_SElEU19hbGw__RMD5bbb47a07bc26a359_at_1628208600_360" in app="HIDS_all" owner="splunk" type="saved"   so it seems like alert manager not support Chinese charater.  

What is the usage for the alerts index on all the indexer? I install the alert manager app on search head, run the alert search on search head, also I see some alert log data on search head alerts index, but no log on indexer, so I want to know What is the usage for the alerts index on all the indexer? If I didn't have alerts index on indexer, if it will affect the alert manager running?

I have a json format of data, I can not use the following method to process the results I want, when metricValue is a new dictionary, I changed how to extract. index="huawei_fc" sourcetype="BW_HWFC:metric:host" | rename value{}.* as * | eval t = mvzip(metricId,metricValue) | mvexpand t | eval mId=mvindex(split(t,","),0),mValue=mvindex(split(t,","),1) | stats values(mValue) as mValue by _time,urn,mId I was unable to extract the data in metricValue from the SPL above        

I am currently using a python API call to retrieve data from Splunk. I am getting approximately 1 day of data when the argument passed is for 30 days or more or lower of the data shown in the Splunk console. Can someone help?

Hello Splunk Community I'm working on a SPL to give _time difference of list of eventTypes as per the algorithm. Currently I'm using the below query. index=apple source=datapipe AccountNumber=* eventType=newyork          OR                               eventType=california         OR                             eventType=boston             OR                             eventType=houston           OR                            eventType=dallas                OR                         eventType=austin               OR                           eventType=Irvine                OR                        eventType=Washington      OR                      eventType=Atlanta               OR                       eventType=San Antonio      OR                  eventType=Brazil                   OR                  eventType=Mumbai              OR                      eventType=Delhi                    OR                    |fieldformat _time=strftime(_time,"%m/%d/%Y%I:%M:%S %p") |sort by AccountNumber,_time |streamstats  range(_time) as diff window=2 |eval DifferenceInTimeByEventtime=strftime(diff,"%M:%S") |table AccountNumber eventType _time DifferenceInTimeByEventtime The query is working..However I need the time difference as per the algorithm. NOT ONLY as per the previous event .The algorithm is as follows A    eventType=newyork                                    B    eventType=california            B-A                         C    eventType=boston                C-B                                  D    eventType=houston             D-C                                 E     eventType=dallas                  E-D                     F     eventType=dallas                   F-D                    G     eventType=Irvine                 G-E                        H     eventType=Irvine                  H-F        I      eventType=Atlanta                I-H                       J    eventType=San Antonio         J-I                  K   eventType=San Antonio         K-I                              L    eventType=Mumbai               L-I                     M   eventType=Delhi                    M-I I'm looking for a _time difference according to the algorithm above Add Avg,Max,Min column to the search     I would appreciate if there is a query optimization Thanks in Advance.

I am trying to get the alert when Excerption error happens but there are many hosts and services. In splunk the services and host arent arranged so manually I added the service name and hosts in csv file. is there a way or similar condition to get log events saying this serivce is getting error is this host with the message

I have two different hosts . hostA-1, hostA-2, hostA-3, hostA-4, hostA-5 . hostB-5, hostB-6, hostB-7, hostB-8. I want to compare the specific value from the logs that are matched like Token which are unique but wanted to find if the value are matched between hostA and hostB and form a table based on that which will show hosts name A and B and below will be the matching token

Hi, I have a dashboard where some user needs access to fetch the details as a report (.pdf) format every day. The trigger will be via background task in Windows server using (API) and the report should be generated via REST API or HEC token calls. I would like to know if HEC token or REST API can be a used as a solution to this requirement. Does it require any API endpoints ? what will be the easiest way to accomplish this.

Hello, I'm trying to connect SCOM with "Splunk Addon for Microsoft SCOM" (Version 4.0.0 - on Splunk Enterprise 7.3 Heavy Forwarder on Windows) The connection itself is working fine and I'm able to retrieve alerts from SCOM e.g. via group=alert which is the following powershell commands from "scom_command_loader.ps1":   "alert" = @('Get-SCOMAlert', 'Get-SCOMAlert | Get-SCOMAlertHistory');   The input looks like this:   & "$SplunkHome\etc\apps\Splunk_TA_microsoft-scom\bin\scom_command_loader.ps1" -groups "alert" -server "SCOM_DEV" -loglevel DEBUG -starttime "2021-08-01T00:00:00+02:00"    Now I don't want to have all alerts which will be produced in SCOM, instead I want to narrow it down only to the events with the name "*Windows Defender*". So for this I've created a new Powershell v3 Modular Input as a copy of the existing one, but using not a group, instead the commands section of the script - see also addon documentation. Section: "Configure inputs through the PowerShell scripted input UI" The example there is:   & "$SplunkHome\etc\apps\Splunk_TA_microsoft-scom\bin\scom_command_loader.ps1" -commands Get-SCOMAlert, Get-SCOMEvent   So I tried to use this. The powershell command is working on the shell when I connect directly to this SCOM system.   & "$SplunkHome\etc\apps\Splunk_TA_microsoft-scom\bin\scom_command_loader.ps1" -commands 'Get-SCOMAlert -Name "*Windows Defender*"' -server "SCOM_DEV" -loglevel DEBUG -starttime "2021-08-01T00:00:00+02:00"   The input is working fine and delivering the Windows Defender Events to Splunk. BUT the problem now is, that it does not create a checkpoint under the path "D:\Splunk\var\lib\splunk\modinputs\scom" like it does when a powershell command without a parameter (-Name "*Windows Defender*") is used. This can be seen in the DEBUG log files of the addon   index=_internal source=*ta_scom.log 2021-08-05 16:37:11 +02:00 [ log_level=WARN pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] End SCOM TA host = ws006914.schaeffler.comsource = D:\Splunk\var\log\splunk\ta_scom.logsourcetype = ms:scom:log:script 2021-08-05 16:37:11 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] Get 13 objects by 'Get-SCOMAlert -Name "*Windows Defender*"' 2021-08-05 16:37:09 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] --> serialize(Get-SCOMAlert -Name "*Windows Defender*") 2021-08-05 16:37:05 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] Get object 'Get-SCOMAlert -Name "*Windows Defender*"' without checkpoint 2021-08-05 16:37:05 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] --> executeCmd SCOM_DEV Get-SCOMAlert -Name "*Windows Defender*" 2021-08-05 16:37:05 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] Command list: Get-SCOMAlert -Name "*Windows Defender*" 2021-08-05 16:37:05 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] --> getCommands (groups=, commands=[Get-SCOMAlert -Name "*Windows Defender*"]) 2021-08-05 16:37:05 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] splunk version 7.3.4 2021-08-05 16:37:02 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] New SCOMManagementGroupConnection success 2021-08-05 16:36:55 +02:00 [ log_level=DEBUG pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] --> run (groups=, commands=[Get-SCOMAlert -Name "*Windows Defender*"], loglevel=DEBUG) 2021-08-05 16:36:55 +02:00 [ log_level=WARN pid=2956 input=_Splunk_TA_microsoft_scom_internal_used_Defender_Alerts_test_default_command ] Start SCOM TA   You can see it is calling the command correctly, but "without checkpoint". When using a default input, it looks like this:   GET Checkpoint [ log_level=DEBUG pid=10384 input=_Splunk_TA_microsoft_scom_internal_used_Events_test ] Got checkpoint '07/26/2021 10:54:39.220' from file 'D:\Splunk\var\lib\splunk\modinputs\scom\###U0NPTV9ERVY=###Get-SCOMAlert' successfully. SET Checkpoint 2021-07-26 14:00:28 +02:00 [ log_level=DEBUG pid=10384 input=_Splunk_TA_microsoft_scom_internal_used_Events_test ] Set checkpoint '07/26/2021 11:54:14.790' to file 'D:\Splunk\var\lib\splunk\modinputs\scom\###U0NPTV9ERVY=###Get-SCOMAlert' successfully.   So the problem will be duplicate data when I would run this regulary. Does anybody has an idea how to fix this? I have the feeling tried everything possible (different formations with _"_ or _'_ at different positions). Also without wildcards in the Name field its not working. I guess it somehow cannot create the checkpoint file. I also tried manipulating the  "scom_command_loader.ps1" script with a new group, which contains my query, but it can also not create the checkpoint file. Thanks in advance Michael

We are running Splunk Stream 7.3. In _internal sourcetype=stream:log we see the following warning messages: " NetFlowDecoder::decodeFlow Unable to decode flow set data. No template with id 256 received for observation domain id xyz from device 172.x.y.z . Dropping flow data set of size xxxx..." Netflow exporters are configured to send out their templates every so many seconds. Eventually the netflow exporter will send the template and the warning messages will stop. My question is whether that data actually dropped or is it cached until the template is received? Am I losing that data? Similar applications that collect netflow (Cisco Stealthwatch, Wireshark) will cache the data until they receive the template. This has implications when load balancing several hundred exporters to an array of Independent Stream Forwarders in order to determine if session persistence is necessary.