Data resembles this pattern. | makeresults | eval _raw="{\"foo\": [{\"randstring1\": {\"fqdn\" : \"ibar.example.com\"}}, {\"randstring2\": {\"fqdn\" : \"jbar.example.com\"} }]}" I am trying to extract the two FQDNs when the containing field name foo{}.* is a random string. Any hints on how to get this data? I've tried a few different options with spath and can't seem to get it to work. I could try a rex, but I was really hoping to avoid that. Basically, what I want at the end is a field (multivalue in this case) that has as value ibar.example.com and jbar.example.com. ... View more

Trying to configure various alerts to use Microsoft Teams. For one alert, it works reliably, each time showing up. Other alerts, I get no notice at all. Overall log of a failed attempt to send an alert according to _internal (anonymized): 06-27-2019 19:33:06.721 +0000 INFO Metrics - group=per_source_thruput, series="/opt/splunk/var/log/splunk/microsoft_teams_webhook_modalert.log", kbps=0.049490623625275856, eps=0.16129343918613137, kb=1.5341796875, ev=5, avg_age=0, max_age=0 06-27-2019 19:33:03.752 +0000 ERROR SearchScheduler - Error in 'sendalert' command: Alert script returned error code 4., search='sendalert microsoft_teams_webhook results_file="/opt/splunk/var/run/splunk/dispatch/scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535/results.csv.gz" results_link="https://host.example.com/en-US/app/splunk_monitoring_console/@go?sid=scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535"' 06-27-2019 19:33:03.751 +0000 WARN sendmodalert - action=microsoft_teams_webhook - Alert action script returned error code=4 06-27-2019 19:33:03.751 +0000 INFO sendmodalert - action=microsoft_teams_webhook - Alert action script completed in duration=2277 ms with exit code=4 2019-06-27 19:33:03,737 ERROR pid=30021 tid=MainThread file=cim_actions.py:message:238 | sendmodaction - signature="Error: 'NoneType' object has no attribute 'split'. Please double check spelling and also verify that a compatible version of Splunk_SA_CIM is installed." action_name="microsoft_teams_webhook" search_name="DMC Alert - Total License Usage Near Daily Quota" sid="scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535" rid="0" app="splunk_monitoring_console" user="nobody" action_mode="saved" action_status="failure" 2019-06-27 19:33:03,737 INFO pid=30021 tid=MainThread file=cim_actions.py:message:238 | sendmodaction - signature="Alert action microsoft_teams_webhook started." action_name="microsoft_teams_webhook" search_name="DMC Alert - Total License Usage Near Daily Quota" sid="scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535" rid="0" app="splunk_monitoring_console" user="nobody" action_mode="saved" action_status="success" 2019-06-27 19:33:01,748 INFO pid=30021 tid=MainThread file=cim_actions.py:message:238 | sendmodaction - signature="Invoking modular action" action_name="microsoft_teams_webhook" search_name="DMC Alert - Total License Usage Near Daily Quota" sid="scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535" rid="0" app="splunk_monitoring_console" user="nobody" action_mode="saved" 06-27-2019 19:33:01.473 +0000 INFO sendmodalert - Invoking modular alert action=microsoft_teams_webhook for search="DMC Alert - Total License Usage Near Daily Quota" sid="scheduler__nobody_c3BsdW5rX21vbml0b3JpbmdfY29uc29sZQ__RMD544e6bdae8b4cae07_at_1561663980_61535" in app="splunk_monitoring_console" owner="nobody" type="saved" Any suggestions as to what might be wrong? I've rechecked the submit URL to confirm it is identical between the one that works and the ones that do not. ... View more