Alerting

Error sending mail - (501, 'Syntax error in parameters or arguments', 'splunk@hostname')

abonuccelli_spl
Splunk Employee
Splunk Employee

Hi,

trying to setup alerting, alert is hit however sendemail.py fails - I've configured this via UI.

./var/log/splunk/splunkd.log:01-29-2014 16:18:29.590 +0000 ERROR ScriptRunner - stderr from '/opt/SPLUNK/splunk/etc/apps/search/bin/sendemail.py': ERROR:root:(501, 'Syntax error in parameters or arguments', 'sender@provider') while sending mail to: recipient1@spam.org,recipient2@spam.org

any ideas?

1 Solution

abonuccelli_spl
Splunk Employee
Splunk Employee

Ok the error message was indeed relevant.
http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml

X.1.3 Bad destination mailbox address syntax 501 The destination address was syntactically invalid. This can apply to any field in the address. This code is only useful for permanent failures.
X.1.8 Bad sender's system address 451, 501 The sender's system specified in the address does not exist or is incapable of accepting return mail. For domain names, this means the address portion to the right of the "@" is invalid for mail.

Please note 501 can be caused by many other reasons, as described in link IANA above.
In this specific it was due to incorrect values being sent by sendemail.py in FROM/TO fields

I've changed below hostnames/emails for obvious reasons.
Figured it out with tcpdump capture with dst=mailserver


17:24:58.534100 IP (tos 0x0, ttl 64, id 21199, offset 0, flags [DF], proto TCP (6), length 79)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa880 (incorrect -> 0x175e), seq 0:27, ack 53, win 229, options [nop,nop,TS val 121075 ecr 826786090], length 27
E..OR.@.@.?.............yZ.................
....1G.*ehlo myhost.mydomain.net
17:24:58.579118 IP (tos 0x0, ttl 64, id 21200, offset 0, flags [DF], proto TCP (6), length 105)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa89a (incorrect -> 0xde92), seq 27:80, ack 166, win 229, options [nop,nop,TS val 121086 ecr 826786101], length 53
E..iR.@.@.?.............yZ./...-...........
....1G.5AUTH PLAIN AGJvbnVjYS5zb2NAZ214LmNvbQBhZGZiNDI5aw==

17:24:59.400131 IP (tos 0x0, ttl 64, id 21201, offset 0, flags [DF], proto TCP (6), length 97)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa892 (incorrect -> 0x73ee), seq 80:125, ack 196, win 229, options [nop,nop,TS val 121292 ecr 826786306], length 45
E..aR.@.@.?.............yZ.d...K...........
....1G..mail FROM:<sender@myprovider> size=1106
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0x2737), seq 125:131, ack 241, win 229, options [nop,nop,TS val 121302 ecr 826786317], length 6
E..:R.@.@.?.............yZ.....x.....k.....
rset1G.

<sender@myprovider> is not correct

Adding
action.email.from = sender@myprovider.com
to relevant savedsearches.conf did the trick

so

egrep action etc/apps/search/local/savedsearches.conf
action.email = 1
action.email.inline = 1
action.email.reportServerEnabled = 0
action.email.sendresults = 1
action.email.from = sender@myprovider.com
action.email.to = recipient1@spam.com,recipient2@spam.com


17:29:16.929763 IP (tos 0x0, ttl 64, id 49861, offset 0, flags [DF], proto TCP (6), length 94)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88f (incorrect -> 0xb5ef), seq 80:122, ack 196, win 229, options [nop,nop,TS val 185674 ecr 826850688], length 42
E..^..@.@...............T....h=............
...J1H..mail FROM:sender@myprovider.com size=1084

17:29:16.979670 IP (tos 0x0, ttl 64, id 49862, offset 0, flags [DF], proto TCP (6), length 86)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa887 (incorrect -> 0xb700), seq 122:156, ack 239, win 229, options [nop,nop,TS val 185687 ecr 826850702], length 34
E..V..@.@...............T....h=............
...W1H..rcpt TO:recipient1@spam.org

17:29:17.048417 IP (tos 0x0, ttl 64, id 49863, offset 0, flags [DF], proto TCP (6), length 91)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88c (incorrect -> 0xe17e), seq 156:195, ack 247, win 229, options [nop,nop,TS val 185704 ecr 826850719], length 39
E..[..@.@...............T..1.h=............
...h1H..rcpt TO:recipient2@spam.org

17:29:17.090234 IP (tos 0x0, ttl 64, id 49864, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xd4da), seq 195:201, ack 255, win 229, options [nop,nop,TS val 185714 ecr 826850729], length 6
E..:..@.@...............T..X.h=......k.....
...r1H..data

17:29:17.128545 IP (tos 0x0, ttl 64, id 49865, offset 0, flags [DF], proto TCP (6), length 1162)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xacbb (incorrect -> 0xd099), seq 201:1311, ack 301, win 229, options [nop,nop,TS val 185724 ecr 826850739], length 1110
E.....@.@..e............T..^.h>............
...|1H..Content-Type: multipart/mixed; boundary="===============7091665037830878556=="
MIME-Version: 1.0
Subject: Splunk Alert: myhost SSH Fail
To: recipient1@spam.org,recipient2@spam.org
From: sender@myprovider.com

--===============7091665037830878556==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

U2F2ZWQgc2VhcmNoIHJlc3VsdHMuCgpOYW1lOiAnQ2FyYm9uYXJhIFNTSCBGYWlsJwpRdWVyeSBU
ZXJtczogJ2luZGV4PW9zIHByb2Nlc3M9c3NoZCBmYWlsKicKTGluayB0byByZXN1bHRzOiBodHRw
czovL3BvZmZlcmJha2tlbi5jaGlja2Vua2lsbGVyLmNvbTo4MDAwL2FwcC9zZWFyY2gvc2VhcmNo
P3E9JTdDbG9hZGpvYiUyMHJ0X3NjaGVkdWxlcl9fYWRtaW5fX3NlYXJjaF9fUk1ENWFjODk2YTdl
MmQ0ZWJlMmVfYXRfMTM5MTAxNjUwOV8wLjElMjAlN0MlMjBoZWFkJTIwMSUyMCU3QyUyMHRhaWwl
MjAxJmVhcmxpZXN0PTAmbGF0ZXN0PW5vdwpBbGVydCB3YXMgdHJpZ2dlcmVkIGJlY2F1c2Ugb2Y6
ICdTYXZlZCBTZWFyY2ggW0NhcmJvbmFyYSBTU0ggRmFpbF06IGFsd2F5cygxKScKCgpKYW4gMjkg
MTc6Mjk6MTIgY2FyYm9uYXJhIHNzaGRbNzY1NV06IEZhaWxlZCBwYXNzd29yZCBmb3IgaW52YWxp
ZCB1c2VyIGZkNDMyM3Nmc2QgZnJvbSA4MC4xNjkuMTk4Ljg2IHBvcnQgMTc0ODMgc3NoMg==

--===============7091665037830878556==--
.

17:29:17.291134 IP (tos 0x0, ttl 64, id 49866, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xcd7c), seq 1311:1317, ack 373, win 229, options [nop,nop,TS val 185764 ecr 826850780], length 6
E..:..@.@...............T....h>L.....k.....
....1H..quit

View solution in original post

abonuccelli_spl
Splunk Employee
Splunk Employee

Ok the error message was indeed relevant.
http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml

X.1.3 Bad destination mailbox address syntax 501 The destination address was syntactically invalid. This can apply to any field in the address. This code is only useful for permanent failures.
X.1.8 Bad sender's system address 451, 501 The sender's system specified in the address does not exist or is incapable of accepting return mail. For domain names, this means the address portion to the right of the "@" is invalid for mail.

Please note 501 can be caused by many other reasons, as described in link IANA above.
In this specific it was due to incorrect values being sent by sendemail.py in FROM/TO fields

I've changed below hostnames/emails for obvious reasons.
Figured it out with tcpdump capture with dst=mailserver


17:24:58.534100 IP (tos 0x0, ttl 64, id 21199, offset 0, flags [DF], proto TCP (6), length 79)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa880 (incorrect -> 0x175e), seq 0:27, ack 53, win 229, options [nop,nop,TS val 121075 ecr 826786090], length 27
E..OR.@.@.?.............yZ.................
....1G.*ehlo myhost.mydomain.net
17:24:58.579118 IP (tos 0x0, ttl 64, id 21200, offset 0, flags [DF], proto TCP (6), length 105)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa89a (incorrect -> 0xde92), seq 27:80, ack 166, win 229, options [nop,nop,TS val 121086 ecr 826786101], length 53
E..iR.@.@.?.............yZ./...-...........
....1G.5AUTH PLAIN AGJvbnVjYS5zb2NAZ214LmNvbQBhZGZiNDI5aw==

17:24:59.400131 IP (tos 0x0, ttl 64, id 21201, offset 0, flags [DF], proto TCP (6), length 97)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa892 (incorrect -> 0x73ee), seq 80:125, ack 196, win 229, options [nop,nop,TS val 121292 ecr 826786306], length 45
E..aR.@.@.?.............yZ.d...K...........
....1G..mail FROM:<sender@myprovider> size=1106
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0x2737), seq 125:131, ack 241, win 229, options [nop,nop,TS val 121302 ecr 826786317], length 6
E..:R.@.@.?.............yZ.....x.....k.....
rset1G.

<sender@myprovider> is not correct

Adding
action.email.from = sender@myprovider.com
to relevant savedsearches.conf did the trick

so

egrep action etc/apps/search/local/savedsearches.conf
action.email = 1
action.email.inline = 1
action.email.reportServerEnabled = 0
action.email.sendresults = 1
action.email.from = sender@myprovider.com
action.email.to = recipient1@spam.com,recipient2@spam.com


17:29:16.929763 IP (tos 0x0, ttl 64, id 49861, offset 0, flags [DF], proto TCP (6), length 94)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88f (incorrect -> 0xb5ef), seq 80:122, ack 196, win 229, options [nop,nop,TS val 185674 ecr 826850688], length 42
E..^..@.@...............T....h=............
...J1H..mail FROM:sender@myprovider.com size=1084

17:29:16.979670 IP (tos 0x0, ttl 64, id 49862, offset 0, flags [DF], proto TCP (6), length 86)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa887 (incorrect -> 0xb700), seq 122:156, ack 239, win 229, options [nop,nop,TS val 185687 ecr 826850702], length 34
E..V..@.@...............T....h=............
...W1H..rcpt TO:recipient1@spam.org

17:29:17.048417 IP (tos 0x0, ttl 64, id 49863, offset 0, flags [DF], proto TCP (6), length 91)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88c (incorrect -> 0xe17e), seq 156:195, ack 247, win 229, options [nop,nop,TS val 185704 ecr 826850719], length 39
E..[..@.@...............T..1.h=............
...h1H..rcpt TO:recipient2@spam.org

17:29:17.090234 IP (tos 0x0, ttl 64, id 49864, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xd4da), seq 195:201, ack 255, win 229, options [nop,nop,TS val 185714 ecr 826850729], length 6
E..:..@.@...............T..X.h=......k.....
...r1H..data

17:29:17.128545 IP (tos 0x0, ttl 64, id 49865, offset 0, flags [DF], proto TCP (6), length 1162)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xacbb (incorrect -> 0xd099), seq 201:1311, ack 301, win 229, options [nop,nop,TS val 185724 ecr 826850739], length 1110
E.....@.@..e............T..^.h>............
...|1H..Content-Type: multipart/mixed; boundary="===============7091665037830878556=="
MIME-Version: 1.0
Subject: Splunk Alert: myhost SSH Fail
To: recipient1@spam.org,recipient2@spam.org
From: sender@myprovider.com

--===============7091665037830878556==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

U2F2ZWQgc2VhcmNoIHJlc3VsdHMuCgpOYW1lOiAnQ2FyYm9uYXJhIFNTSCBGYWlsJwpRdWVyeSBU
ZXJtczogJ2luZGV4PW9zIHByb2Nlc3M9c3NoZCBmYWlsKicKTGluayB0byByZXN1bHRzOiBodHRw
czovL3BvZmZlcmJha2tlbi5jaGlja2Vua2lsbGVyLmNvbTo4MDAwL2FwcC9zZWFyY2gvc2VhcmNo
P3E9JTdDbG9hZGpvYiUyMHJ0X3NjaGVkdWxlcl9fYWRtaW5fX3NlYXJjaF9fUk1ENWFjODk2YTdl
MmQ0ZWJlMmVfYXRfMTM5MTAxNjUwOV8wLjElMjAlN0MlMjBoZWFkJTIwMSUyMCU3QyUyMHRhaWwl
MjAxJmVhcmxpZXN0PTAmbGF0ZXN0PW5vdwpBbGVydCB3YXMgdHJpZ2dlcmVkIGJlY2F1c2Ugb2Y6
ICdTYXZlZCBTZWFyY2ggW0NhcmJvbmFyYSBTU0ggRmFpbF06IGFsd2F5cygxKScKCgpKYW4gMjkg
MTc6Mjk6MTIgY2FyYm9uYXJhIHNzaGRbNzY1NV06IEZhaWxlZCBwYXNzd29yZCBmb3IgaW52YWxp
ZCB1c2VyIGZkNDMyM3Nmc2QgZnJvbSA4MC4xNjkuMTk4Ljg2IHBvcnQgMTc0ODMgc3NoMg==

--===============7091665037830878556==--
.

17:29:17.291134 IP (tos 0x0, ttl 64, id 49866, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xcd7c), seq 1311:1317, ack 373, win 229, options [nop,nop,TS val 185764 ecr 826850780], length 6
E..:..@.@...............T....h>L.....k.....
....1H..quit

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...