Alerting

Error sending mail - (501, 'Syntax error in parameters or arguments', 'splunk@hostname')

abonuccelli_spl
Splunk Employee
Splunk Employee

Hi,

trying to setup alerting, alert is hit however sendemail.py fails - I've configured this via UI.

./var/log/splunk/splunkd.log:01-29-2014 16:18:29.590 +0000 ERROR ScriptRunner - stderr from '/opt/SPLUNK/splunk/etc/apps/search/bin/sendemail.py': ERROR:root:(501, 'Syntax error in parameters or arguments', 'sender@provider') while sending mail to: [email protected],[email protected]

any ideas?

1 Solution

abonuccelli_spl
Splunk Employee
Splunk Employee

Ok the error message was indeed relevant.
http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml

X.1.3 Bad destination mailbox address syntax 501 The destination address was syntactically invalid. This can apply to any field in the address. This code is only useful for permanent failures.
X.1.8 Bad sender's system address 451, 501 The sender's system specified in the address does not exist or is incapable of accepting return mail. For domain names, this means the address portion to the right of the "@" is invalid for mail.

Please note 501 can be caused by many other reasons, as described in link IANA above.
In this specific it was due to incorrect values being sent by sendemail.py in FROM/TO fields

I've changed below hostnames/emails for obvious reasons.
Figured it out with tcpdump capture with dst=mailserver


17:24:58.534100 IP (tos 0x0, ttl 64, id 21199, offset 0, flags [DF], proto TCP (6), length 79)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa880 (incorrect -> 0x175e), seq 0:27, ack 53, win 229, options [nop,nop,TS val 121075 ecr 826786090], length 27
E..OR.@.@.?.............yZ.................
....1G.*ehlo myhost.mydomain.net
17:24:58.579118 IP (tos 0x0, ttl 64, id 21200, offset 0, flags [DF], proto TCP (6), length 105)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa89a (incorrect -> 0xde92), seq 27:80, ack 166, win 229, options [nop,nop,TS val 121086 ecr 826786101], length 53
E..iR.@.@.?.............yZ./...-...........
....1G.5AUTH PLAIN AGJvbnVjYS5zb2NAZ214LmNvbQBhZGZiNDI5aw==

17:24:59.400131 IP (tos 0x0, ttl 64, id 21201, offset 0, flags [DF], proto TCP (6), length 97)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa892 (incorrect -> 0x73ee), seq 80:125, ack 196, win 229, options [nop,nop,TS val 121292 ecr 826786306], length 45
E..aR.@.@.?.............yZ.d...K...........
....1G..mail FROM:<sender@myprovider> size=1106
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0x2737), seq 125:131, ack 241, win 229, options [nop,nop,TS val 121302 ecr 826786317], length 6
E..:R.@.@.?.............yZ.....x.....k.....
rset1G.

<sender@myprovider> is not correct

Adding
action.email.from = [email protected]
to relevant savedsearches.conf did the trick

so

egrep action etc/apps/search/local/savedsearches.conf
action.email = 1
action.email.inline = 1
action.email.reportServerEnabled = 0
action.email.sendresults = 1
action.email.from = [email protected]
action.email.to = [email protected],[email protected]


17:29:16.929763 IP (tos 0x0, ttl 64, id 49861, offset 0, flags [DF], proto TCP (6), length 94)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88f (incorrect -> 0xb5ef), seq 80:122, ack 196, win 229, options [nop,nop,TS val 185674 ecr 826850688], length 42
E..^..@[email protected]=............
...J1H..mail FROM:[email protected] size=1084

17:29:16.979670 IP (tos 0x0, ttl 64, id 49862, offset 0, flags [DF], proto TCP (6), length 86)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa887 (incorrect -> 0xb700), seq 122:156, ack 239, win 229, options [nop,nop,TS val 185687 ecr 826850702], length 34
E..V..@[email protected]=............
...W1H..rcpt TO:[email protected]

17:29:17.048417 IP (tos 0x0, ttl 64, id 49863, offset 0, flags [DF], proto TCP (6), length 91)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88c (incorrect -> 0xe17e), seq 156:195, ack 247, win 229, options [nop,nop,TS val 185704 ecr 826850719], length 39
E..[..@[email protected]=............
...h1H..rcpt TO:[email protected]

17:29:17.090234 IP (tos 0x0, ttl 64, id 49864, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xd4da), seq 195:201, ack 255, win 229, options [nop,nop,TS val 185714 ecr 826850729], length 6
E..:..@[email protected]=......k.....
...r1H..data

17:29:17.128545 IP (tos 0x0, ttl 64, id 49865, offset 0, flags [DF], proto TCP (6), length 1162)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xacbb (incorrect -> 0xd099), seq 201:1311, ack 301, win 229, options [nop,nop,TS val 185724 ecr 826850739], length 1110
E.....@[email protected]..^.h>............
...|1H..Content-Type: multipart/mixed; boundary="===============7091665037830878556=="
MIME-Version: 1.0
Subject: Splunk Alert: myhost SSH Fail
To: [email protected],[email protected]
From: [email protected]

--===============7091665037830878556==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

U2F2ZWQgc2VhcmNoIHJlc3VsdHMuCgpOYW1lOiAnQ2FyYm9uYXJhIFNTSCBGYWlsJwpRdWVyeSBU
ZXJtczogJ2luZGV4PW9zIHByb2Nlc3M9c3NoZCBmYWlsKicKTGluayB0byByZXN1bHRzOiBodHRw
czovL3BvZmZlcmJha2tlbi5jaGlja2Vua2lsbGVyLmNvbTo4MDAwL2FwcC9zZWFyY2gvc2VhcmNo
P3E9JTdDbG9hZGpvYiUyMHJ0X3NjaGVkdWxlcl9fYWRtaW5fX3NlYXJjaF9fUk1ENWFjODk2YTdl
MmQ0ZWJlMmVfYXRfMTM5MTAxNjUwOV8wLjElMjAlN0MlMjBoZWFkJTIwMSUyMCU3QyUyMHRhaWwl
MjAxJmVhcmxpZXN0PTAmbGF0ZXN0PW5vdwpBbGVydCB3YXMgdHJpZ2dlcmVkIGJlY2F1c2Ugb2Y6
ICdTYXZlZCBTZWFyY2ggW0NhcmJvbmFyYSBTU0ggRmFpbF06IGFsd2F5cygxKScKCgpKYW4gMjkg
MTc6Mjk6MTIgY2FyYm9uYXJhIHNzaGRbNzY1NV06IEZhaWxlZCBwYXNzd29yZCBmb3IgaW52YWxp
ZCB1c2VyIGZkNDMyM3Nmc2QgZnJvbSA4MC4xNjkuMTk4Ljg2IHBvcnQgMTc0ODMgc3NoMg==

--===============7091665037830878556==--
.

17:29:17.291134 IP (tos 0x0, ttl 64, id 49866, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xcd7c), seq 1311:1317, ack 373, win 229, options [nop,nop,TS val 185764 ecr 826850780], length 6
E..:..@[email protected]>L.....k.....
....1H..quit

View solution in original post

abonuccelli_spl
Splunk Employee
Splunk Employee

Ok the error message was indeed relevant.
http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml

X.1.3 Bad destination mailbox address syntax 501 The destination address was syntactically invalid. This can apply to any field in the address. This code is only useful for permanent failures.
X.1.8 Bad sender's system address 451, 501 The sender's system specified in the address does not exist or is incapable of accepting return mail. For domain names, this means the address portion to the right of the "@" is invalid for mail.

Please note 501 can be caused by many other reasons, as described in link IANA above.
In this specific it was due to incorrect values being sent by sendemail.py in FROM/TO fields

I've changed below hostnames/emails for obvious reasons.
Figured it out with tcpdump capture with dst=mailserver


17:24:58.534100 IP (tos 0x0, ttl 64, id 21199, offset 0, flags [DF], proto TCP (6), length 79)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa880 (incorrect -> 0x175e), seq 0:27, ack 53, win 229, options [nop,nop,TS val 121075 ecr 826786090], length 27
E..OR.@.@.?.............yZ.................
....1G.*ehlo myhost.mydomain.net
17:24:58.579118 IP (tos 0x0, ttl 64, id 21200, offset 0, flags [DF], proto TCP (6), length 105)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa89a (incorrect -> 0xde92), seq 27:80, ack 166, win 229, options [nop,nop,TS val 121086 ecr 826786101], length 53
E..iR.@.@.?.............yZ./...-...........
....1G.5AUTH PLAIN AGJvbnVjYS5zb2NAZ214LmNvbQBhZGZiNDI5aw==

17:24:59.400131 IP (tos 0x0, ttl 64, id 21201, offset 0, flags [DF], proto TCP (6), length 97)
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa892 (incorrect -> 0x73ee), seq 80:125, ack 196, win 229, options [nop,nop,TS val 121292 ecr 826786306], length 45
E..aR.@.@.?.............yZ.d...K...........
....1G..mail FROM:<sender@myprovider> size=1106
myhost.58821 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0x2737), seq 125:131, ack 241, win 229, options [nop,nop,TS val 121302 ecr 826786317], length 6
E..:R.@.@.?.............yZ.....x.....k.....
rset1G.

<sender@myprovider> is not correct

Adding
action.email.from = [email protected]
to relevant savedsearches.conf did the trick

so

egrep action etc/apps/search/local/savedsearches.conf
action.email = 1
action.email.inline = 1
action.email.reportServerEnabled = 0
action.email.sendresults = 1
action.email.from = [email protected]
action.email.to = [email protected],[email protected]


17:29:16.929763 IP (tos 0x0, ttl 64, id 49861, offset 0, flags [DF], proto TCP (6), length 94)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88f (incorrect -> 0xb5ef), seq 80:122, ack 196, win 229, options [nop,nop,TS val 185674 ecr 826850688], length 42
E..^..@[email protected]=............
...J1H..mail FROM:[email protected] size=1084

17:29:16.979670 IP (tos 0x0, ttl 64, id 49862, offset 0, flags [DF], proto TCP (6), length 86)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa887 (incorrect -> 0xb700), seq 122:156, ack 239, win 229, options [nop,nop,TS val 185687 ecr 826850702], length 34
E..V..@[email protected]=............
...W1H..rcpt TO:[email protected]

17:29:17.048417 IP (tos 0x0, ttl 64, id 49863, offset 0, flags [DF], proto TCP (6), length 91)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa88c (incorrect -> 0xe17e), seq 156:195, ack 247, win 229, options [nop,nop,TS val 185704 ecr 826850719], length 39
E..[..@[email protected]=............
...h1H..rcpt TO:[email protected]

17:29:17.090234 IP (tos 0x0, ttl 64, id 49864, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xd4da), seq 195:201, ack 255, win 229, options [nop,nop,TS val 185714 ecr 826850729], length 6
E..:..@[email protected]=......k.....
...r1H..data

17:29:17.128545 IP (tos 0x0, ttl 64, id 49865, offset 0, flags [DF], proto TCP (6), length 1162)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xacbb (incorrect -> 0xd099), seq 201:1311, ack 301, win 229, options [nop,nop,TS val 185724 ecr 826850739], length 1110
E.....@[email protected]..^.h>............
...|1H..Content-Type: multipart/mixed; boundary="===============7091665037830878556=="
MIME-Version: 1.0
Subject: Splunk Alert: myhost SSH Fail
To: [email protected],[email protected]
From: [email protected]

--===============7091665037830878556==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

U2F2ZWQgc2VhcmNoIHJlc3VsdHMuCgpOYW1lOiAnQ2FyYm9uYXJhIFNTSCBGYWlsJwpRdWVyeSBU
ZXJtczogJ2luZGV4PW9zIHByb2Nlc3M9c3NoZCBmYWlsKicKTGluayB0byByZXN1bHRzOiBodHRw
czovL3BvZmZlcmJha2tlbi5jaGlja2Vua2lsbGVyLmNvbTo4MDAwL2FwcC9zZWFyY2gvc2VhcmNo
P3E9JTdDbG9hZGpvYiUyMHJ0X3NjaGVkdWxlcl9fYWRtaW5fX3NlYXJjaF9fUk1ENWFjODk2YTdl
MmQ0ZWJlMmVfYXRfMTM5MTAxNjUwOV8wLjElMjAlN0MlMjBoZWFkJTIwMSUyMCU3QyUyMHRhaWwl
MjAxJmVhcmxpZXN0PTAmbGF0ZXN0PW5vdwpBbGVydCB3YXMgdHJpZ2dlcmVkIGJlY2F1c2Ugb2Y6
ICdTYXZlZCBTZWFyY2ggW0NhcmJvbmFyYSBTU0ggRmFpbF06IGFsd2F5cygxKScKCgpKYW4gMjkg
MTc6Mjk6MTIgY2FyYm9uYXJhIHNzaGRbNzY1NV06IEZhaWxlZCBwYXNzd29yZCBmb3IgaW52YWxp
ZCB1c2VyIGZkNDMyM3Nmc2QgZnJvbSA4MC4xNjkuMTk4Ljg2IHBvcnQgMTc0ODMgc3NoMg==

--===============7091665037830878556==--
.

17:29:17.291134 IP (tos 0x0, ttl 64, id 49866, offset 0, flags [DF], proto TCP (6), length 58)
myhost.59090 > mail.myprovider.com.smtp: Flags [P.], cksum 0xa86b (incorrect -> 0xcd7c), seq 1311:1317, ack 373, win 229, options [nop,nop,TS val 185764 ecr 826850780], length 6
E..:..@[email protected]>L.....k.....
....1H..quit

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

New Release of Federated Search: Bringing Splunk Analytics to More of Your Data

Organizations today are generating more data than ever and storing it across cloud object stores, data lakes, ...

Inside Event Intelligence: How ITSI Turns Network Alerts into Actionable Incidents

Tech Talk Inside Event Intelligence: How ITSI Turns Network Alerts into Actionable Incidents   Correlating ...

Observability Simplified: Combining User Experience, Application Performance & ...

  Tech Talk Network to App: Observability Unlocked   Today’s digital environments span applications, ...