6

I have a code in Python that accepts (E)SMTP requests via aiosmtp but since I pushed this code on Debian 10, I'm having a few errors that wasn't present before (and my code didn't changed):

[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)

SSL handshake failed protocol: transport: <_SelectorSocketTransport fd=11 read=polling write=>

SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
  File "asyncio/sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()

and:

[SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)

SSL error in data received protocol: transport: <_SelectorSocketTransport fd=15 read=polling write=>

SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)
  File "asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "asyncio/sslproto.py", line 207, in feed_ssldata
    self._sslobj.unwrap()
  File "ssl.py", line 767, in unwrap
    return self._sslobj.shutdown()

I think these two issues are related.

Unfortunately the two stacktrace don't show anything related to my code, which makes it harder for me to better see where this is happening, and the exception isn't related to another exception (Python3).

Here's the versions of my package:

uname -a : Linux my-server 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5+deb10u2 (2019-08-08) x86_64 GNU/Linux

python --version: Python 3.7.3

pip freeze

aiomysql==0.0.20
aiosmtpd==1.2
asn1crypto==0.24.0
atpublic==1.0
authres==1.2.0
beanstalkc3==0.4.0
blinker==1.4
certifi==2018.8.24
cffi==1.12.3
chardet==3.0.4
Click==7.0
cloud-init==18.3
configobj==5.0.6
cryptography==2.6.1
distro-info==0.21
dkimpy==0.9.4
dnspython==1.16.0
fail2ban==0.10.2
Flask==1.1.1
idna==2.6
itsdangerous==1.1.0
Jinja2==2.10.1
jsonpatch==1.21
jsonpointer==1.10
jsonschema==2.6.0
MarkupSafe==1.1.0
mysqlclient==1.4.4
oauthlib==2.1.0
psutil==5.6.3
py3dns==3.2.1
pycparser==2.19
PyGObject==3.30.4
pyinotify==0.9.6
PyJWT==1.7.0
PyMySQL==0.9.2
PyNaCl==1.3.0
pyspf==2.0.13
pysrs==1.0.3
python-apt==1.8.4
python-dotenv==0.10.3
PyYAML==3.13
requests==2.21.0
sentry-sdk==0.12.3
six==1.12.0
systemd-python==234
unattended-upgrades==0.1
urllib3==1.24.1
uWSGI==2.0.18
Werkzeug==0.16.0

I believe that if something was wrong with my code, I would have had this error on Debian 9 and earlier which I never had.

I searched on SO and Google about this error but didn't find anything. I suspect some issue on a specific version of a specific project (aiosmtpd, async or python) but don't have any clue.

I'm hoping you'll be able to help me :)

Update:

I've added tracking of ciphers in the communication. The shared ciphers are:

[[TLS_AES_256_GCM_SHA384, TLSv1.3, 256], [TLS_CHACHA20_POLY1305_SHA256, TLSv1.3, 256], [TLS_AES_128_GCM_SHA256, TLSv1.3, 128], [ECDHE-ECDSA-AES256-GCM-SHA384, TLSv1.2, 256], [ECDHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256], [DHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256], [ECDHE-ECDSA-CHACHA20-POLY1305, TLSv1.2, 256], [ECDHE-RSA-CHACHA20-POLY1305, TLSv1.2, 256], [DHE-RSA-CHACHA20-POLY1305, TLSv1.2, 256], [ECDHE-ECDSA-AES128-GCM-SHA256, TLSv1.2, 128], [ECDHE-RSA-AES128-GCM-SHA256, TLSv1.2, 128]]

And the Cipher for the socket is: [ECDHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256] which is in the shared ciphers.

Update 2

I can reproduce the error, but only under specific conditions.

On the new server, here's the code I run:

import asyncio, logging, sys, signal, ssl
from aiosmtpd.controller import Controller
from aiosmtpd.handlers import Debugging
from aiosmtpd.smtp import SMTP

class ControllerTls(Controller):
    def factory(self):
        context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
        context.load_cert_chain('./certs/certificate.pem', './certs/id_rsa')
        context.load_dh_params('./certs/dhparams.pem')
        return SMTP(
            self.handler,
            tls_context=context
        )


# Temporary outputing errors from mail.log
streamHandler = logging.StreamHandler(sys.stdout)
streamHandler.setFormatter(logging.Formatter('[%(asctime)-15s] (%(levelname)s) - %(message)s'))
streamHandler.setLevel(logging.INFO)

maillog = logging.getLogger('mail.log')
maillog.setLevel(logging.INFO)
maillog.addHandler(streamHandler)

controller = ControllerTls(Debugging(), hostname='0.0.0.0', port=2125)
controller.start()
print('Controller started!')
sig = signal.sigwait([signal.SIGINT, signal.SIGQUIT])
controller.stop()

It's a basic script that helps me reproduce the issue.

On the Old server, I run this code:

import smtplib, ssl, sys

port = 25
if len(sys.argv) == 3:
    port = sys.argv[2]

def com(client, command, *args, **kwargs):
    result = getattr(client, command)(*args, **kwargs)
    if result[0] > 500:
        print('[FATAL] - An error occured!')
        print(result)
        client.quit()
        exit()

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('/var/www/towboat/certs/certificate.pem', '/var/www/towboat/certs/id_rsa')
context.set_ciphers('ECDHE-ECDSA-AES256-GCM-SHA384')
client = smtplib.SMTP(sys.argv[1], port=port)
com(client, 'ehlo')
com(client, 'starttls', context=context)
com(client, 'ehlo')
com(client, 'mail', '[email protected]')
com(client, 'rcpt', '[email protected]')
com(client, 'quit')

print('All good !')

Which I call with :

sendmail.py {ip.of.new.server} 2125

On the old server (the one running the script), I get this error:

Controller started!
[2019-10-08 15:57:11,878] (INFO) - Peer: ('ip.of.old.server', 45492)
[2019-10-08 15:57:11,878] (INFO) - ('ip.of.old.server', 45492) handling connection
[2019-10-08 15:57:11,880] (INFO) - ('ip.of.old.server', 45492) Data: b'ehlo {name old server}'
[2019-10-08 15:57:11,883] (INFO) - ('ip.of.old.server', 45492) Data: b'STARTTLS'
[2019-10-08 15:57:11,883] (INFO) - ('ip.of.old.server', 45492) STARTTLS
SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f04d33d7d30>
transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f04d33d7d30>
transport: <_SelectorSocketTransport closing fd=7 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
[2019-10-08 15:58:33,909] (INFO) - Connection lost during _handle_client()

What's super odd, is that if I copy the sendmail script on my local machine, and run it pointing to the new server, I don't have the error anymore!

(So the issue must be related to the old server? But why the new server shows the exception?!)

If I switch the scripts (testing sending an email from the new server to the old), it works...

Improve this question
33
  • 4
    There are too much unknowns for me in this question, specifically there is no code for the server and there are no information about the client. It could for example be that the client uses some old and insecure cipher like RC4 or 3DES which are disabled in newer versions of Debian. Or it could be that the server explicitly tries to set such ciphers. It could also be that the server certificates are not properly installed in which case no useful ciphers are available. See also How to create a Minimal, Reproducible Example. Commented Oct 5, 2019 at 7:50
  • 2
    For a start, investigate which precise SSL/TLS versions the remote clients support. Commented Oct 5, 2019 at 7:57
  • 1
    The remote SMTP client connecting to your service which is attempting to set up an SSL connection. The error means you are offering one set of ciphers and they support another set and there is no common intersection. Commented Oct 5, 2019 at 8:03
  • 1
    openssl should be able to connect to anything which tries to speak SSL. Or you could write a simple Python client which connects to the unencrypted connection and then takes you through the STARTTLS negotiation. Commented Oct 5, 2019 at 8:35
  • 1
    @CyrilN.: "... if there is hundreds of them connecting, should I ask each of them by getting the ip back, asking their ISP their phone number and asking them to try again ??" - Nothing in your question suggests that you have this problem only with a specific client but that you know nothing about this client and that it is out of your control what this client is doing. Again, there is a lack of essential details in your question. Commented Oct 5, 2019 at 10:43

2 Answers 2

Reset to default
2
+100

I think that is the cause:

v1.1.1d on the new server, 1.1.0d on the old one

The 1.1.1 line introduces the TLSv3 and many other quite important changes - see the change log.

As I have seen you opening a ticket at aiosmtpd github you have guessed correctly that the reason why you are getting the error is the aiosmtpd. The reason being there is it supports You need at least Python 3.5, which has no support for openssl 1.1.1. Only python 3.7 (it has not been fully backported even to python 3.6) currently supports openssl 1.1.1.

Since the latest version of aiosmtpd is 1.2 (2018-09-01) it is save to assume (did not see any PR(s) for that) that they have not yet implemented the new openssl 1.1.1 [11 Sep 2018] which is introduces major changes.

Your only option, beside providing a PR for aiosmtpd, is to downgrade your openssl to the latest of the 1.1.0 line which is currently 1.1.0i.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you for your response. I have one question: What the risk of not downgrading? The errors I get aren't clear. Does it means that old ciphers won't be able to connect to my server, or does it means my server is not well configured and must be downgraded to work with current industries ciphers implementation?
@CyrilN. You are welcome. I think the risk is too great for you to even consider using it. You are facing multiple issues here. If the changes in openssl would be only for the TLSv1.3 you may stand a chance, but the changes that were made were also around the TLSv1.2 (also other) code so the chances for you getting this work are minimal. It may appear to work but you never know where a bug may propagate. Also you have witness the peril of mismatched versions - the error messages maybe completely useless as they may show error, but something else is causing it.
Just to be clear, what you recommend is to stay with the latest version, not downgrading? The issues related to no shared cipher means the client is connecting with an old cipher and should upgrade - which would say that my new server is more secure? (just on that point of course). Thank you for your clarification.
@CyrilN. What I recommend is downgrade the openssl to 1.1.0i line on the server (and keep the line updated) till the aiosmtpd catches up to the openssl 1.1.1 changes. You could get unanticipated results otherwise.
1
+200

I see at first that cipher ECDHE-ECDSA-AES256-GCM-SHA384 cannot work because both sides using RSA certificates (I'm wondering whether client really uses his certificate for authentication or whether it's only wrongly configured in server mode).
If You run the same client script on a python working with OpenSSL 1.1.1 the servers will agree to TLSv1.3, where the cipher suites cannot be disabled anymore so are still allowed even with the set_ciphers.
I assume if You choose proper ECDHE-RSA-AES256-GCM-SHA384 or change nothing at all in that regard on Your "old" (Openssl 1.1.0) Debian 9 it will connect without issues with TLSv1.2 to the new server.

That said, the issue with the old script to determine cipher-suites on new server is also related to the fact, that the TLSv1.3 cipher suites cannot be disabled and the script expects it can disable any cipher suite for the test (that is the way it works).

There are some cipher suites that are now completely out of OpenSSLv1.1.1, but there are others that are just disabled by default (current Python only allows HIGH ciphers and no MD5/RC4 by default - and no ciphers without athentication - and of course no SSLv3 and older).
Since python 3.6 it's very simple to get the list of offered ciphers (so the broken script is not needed anymore):

root@somehost:~# python3
Python 3.7.3 (default, Oct  7 2019, 12:56:13)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
>>> for cipher in ctx.get_ciphers(): print(cipher['name']+' '+cipher['protocol']) if cipher['auth'] == 'auth-rsa' else None
...
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2
DHE-RSA-AES256-GCM-SHA384 TLSv1.2
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2
DHE-RSA-CHACHA20-POLY1305 TLSv1.2
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
DHE-RSA-AES128-GCM-SHA256 TLSv1.2
ECDHE-RSA-AES256-SHA384 TLSv1.2
DHE-RSA-AES256-SHA256 TLSv1.2
ECDHE-RSA-AES128-SHA256 TLSv1.2
DHE-RSA-AES128-SHA256 TLSv1.2
ECDHE-RSA-AES256-SHA TLSv1.0
DHE-RSA-AES256-SHA SSLv3
ECDHE-RSA-AES128-SHA TLSv1.0
DHE-RSA-AES128-SHA SSLv3
AES256-GCM-SHA384 TLSv1.2
AES128-GCM-SHA256 TLSv1.2
AES256-SHA256 TLSv1.2
AES128-SHA256 TLSv1.2
AES256-SHA SSLv3
AES128-SHA SSLv3

So fo a TLSv1.2 connection the minimum requirement is AES128-SHA256 (without [EC]DHE KEX) or DHE-RSA-AES128-SHA256/ECDHE-RSA-AES128-SHA256 / minimum OpenSSL Version 1.0.1 Released 14-March-2012. So not OpenSSL version supporting TLSv1.2 should be unable to connect.
In production, thought, there may be some OpenSSLv0.98 clients (at least I know some I have still to maintain - thought I have all Tools I need for production build against a newer OpenSSL). These can only talk TLSv1.0 if not the "forbidden" SSLv3. They could at least use the suites shown as SSLv3.

So there shouldn't be many clients that cannot use one of the default proposals, but still there could be some that are eigther configured for specific cipher suites that are not preferrable from todays point of view, or even older systems, less powerfull hardware, different SSL library vendor, ... So You need to find an example that is really not working to see which additional cyphersuite has to be allowed - or maybe different authentication mechanism.

For the other point in this discussion like downgrading OpenSSL, that is definitly not the way to go. There cannot be many clients that cannot connect with default setting and even less that cannot connect at all with all available settings OpenSSLv1.1.1 still offers. And if there are they need an update, definitly..

If You really wanted/needed You can compile an older OpenSSL to go in a distinct location and a Python against that older OpenSSL version. Maybe start a second server on a second port just for that clients. Or You can run a container instead for that, but You cannot downgrade the system-OpenSSL.

The 2nd error, KRB5_S_INIT, is indeed a bug. It looks like a bug in python-core async module which was introduced with Python 7.3. But this error happens only if the connection is anyway unusable (so after the connection was aandoned because of the no common cipher suites case).

Improve this answer

17 Comments

Nice descriptions of the openssl and cyphers. What you left out of the equation is the aiosmtp which is the intermediary and does not yet support openssl 1.1.1.
@tukan You ar wrong. aiosmtp does not need to support openssl version. the ssl processing is done by pythons ssl module which do not change much with openssl 1.1.1 (if at all, besides the fact the TLS1.3 ciphers cannot be removed). it does nothing with SSL but hand through the ssl context and initiates probmotion of the connection to SSL when STARTLS command detected - nothing changes in that path with openssl version.
The ssl module indeed changes (what is "much" is rather subjective) - to view the changes github.com/python/cpython/commit/… and to quote "Python has provisional and Experimental support for TLS 1.3 with OpenSSL 1.1.1. The new protocol behaves slightly differently than previous version of TLS/SSL. Some new TLS 1.3 features are not yet available." Yes, it does use the SSL but if the protocol behaves differently, as written in the PR comment, you need to test it if it works correctly as intended. There is no such test for aiosmtp.
That is for me show stopper on production. You can have different view, but that does not mean my I'm wrong.
No thats wrong. For application that is using the socjket and the SSL module nothing changes. It just establishes connection and transfers data as before. So no module needs to take the chanages into account, as long as the do not care for setting up the context (that is exactly what the most modules not do - they leave it up to user to provide a context or may offer to create a default context as python is offering for user). @tukan
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.