External documentation
This patch completes the README.txt placeholder file. The first version of external (as opposed to in-line) documentation is complete. The unused file README.md that was created by the GitHub repository establishment procedure is deleted.incoming
Ray Brown
parent
9add480710
commit
f62462d5f8
324
README.txt
324
README.txt
|
|
@ -2,3 +2,327 @@
|
|||
Datagram Transport Layer Security for Python
|
||||
============================================
|
||||
|
||||
PyDTLS brings Datagram Transport Layer Security (DTLS - RFC 6347:
|
||||
http://tools.ietf.org/html/rfc6347) to the Python environment. In a
|
||||
nutshell, DTLS brings security (encryption, server authentication,
|
||||
user authentication, and message authentication) to UDP datagram
|
||||
payloads in a manner equivalent to what SSL/TLS does for TCP stream
|
||||
content.
|
||||
|
||||
DTLS is now very easy to use in Python. If you're familiar with the
|
||||
ssl module in Python's standard library, you already know how. All it
|
||||
takes is passing a datagram/UDP socket to the *wrap_socket* function
|
||||
instead of a stream/TCP socket. Here's how one sets up the client side
|
||||
of a connection::
|
||||
|
||||
import ssl
|
||||
from socket import socket, AF_INET, SOCK_DGRAM
|
||||
from dtls import do_patch
|
||||
do_patch()
|
||||
sock = ssl.wrap_socket(socket(AF_INET, SOCK_DGRAM))
|
||||
sock.connect(('foo.bar.com', 1234))
|
||||
sock.send('Hi there')
|
||||
|
||||
Design Goals
|
||||
============
|
||||
|
||||
The primary design goal of PyDTLS is broad availability. It has therefore
|
||||
been built to be widely compatible with the following:
|
||||
|
||||
* Operating systems: apart from the Python standard library, PyDTLS
|
||||
relies on the OpenSSL library only. OpenSSL is widely ported, and
|
||||
in fact the Python standard library's *ssl* module also uses it.
|
||||
* Python runtime environments: PyDTLS is a package consisting of
|
||||
pure Python modules only. It should therefore be portable to many
|
||||
interpreters and runtime environments. It interfaces with OpenSSL
|
||||
strictly through the standard library's *ctypes* foreign function
|
||||
library.
|
||||
* The Python standard library: the standard library's *ssl* module is
|
||||
Python's de facto interface to SSL/TLS. PyDTLS aims to be compatible
|
||||
with the full public interface presented by this module. The ssl
|
||||
module ought to behave identically with respect to all of its
|
||||
functions and options when used in conjunction with PyDTLS and
|
||||
datagram sockets as when used without PyDTLS and stream sockets.
|
||||
* Connection-based protocols: as outlined below, layering security
|
||||
on top of datagram sockets requires introducing certain
|
||||
connection constructs normally absent from datagram
|
||||
sockets. These constructs have been added in such a way as to be
|
||||
compatible with code that expects to interoperate with
|
||||
connection-oriented stream sockets. For example, code that
|
||||
expects to go through server-side bind/listen/accept connection
|
||||
establishment should be reusable with PyDTLS sockets.
|
||||
|
||||
Distributions
|
||||
=============
|
||||
|
||||
PyDTLS requires version 1.0.0 or higher of the OpenSSL
|
||||
library. Earlier versions are reported not to offer stable DTLS
|
||||
support. Since packaged distributions of this version of OpenSSL are
|
||||
available for many popular operating systems, OpenSSL-1.0.0 is an
|
||||
installation requirement before PyDTLS functionality can be called.
|
||||
On Ubuntu 12.04 LTS, for example, the Python interpreter links with
|
||||
libcrypto.so.1.0.0 and libssl.so.1.0.0, and so use of PyDTLS requires
|
||||
no further installation steps.
|
||||
|
||||
In comparison, installation of OpenSSL on Microsoft Windows operating
|
||||
systems is inconvenient. For this reason, source distributions of
|
||||
PyDTLS are available that include OpenSSL dll's for 32-bit and 64-bit
|
||||
Windows. For 32-bit Windows, a version built with the MinGW toolchain
|
||||
is also available. Its archive includes stripped as well as
|
||||
non-stripped dll's. The latter can be debugged with gdb on
|
||||
Windows. All dll's have been linked with the Visual Studio 2008
|
||||
version of the Microsoft C runtime library, msvcr90.dll, the version
|
||||
used by CPython 2.7. Installation of Microsoft redistributable runtime
|
||||
packages should therefore not be required on machines with CPython
|
||||
2.7. The version of OpenSSL distributed with PyDTLS 0.1.0 is 1.0.1c.
|
||||
|
||||
The OpenSSL version used by PyDTLS can be determined from the values
|
||||
of *sslconnection's* DTLS_OPENSSL_VERSION_NUMBER,
|
||||
DTLS_OPENSSL_VERSION, and DTLS_OPENSSL_VERSION_INFO. These variables
|
||||
are available through the *ssl* module also if *do_patch* has been
|
||||
called (see below). Note that the OpenSSL version used by PyDTLS may
|
||||
differ from the one used by the *ssl* module.
|
||||
|
||||
Interfaces
|
||||
==========
|
||||
|
||||
PyDTLS' top-level package, *dtls*, provides DTLS support through the
|
||||
**SSLConnection** class of its *sslconnection*
|
||||
module. **SSLConnection** is in-line documented, and
|
||||
dtls/test/echo_seq.py demonstrates how to take a simple echo server
|
||||
through a listen/accept/echo/shutdown sequence using this class. The
|
||||
corresponding client side can look like the snippet at the top of this
|
||||
document, followed by a call to the *unwrap* method for shutdown (or a
|
||||
call to the **SSLConnection** *shutdown* method, if an instance of
|
||||
this class is used for the client side also). Note that the *dtls*
|
||||
package does not depend on the standard library's *ssl* module, and
|
||||
**SSLConnection** can therefore be used in environments where *ssl* is
|
||||
unavailable or incompatible.
|
||||
|
||||
It is expected that with the *ssl* module being an established, familiar
|
||||
interface to TLS, it will be the preferred module through which to
|
||||
access DTLS. To do so, one must call the *dtls* package's *do_patch*
|
||||
function before passing sockets of type SOCK_DGRAM to either *ssl's*
|
||||
*wrap_socket* function, or *ssl's* **SSLSocket** constructor.
|
||||
|
||||
It should be noted that once *do_patch* is called, *dtls* will raise
|
||||
exceptions of type **ssl.SSLError** instead of its default
|
||||
**dtls.err.SSLError**. This allows users' error handling code paths to
|
||||
remain identical when interfacing with *ssl* across stream and
|
||||
datagram sockets.
|
||||
|
||||
Connection Handling
|
||||
===================
|
||||
|
||||
The DTLS protocol implies a connection as an association between two
|
||||
network peers where the overall association state is characterized by the
|
||||
handshake status of each peer endpoint (see RFC 6347). The OpenSSL library
|
||||
records this handshake status in "SSL" type instances (a.k.a. struct
|
||||
ssl_st). Datagrams can be securely sent and received by referring to a
|
||||
unique "SSL" instance after handshaking has been completed with this
|
||||
instance and its network peer. A connection is implied in that traffic
|
||||
may be directed to or received from only that network peer with whose
|
||||
"SSL" instance the handshake has been completed. The fact that the
|
||||
underlying network protocol, UDP in most cases, is itself connectionless
|
||||
is immaterial.
|
||||
|
||||
Further, in order to prevent denial-of-service attacks on UDP DTLS
|
||||
servers, clients must undergo a cookie exchange phase early in the
|
||||
handshaking protocol, and before server-side resources are committed to
|
||||
a particular client (see section 4.2.1 of RFC 6347). The cookie exchange
|
||||
proves to the server that a client can indeed receive IP traffic at
|
||||
the source IP address with which its handshake-initiating ClientHello
|
||||
datagram is marked.
|
||||
|
||||
PyDTLS implements this connection establishment through the *connect*
|
||||
method on the client side, and the *accept* method on the server side.
|
||||
The latter returns a new **dtls.SSLConnection** or **ssl.SSLSocket**
|
||||
object (depending on which interface is used, see above), which is
|
||||
"connected" to its peer. In addition to the *read* and *write* methods
|
||||
(at both interface levels), **SSLSocket's** *send* and *recv* methods
|
||||
can be used; use of *sendto* and *recvfrom* on connected sockets is
|
||||
prohibited by *ssl*. *accept* returns peer address information, as
|
||||
expected. Note that when using the *ssl* interface to *dtls*, *listen*
|
||||
must be called before calling *accept*.
|
||||
|
||||
Demultiplexing
|
||||
==============
|
||||
|
||||
At the network io layer, only datagrams from its connected peer must be
|
||||
passed to a **SSLConnection** or **SSLSocket** object (unless the object
|
||||
is unconnected on the server-side, in which case in can be in listening
|
||||
mode, the initial server-side socket whose role it is to listen for
|
||||
incoming client connection requests).
|
||||
|
||||
The initial server-side listening socket is not useful for performing this
|
||||
datagram routing function. This is because it must remain unconnected and
|
||||
ready to receive additional connection requests from new, unknown clients.
|
||||
|
||||
The function of passing incoming datagrams to the proper connection is
|
||||
performed by the *dtls.demux* package. **SSLConnection** requests a new
|
||||
connection from the demux when a handshake has cleared the cookie
|
||||
exchange phase. An efficient implementation of this request is provided
|
||||
by the *osnet* module of the demux package: it creates a new socket that
|
||||
is bound to the same network interface and port as the listening socket,
|
||||
but connected to the peer. UDP stacks such as the one included with Linux
|
||||
route incoming datagrams to such a connected socket in preference to an
|
||||
unconnected socket bound to the same port.
|
||||
|
||||
Unfortunately such is not the behavior on Microsoft Windows. Windows
|
||||
UDP routes datagrams to whichever currently existing socket bound to
|
||||
the particular port the earliest (and whether or not that socket is
|
||||
unconnected, or connected to the datagram's peer, or a different
|
||||
peer). Other sockets bound to the same port will not receive traffic,
|
||||
if and until they become the earliest bound socket because another
|
||||
socket is closed.
|
||||
|
||||
The demux package therefore provides and automatically selects the module
|
||||
*router* on Windows platforms. This module also creates a new socket when
|
||||
receiving a new connection request; but instead of binding this socket
|
||||
to the same port as the listening socket, it binds to a new ephemeral
|
||||
port. *router* then forwards datagrams originating from the peer for which
|
||||
a connection was requested to the corresponding socket.
|
||||
|
||||
For efficiency's sake, no forwarding is performed on outgoing traffic.
|
||||
Instead, **SSLConnection** directs outgoing traffic from the original
|
||||
listening socket, using *sendto*. At the OpenSSL level this requires
|
||||
separate read and write datagram BIO's for an "SSL" instance, one in
|
||||
"connected" state and one in "peer set" state, respectively, and
|
||||
associated with two separate network sockets.
|
||||
|
||||
From the perspective of a PyDTLS user, this selection of and
|
||||
difference between demux implementations should be transparent, with
|
||||
the possible exception of performance deviation. This transparency
|
||||
does however have some limits: for example, when *router* is in use,
|
||||
the *accept* methods can return *None*. This happens when
|
||||
**SSLConnection** detects that the demux has forwarded a datagram to a
|
||||
known connection instead of initiating a connection to a new peer
|
||||
through *accept*. Returning *None* in this case is important whenever
|
||||
non-blocking sockets or sockets with timeouts are used, since another
|
||||
socket might now be readable as a result of the forwarded
|
||||
datagram. *accept* must return so that the application can iterate on
|
||||
its asynchronous *select* loop.
|
||||
|
||||
Shutdown and Unwrapping
|
||||
=======================
|
||||
|
||||
PyDTLS implements the SSL/TLS shutdown protocol as it has been adapted
|
||||
for DTLS. **SSLConnection's** *shutdown* and **SSLSocket's** *unwrap*
|
||||
invoke this protocol. As is the case with DTLS handshaking in general,
|
||||
applications must be prepared to use the *get_timeout* and
|
||||
*handle_timeout* methods in addition to re-invoking *shutdown* or
|
||||
*unwrap* when sockets become readable and an exception carried
|
||||
SSL_ERROR_WANT_READ. (See more on asynchronous IO in the Testing section.)
|
||||
|
||||
**SSLConnection's** *shutdown* and **SSLSocket's** *unwrap* return a
|
||||
(possibly new) socket that can be used for unsecured communication
|
||||
with the peer, as set forth by the *ssl* module. The demux
|
||||
infrastructure remains in use for this communication until the
|
||||
returned socket is cleaned up. Note that when the *router* demux is
|
||||
in use, the object returned will be one derived from
|
||||
*socket.socket*. This is because the send and recv paths must still be
|
||||
directed to two different OS sockets. In addition, the right thing
|
||||
happens if secured communication is resumed over such a socket by
|
||||
passing it to *ssl.wrap_socket* or the **SSLConnection**
|
||||
constructor. If *osnet* is used, an actual *socket.socket* instance is
|
||||
returned.
|
||||
|
||||
Framework Compatibility
|
||||
=======================
|
||||
|
||||
PyDTLS sockets have been tested under the following usage modes:
|
||||
|
||||
* Using blocking sockets and sockets with timeouts in
|
||||
multi-threaded UDP servers
|
||||
* Using non-blocking sockets, and in conjunction with the
|
||||
asynchronous socket handler, asyncore
|
||||
* Using blocking sockets, and in conjunction with the network
|
||||
server framework SocketServer - ThreadingTCPServer (this works
|
||||
because of PyDTLS's emulation of connection-related calls)
|
||||
|
||||
Multi-thread Support
|
||||
====================
|
||||
|
||||
Using multiple threads with OpenSSL requires implementing a locking
|
||||
callback. PyDTLS does implement this, and therefore multi-threaded
|
||||
programming with PyDTLS is safe in any environment. However, being a
|
||||
pure Python library, these callbacks do carry some overhead. The *ssl*
|
||||
module implements an equivalent locking callback in its C extension
|
||||
module. Not requiring interpreter re-entry, this approach can be
|
||||
expected to perform better. PyDTLS therefore queries OpenSSL as to
|
||||
whether a locking callback is already in place, and does not overwrite
|
||||
it if there is. Loading *ssl* can therefore improve performance, even
|
||||
when only the *sslconnection* interface is used.
|
||||
|
||||
Note that loading order does not matter: to obtain the performance
|
||||
benefit, *ssl* can be loaded before or after the dtls package. This is
|
||||
because *ssl* does not do an equivalent existing locking callback
|
||||
check, and will simply overwrite the PyDTLS callback if it has already
|
||||
been installed. But *ssl* should not be loaded while *dtls* operation
|
||||
is already in progress, when some locks may be in their acquired
|
||||
state.
|
||||
|
||||
Also note that this performance enhancement is available only on
|
||||
platforms where PyDTLS loads the same OpenSSL shared object as
|
||||
*ssl*. On Ubuntu 12.04, for example, this is the case, but on
|
||||
Microsoft Windows it is not.
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
Almost all of the Python standard library's *ssl* unit tests from the
|
||||
module *test_ssl.py* have been ported to *dtls.test.unit.py*. All tests
|
||||
have been adjusted to operate with datagram sockets. On Linux, each
|
||||
test is executed four times, varying the address family among IPv4 and
|
||||
IPv6 and the demux among *osnet* and *router*. On Windows, where
|
||||
*osnet* is unavailable, each test is run twice, once with IPv4 and once
|
||||
with IPv6.
|
||||
|
||||
The unit test suite includes tests for each of the above-mentioned
|
||||
compatible frameworks. The class **AsyncoreEchoServer** serves as an
|
||||
example of how to use non-blocking datagram sockets and implement the
|
||||
resulting timeout detection requirements. DTLS in general and OpenSSL
|
||||
in particular require being called back when used with non-blocking
|
||||
sockets (or sockets with timeout option) after DTLS timeouts expire to
|
||||
handle packet loss using re-transmission during a
|
||||
handshake. Handshaking may occur during any read or write operation,
|
||||
even after an initial handshake completes successfully, in case
|
||||
renegotiation is requested by a peer.
|
||||
|
||||
Running with the -v switch executes all unit tests in verbose mode.
|
||||
|
||||
dtls/test/test_perf.py implements an interactive performance test
|
||||
suite that compares the raw throughput of TCP, UDP, SSL, and DTLS.
|
||||
It can be executed locally through the loopback interface, or between
|
||||
remote clients and servers. In the latter case, test jobs are sent to
|
||||
remote connected clients whenever a suite run is initiated through the
|
||||
interactive interface. Run test_perf.py -h for more information.
|
||||
|
||||
It should be noted that comparing the performance of protocols that
|
||||
don't offer congestion control (UDP and DTLS) with those that do (TCP
|
||||
and SSL) is a difficult undertaking. Raw throughput even across
|
||||
gigabit network links can be expected to suffer without congestion
|
||||
control and peers that generate data as fast as possible without
|
||||
throttling (as this test does): the link's throughput will drop
|
||||
significantly as it enters congestion collapse. Similarly, loopback is
|
||||
an imperfect test interface since it rarely drops packets, and never
|
||||
duplicates or reorders them (thus negating the relative performance
|
||||
benefits of DTLS over SSL). Nevertheless, some useful insights can be
|
||||
gained by observing the operation of test_perf.py, including software
|
||||
stack behavior in the presence of some amount of packet loss.
|
||||
|
||||
Logging
|
||||
=======
|
||||
|
||||
The *dtls* package and its sub-packages log various occurrences,
|
||||
primarily events that can aid debugging. Especially *router* emits many
|
||||
messages when the logging level is set to at least *logging.DEBUG*.
|
||||
dtls/test/echo_seq.py activates this logging level during its operation.
|
||||
|
||||
Currently Supported Platforms
|
||||
=============================
|
||||
|
||||
At the time of initial release, PyDTLS 0.1.0 has been tested on Ubuntu
|
||||
12.04.1 LTS 32-bit and 64-bit, as well as Microsoft Windows 7 32-bit
|
||||
and 64-bit, using CPython 2.7.3. Patches with additional platform
|
||||
ports are welcome.
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ has the following effects:
|
|||
* The constant PROTOCOL_DTLSv1 is added at ssl module level
|
||||
* DTLSv1's protocol name is added to the ssl module's id-to-name dictionary
|
||||
* The constants DTLS_OPENSSL_VERSION* are added at the ssl module level
|
||||
* Instntiation of ssl.SSLSocket with sock.type == socket.SOCK_DGRAM is
|
||||
* Instantiation of ssl.SSLSocket with sock.type == socket.SOCK_DGRAM is
|
||||
supported and leads to substitution of this module's DTLS code paths for
|
||||
that SSLSocket instance
|
||||
* Direct instantiation of SSLSocket as well as instantiation through
|
||||
|
|
|
|||
Loading…
Reference in New Issue