From f62462d5f893ffbbfd6f1c1ee8f8f74e10cc7da0 Mon Sep 17 00:00:00 2001 From: Ray Brown Date: Wed, 19 Dec 2012 00:49:00 -0800 Subject: [PATCH] External documentation This patch completes the README.txt placeholder file. The first version of external (as opposed to in-line) documentation is complete. The unused file README.md that was created by the GitHub repository establishment procedure is deleted. --- README.md | 2 - README.txt | 324 ++++++++++++++++++++++++++++++++++++++++++++++++++ dtls/patch.py | 2 +- 3 files changed, 325 insertions(+), 3 deletions(-) delete mode 100644 README.md diff --git a/README.md b/README.md deleted file mode 100644 index cdcfa59..0000000 --- a/README.md +++ /dev/null @@ -1,2 +0,0 @@ -vdc -=== \ No newline at end of file diff --git a/README.txt b/README.txt index f546386..bb4316e 100644 --- a/README.txt +++ b/README.txt @@ -2,3 +2,327 @@ Datagram Transport Layer Security for Python ============================================ +PyDTLS brings Datagram Transport Layer Security (DTLS - RFC 6347: +http://tools.ietf.org/html/rfc6347) to the Python environment. In a +nutshell, DTLS brings security (encryption, server authentication, +user authentication, and message authentication) to UDP datagram +payloads in a manner equivalent to what SSL/TLS does for TCP stream +content. + +DTLS is now very easy to use in Python. If you're familiar with the +ssl module in Python's standard library, you already know how. All it +takes is passing a datagram/UDP socket to the *wrap_socket* function +instead of a stream/TCP socket. Here's how one sets up the client side +of a connection:: + + import ssl + from socket import socket, AF_INET, SOCK_DGRAM + from dtls import do_patch + do_patch() + sock = ssl.wrap_socket(socket(AF_INET, SOCK_DGRAM)) + sock.connect(('foo.bar.com', 1234)) + sock.send('Hi there') + +Design Goals +============ + +The primary design goal of PyDTLS is broad availability. It has therefore +been built to be widely compatible with the following: + + * Operating systems: apart from the Python standard library, PyDTLS + relies on the OpenSSL library only. OpenSSL is widely ported, and + in fact the Python standard library's *ssl* module also uses it. + * Python runtime environments: PyDTLS is a package consisting of + pure Python modules only. It should therefore be portable to many + interpreters and runtime environments. It interfaces with OpenSSL + strictly through the standard library's *ctypes* foreign function + library. + * The Python standard library: the standard library's *ssl* module is + Python's de facto interface to SSL/TLS. PyDTLS aims to be compatible + with the full public interface presented by this module. The ssl + module ought to behave identically with respect to all of its + functions and options when used in conjunction with PyDTLS and + datagram sockets as when used without PyDTLS and stream sockets. + * Connection-based protocols: as outlined below, layering security + on top of datagram sockets requires introducing certain + connection constructs normally absent from datagram + sockets. These constructs have been added in such a way as to be + compatible with code that expects to interoperate with + connection-oriented stream sockets. For example, code that + expects to go through server-side bind/listen/accept connection + establishment should be reusable with PyDTLS sockets. + +Distributions +============= + +PyDTLS requires version 1.0.0 or higher of the OpenSSL +library. Earlier versions are reported not to offer stable DTLS +support. Since packaged distributions of this version of OpenSSL are +available for many popular operating systems, OpenSSL-1.0.0 is an +installation requirement before PyDTLS functionality can be called. +On Ubuntu 12.04 LTS, for example, the Python interpreter links with +libcrypto.so.1.0.0 and libssl.so.1.0.0, and so use of PyDTLS requires +no further installation steps. + +In comparison, installation of OpenSSL on Microsoft Windows operating +systems is inconvenient. For this reason, source distributions of +PyDTLS are available that include OpenSSL dll's for 32-bit and 64-bit +Windows. For 32-bit Windows, a version built with the MinGW toolchain +is also available. Its archive includes stripped as well as +non-stripped dll's. The latter can be debugged with gdb on +Windows. All dll's have been linked with the Visual Studio 2008 +version of the Microsoft C runtime library, msvcr90.dll, the version +used by CPython 2.7. Installation of Microsoft redistributable runtime +packages should therefore not be required on machines with CPython +2.7. The version of OpenSSL distributed with PyDTLS 0.1.0 is 1.0.1c. + +The OpenSSL version used by PyDTLS can be determined from the values +of *sslconnection's* DTLS_OPENSSL_VERSION_NUMBER, +DTLS_OPENSSL_VERSION, and DTLS_OPENSSL_VERSION_INFO. These variables +are available through the *ssl* module also if *do_patch* has been +called (see below). Note that the OpenSSL version used by PyDTLS may +differ from the one used by the *ssl* module. + +Interfaces +========== + +PyDTLS' top-level package, *dtls*, provides DTLS support through the +**SSLConnection** class of its *sslconnection* +module. **SSLConnection** is in-line documented, and +dtls/test/echo_seq.py demonstrates how to take a simple echo server +through a listen/accept/echo/shutdown sequence using this class. The +corresponding client side can look like the snippet at the top of this +document, followed by a call to the *unwrap* method for shutdown (or a +call to the **SSLConnection** *shutdown* method, if an instance of +this class is used for the client side also). Note that the *dtls* +package does not depend on the standard library's *ssl* module, and +**SSLConnection** can therefore be used in environments where *ssl* is +unavailable or incompatible. + +It is expected that with the *ssl* module being an established, familiar +interface to TLS, it will be the preferred module through which to +access DTLS. To do so, one must call the *dtls* package's *do_patch* +function before passing sockets of type SOCK_DGRAM to either *ssl's* +*wrap_socket* function, or *ssl's* **SSLSocket** constructor. + +It should be noted that once *do_patch* is called, *dtls* will raise +exceptions of type **ssl.SSLError** instead of its default +**dtls.err.SSLError**. This allows users' error handling code paths to +remain identical when interfacing with *ssl* across stream and +datagram sockets. + +Connection Handling +=================== + +The DTLS protocol implies a connection as an association between two +network peers where the overall association state is characterized by the +handshake status of each peer endpoint (see RFC 6347). The OpenSSL library +records this handshake status in "SSL" type instances (a.k.a. struct +ssl_st). Datagrams can be securely sent and received by referring to a +unique "SSL" instance after handshaking has been completed with this +instance and its network peer. A connection is implied in that traffic +may be directed to or received from only that network peer with whose +"SSL" instance the handshake has been completed. The fact that the +underlying network protocol, UDP in most cases, is itself connectionless +is immaterial. + +Further, in order to prevent denial-of-service attacks on UDP DTLS +servers, clients must undergo a cookie exchange phase early in the +handshaking protocol, and before server-side resources are committed to +a particular client (see section 4.2.1 of RFC 6347). The cookie exchange +proves to the server that a client can indeed receive IP traffic at +the source IP address with which its handshake-initiating ClientHello +datagram is marked. + +PyDTLS implements this connection establishment through the *connect* +method on the client side, and the *accept* method on the server side. +The latter returns a new **dtls.SSLConnection** or **ssl.SSLSocket** +object (depending on which interface is used, see above), which is +"connected" to its peer. In addition to the *read* and *write* methods +(at both interface levels), **SSLSocket's** *send* and *recv* methods +can be used; use of *sendto* and *recvfrom* on connected sockets is +prohibited by *ssl*. *accept* returns peer address information, as +expected. Note that when using the *ssl* interface to *dtls*, *listen* +must be called before calling *accept*. + +Demultiplexing +============== + +At the network io layer, only datagrams from its connected peer must be +passed to a **SSLConnection** or **SSLSocket** object (unless the object +is unconnected on the server-side, in which case in can be in listening +mode, the initial server-side socket whose role it is to listen for +incoming client connection requests). + +The initial server-side listening socket is not useful for performing this +datagram routing function. This is because it must remain unconnected and +ready to receive additional connection requests from new, unknown clients. + +The function of passing incoming datagrams to the proper connection is +performed by the *dtls.demux* package. **SSLConnection** requests a new +connection from the demux when a handshake has cleared the cookie +exchange phase. An efficient implementation of this request is provided +by the *osnet* module of the demux package: it creates a new socket that +is bound to the same network interface and port as the listening socket, +but connected to the peer. UDP stacks such as the one included with Linux +route incoming datagrams to such a connected socket in preference to an +unconnected socket bound to the same port. + +Unfortunately such is not the behavior on Microsoft Windows. Windows +UDP routes datagrams to whichever currently existing socket bound to +the particular port the earliest (and whether or not that socket is +unconnected, or connected to the datagram's peer, or a different +peer). Other sockets bound to the same port will not receive traffic, +if and until they become the earliest bound socket because another +socket is closed. + +The demux package therefore provides and automatically selects the module +*router* on Windows platforms. This module also creates a new socket when +receiving a new connection request; but instead of binding this socket +to the same port as the listening socket, it binds to a new ephemeral +port. *router* then forwards datagrams originating from the peer for which +a connection was requested to the corresponding socket. + +For efficiency's sake, no forwarding is performed on outgoing traffic. +Instead, **SSLConnection** directs outgoing traffic from the original +listening socket, using *sendto*. At the OpenSSL level this requires +separate read and write datagram BIO's for an "SSL" instance, one in +"connected" state and one in "peer set" state, respectively, and +associated with two separate network sockets. + +From the perspective of a PyDTLS user, this selection of and +difference between demux implementations should be transparent, with +the possible exception of performance deviation. This transparency +does however have some limits: for example, when *router* is in use, +the *accept* methods can return *None*. This happens when +**SSLConnection** detects that the demux has forwarded a datagram to a +known connection instead of initiating a connection to a new peer +through *accept*. Returning *None* in this case is important whenever +non-blocking sockets or sockets with timeouts are used, since another +socket might now be readable as a result of the forwarded +datagram. *accept* must return so that the application can iterate on +its asynchronous *select* loop. + +Shutdown and Unwrapping +======================= + +PyDTLS implements the SSL/TLS shutdown protocol as it has been adapted +for DTLS. **SSLConnection's** *shutdown* and **SSLSocket's** *unwrap* +invoke this protocol. As is the case with DTLS handshaking in general, +applications must be prepared to use the *get_timeout* and +*handle_timeout* methods in addition to re-invoking *shutdown* or +*unwrap* when sockets become readable and an exception carried +SSL_ERROR_WANT_READ. (See more on asynchronous IO in the Testing section.) + +**SSLConnection's** *shutdown* and **SSLSocket's** *unwrap* return a +(possibly new) socket that can be used for unsecured communication +with the peer, as set forth by the *ssl* module. The demux +infrastructure remains in use for this communication until the +returned socket is cleaned up. Note that when the *router* demux is +in use, the object returned will be one derived from +*socket.socket*. This is because the send and recv paths must still be +directed to two different OS sockets. In addition, the right thing +happens if secured communication is resumed over such a socket by +passing it to *ssl.wrap_socket* or the **SSLConnection** +constructor. If *osnet* is used, an actual *socket.socket* instance is +returned. + +Framework Compatibility +======================= + +PyDTLS sockets have been tested under the following usage modes: + + * Using blocking sockets and sockets with timeouts in + multi-threaded UDP servers + * Using non-blocking sockets, and in conjunction with the + asynchronous socket handler, asyncore + * Using blocking sockets, and in conjunction with the network + server framework SocketServer - ThreadingTCPServer (this works + because of PyDTLS's emulation of connection-related calls) + +Multi-thread Support +==================== + +Using multiple threads with OpenSSL requires implementing a locking +callback. PyDTLS does implement this, and therefore multi-threaded +programming with PyDTLS is safe in any environment. However, being a +pure Python library, these callbacks do carry some overhead. The *ssl* +module implements an equivalent locking callback in its C extension +module. Not requiring interpreter re-entry, this approach can be +expected to perform better. PyDTLS therefore queries OpenSSL as to +whether a locking callback is already in place, and does not overwrite +it if there is. Loading *ssl* can therefore improve performance, even +when only the *sslconnection* interface is used. + +Note that loading order does not matter: to obtain the performance +benefit, *ssl* can be loaded before or after the dtls package. This is +because *ssl* does not do an equivalent existing locking callback +check, and will simply overwrite the PyDTLS callback if it has already +been installed. But *ssl* should not be loaded while *dtls* operation +is already in progress, when some locks may be in their acquired +state. + +Also note that this performance enhancement is available only on +platforms where PyDTLS loads the same OpenSSL shared object as +*ssl*. On Ubuntu 12.04, for example, this is the case, but on +Microsoft Windows it is not. + +Testing +======= + +Almost all of the Python standard library's *ssl* unit tests from the +module *test_ssl.py* have been ported to *dtls.test.unit.py*. All tests +have been adjusted to operate with datagram sockets. On Linux, each +test is executed four times, varying the address family among IPv4 and +IPv6 and the demux among *osnet* and *router*. On Windows, where +*osnet* is unavailable, each test is run twice, once with IPv4 and once +with IPv6. + +The unit test suite includes tests for each of the above-mentioned +compatible frameworks. The class **AsyncoreEchoServer** serves as an +example of how to use non-blocking datagram sockets and implement the +resulting timeout detection requirements. DTLS in general and OpenSSL +in particular require being called back when used with non-blocking +sockets (or sockets with timeout option) after DTLS timeouts expire to +handle packet loss using re-transmission during a +handshake. Handshaking may occur during any read or write operation, +even after an initial handshake completes successfully, in case +renegotiation is requested by a peer. + +Running with the -v switch executes all unit tests in verbose mode. + +dtls/test/test_perf.py implements an interactive performance test +suite that compares the raw throughput of TCP, UDP, SSL, and DTLS. +It can be executed locally through the loopback interface, or between +remote clients and servers. In the latter case, test jobs are sent to +remote connected clients whenever a suite run is initiated through the +interactive interface. Run test_perf.py -h for more information. + +It should be noted that comparing the performance of protocols that +don't offer congestion control (UDP and DTLS) with those that do (TCP +and SSL) is a difficult undertaking. Raw throughput even across +gigabit network links can be expected to suffer without congestion +control and peers that generate data as fast as possible without +throttling (as this test does): the link's throughput will drop +significantly as it enters congestion collapse. Similarly, loopback is +an imperfect test interface since it rarely drops packets, and never +duplicates or reorders them (thus negating the relative performance +benefits of DTLS over SSL). Nevertheless, some useful insights can be +gained by observing the operation of test_perf.py, including software +stack behavior in the presence of some amount of packet loss. + +Logging +======= + +The *dtls* package and its sub-packages log various occurrences, +primarily events that can aid debugging. Especially *router* emits many +messages when the logging level is set to at least *logging.DEBUG*. +dtls/test/echo_seq.py activates this logging level during its operation. + +Currently Supported Platforms +============================= + +At the time of initial release, PyDTLS 0.1.0 has been tested on Ubuntu +12.04.1 LTS 32-bit and 64-bit, as well as Microsoft Windows 7 32-bit +and 64-bit, using CPython 2.7.3. Patches with additional platform +ports are welcome. diff --git a/dtls/patch.py b/dtls/patch.py index 19d404f..d7d73d4 100644 --- a/dtls/patch.py +++ b/dtls/patch.py @@ -25,7 +25,7 @@ has the following effects: * The constant PROTOCOL_DTLSv1 is added at ssl module level * DTLSv1's protocol name is added to the ssl module's id-to-name dictionary * The constants DTLS_OPENSSL_VERSION* are added at the ssl module level - * Instntiation of ssl.SSLSocket with sock.type == socket.SOCK_DGRAM is + * Instantiation of ssl.SSLSocket with sock.type == socket.SOCK_DGRAM is supported and leads to substitution of this module's DTLS code paths for that SSLSocket instance * Direct instantiation of SSLSocket as well as instantiation through