Produce a Pure Python Wheel instead of a source distribution for Linux.
This speeds up installation and simplifies distribution building.
* setup.py: Drop support for sdist; add support for bdist_wheel
without --plat-name switch; persist README.rst; increment version
* dtls/__init__.py: Increment version
Installation as well as distribution generation are now simplified.
With the elimination of MinGW support there is now a unique set of
platform binaries for 32-bit and 64-bit Windows. Pip will now
automatically choose the proper binary wheel or the source
distribution, depending under which platform and Python version it
is being invoked.
* README.md: add installation section
* setup.py: rewrite to use bdist_wheel for Windows
* README.txt -> README.md: renamed
* dtls/sslconnection.py: Reduce the default MTU in effect while
handshaking to 576, suitable for various path MTUs and PPPoE
* dtls/prebuilt/win32-x86[_64]: Rebuilt with Visual C++ 2008 to
eliminate requirement to install a C++ redistributable package
* dtls/prebuilt/mingw-x86: mingw support is deprecated
* dtls/__init__.py: VERSION introduced
* setup.py: Version incremented to 1.2.0
* dtls/sslconnection.py: Hardcoded setting of the MTU size only for Windows and in case it is not already configured
* dtls/test/unit_wrapper.py: No user config of the MTU size; using the hardcoded one from SSLConnection
* dtls/__init__.py: Removed wrapper import
* dtls/openssl.py: Fixed line endings to LF
* dtls/patch.py: Removed PROTOCOL_SSLv3 import and fixed line endings to LF
* dtls/sslconnection.py: Fixed line endings to LF
* dtls/test/certs/*_ec.pem: Fixed line endings to LF
* dtls/test/echo_seq.py: Fixed line endings to LF
* dtls/test/simple_client.py: Fixed line endings to LF
* dtls/test/unit.py: Fixed line endings to LF
* dtls/test/unit_wrapper.py: Corrected wrapper import and fixed line endings to LF
* dtls/util.py: Fixed line endings to LF
* dtls/wrapper.py: Corrected function naming to wrap_client() and wrap_server(); Fixed line endings to LF
* dtls/x509.py: Fixed line endings to LF
* dtls/__init__.py: Added DtlsSocket() from wrapper and aliases for wrap_server() and wrap_client()
* dtls/err.py: Added patch_ssl_errors() to patch ssl-Module with ERR_* constants
* dtls/patch.py: Patched ssl-Module with SSL_BUILD_* constants and added call to patch_ssl_errors()
* dtls/wrapper.py:
- Added a server and client function to alias/wrap DtlsSocket() creation
- Cleanup of DtlsSocket.__init__()
- Cleanup of exception handling in all member methods
- Cleanup sendto() from client: no endless loop and first do a connect if not already connected
* dtls/test/unit_wrapper.py: Adopt the changes made described above
* dtls/__init__.py: Import SSLContext() and SSL() for external use
* dtls/wrapper.py: Added class DtlsSocket() to be used as client or server
* dtls/test/unit_wrapper.py: unit test for DtlsSocket()
* dtls/__init__.py: import error codes from err.py as error_codes for external access
* dtls/err.py: Added errors for ERR_WRONG_SSL_VERSION, ERR_CERTIFICATE_VERIFY_FAILED, ERR_NO_SHARED_CIPHER and ERR_SSL_HANDSHAKE_FAILURE
* dtls/openssl.py:
- Added constant SSL_BUILD_CHAIN_FLAG_NONE for SSL_CTX_build_cert_chain()
- Added method SSL_get_peer_cert_chain()
* dtls/patch.py: Added getpeercertchain() as method to ssl.SSLSocket()
* dtls/sslconnection.py:
- Bugfix SSLContext.set_ecdh_curve() returns 1 for success and 0 for failure
- SSLContext.build_cert_chain() changed default flags to SSL_BUILD_CHAIN_FLAG_NONE
- In SSLConnection() the mtu size gets only set if no user config function is given
- SSLConnection.listen() raises an exception for ERR_WRONG_VERSION_NUMBER, ERR_COOKIE_MISMATCH, ERR_NO_SHARED_CIPHER and all other unknown errors
- SSLConnection.read() and write() now can also raise ERR_PORT_UNREACHABLE
- If SSLConnection.write() successfully writes bytes to the peer, then the handshake is assumed to be okay
- Added method SSLConnection.getpeercertchain()
* dtls/test/unit.py: ThreadedEchoServer() with an extra exception branch for the newly raised exceptions in SSLConnection.listen()
* dtls/test/makecerts_ec.bat: creates ca-cert_ec.pem, keycert_ec.pem and server-cert_ec.pem
* dtls/test/openssl_ca.cnf and openssl_server.cnf: Added HOME to be able to use the conf file under windows
* dtls/openssl.py:
- Added utility fucntions EC_curve_nist2nid() and EC_curve_nid2nist()
* dtls/patch.py:
- Extended wrap_socket() arguments with callbacks for user config functions of ssl context and ssl session values
- Extended SSLSocket() arguments with callbacks for user config functions of ssl context and ssl session values
* dtls/sslconnection.py:
- Extended SSLConnection() arguments with callbacks for user config functions of ssl context and ssl session values
- During the init of client and server the corresponding user config functions are called (if given)
- Added new classes SSLContext() [set_ciphers(), set_sigalgs(), set_curves(), set_ecdh_curve(), build_cert_chain(),
set_ssl_logging()] and SSL() [set_mtu(), set_link_mtu()]
* dtls/openssl.py:
- Added class _EllipticCurve() for easy handling of the builtin curves
- Added wrapper get_elliptic_curves() - which uses _EllipticCurve()
- Added EC_get_builtin_curves(), EC_KEY_new_by_curve_name() and EC_KEY_free()
- Added OBJ_nid2sn() for translating numeric ids to names
* dtls/util.py: Added _EC_KEY() derived from _Rsrc() with own free/del method
* dtls/err.py: Added error code ERR_WRONG_VERSION_NUMBER
* dtls/openssl.py: Added DTLS_server_method(), DTLSv1_2_server_method() and DTLSv1_2_client_method()
* dtls/patch.py: Default protocol DTLS for ssl.wrap_socket() and ssl.SSLSocket()
* dtls/sslconnection.py:
- Introduced PROTOCOL_DTLSv1_2 and PROTOCOL_DTLS (the latter one is a synonym for the "higher" version)
- Updated _init_client() and _init_server() with the new protocol methods
- Default protocol DTLS for SSLConnection()
- Return on ERR_WRONG_VERSION_NUMBER if client and server cannot agree on protocol version
* dtls/test/unit.py:
- Extended test_get_server_certificate() to iterate over the different protocol combinations
- Extended test_protocol_dtlsv1() to try the different protocol combinations between client and server
* dtls/openssl.py: Added mtu-functions SSL_set_mtu() and DTLS_set_link_mtu()
* dtls/prebuilt/win32-*: Updated libs for x86 and x86_64 to version 1.0.2l-dev
* dtls/sslconnection.py: mtu size set hardcoded to 1500 - otherwise the windows implementation has problems
* dtls/openssl.py:
- Added methods SSL_CTX_set_info_callback(), SSL_state_string_long(), SSL_alert_type_string_long() and SSL_alert_desc_string_long()
- Added constants for state and error evaluation during callback
* dtls/sslconnection.py: Added _ssl_logging_cb() as default callback function - only outputs messages when logger is active
* dtls/openssl.py: SSL_write() can handle ctypes.Array data
* dtls/sslconnection.py: Added missing import ERR_BOTH_KEY_CERT_FILES
* dtls/test/simple_client.py: Added basic test client to use with dtls/test/echo_seq.py
* dtls/openssl.py:
- Ordered constants according to header file from openSSL
- Beautified __all__-list and map for _make_function() in order to easy merges in the future
- Added a few returns in order to evaluate the success of the called methods
* dtls/patch.py: Grouped imports in the following order - system, local
* dtls/sslconnection.py: ssl protocol not hardcoded anymore for forked objects
* dtls/x509.py: logger messages working again
* dtls/openssl.py: support reading directly into given buffer instead
of forcing buffer copy (for ssl module compatibility)
* dtls/sslconnection.py: in-situ receive support, as above
* dtls/patch.py: various changes for compatibility with the ssl module
of Python 2.7.12; note that the ssl module's new
SSLContext is not supported
* dtls/test/unit.py: changes to support the updated ssl module,
including fix of deprecation warnings
* setup.py: increase version to 1.0.2
Increase version to 1.0.1, in preparation for PyPI release. This release
includes a fix by Jayson Vantuyl to an error code import. A proper error
is now emitted when omitting the keyfile on a PyDTLS server.
The version number is incremented to 1.0.0.
Thanks to doneir for reporting that a number of unit test were failing.
This was because two test certificates had expired. This commit replaces
these with updated certificates, along with the tool used for programmatic
certificate generation. The new certificates are set to remain valid for
approximately ten years.
This patch allows the module test_perf.py to be run on Windows. It
contains fixes primarily accounting for differences in the
implementation of the multiprocessing module among platforms.
This patch completes the README.txt placeholder file. The first version of
external (as opposed to in-line) documentation is complete.
The unused file README.md that was created by the GitHub repository
establishment procedure is deleted.
This project is now licensed under the Apache license. Individual files
now have a license reference header.
The Apache 2.0 license text is copied to the file LICENSE. The file
NOTICE, referred to in the license text, has been added. A placeholder
README.txt has been added.
These three new files are integrated into the distribution/installation
machinery, and are placed into the package directory upon installation.
The new module setup.py handles creation of package distributions in
compressed formats, and performs installation of said distributions
after decompression. Distribution strategy for PyDTLS is in the form
of source distributions only.
Building the OpenSSL library on Windows is non-trivial. Requirements
include linking against the same version of the C runtime that is used
by the Python interpreter, or else Microsoft redistributable
installation may also need to be performed. For the convenience of
Windows users, the distribution procedure includes creation of
prebuilt versions of the OpenSSL library for both 32- and 64-bit
versions of Windows. Despite containing binaries, these are still
source distributions that are installed with "python setup.py
install."
A version of OpenSSL compiled with the MinGW toolchain is included.
cygcrypto-1.0.0.dll and cygssl-1.0.0.dll contain symbols and can be
debugged with gdb. All MinGW OpenSSL dll's link with msvcr90.dll.
In order to avoid making installation a requirement after cloning, the
package initializer looks for prebuilts of the currently executing
platform and copies them into the dtls package directory before
continuing module initialization.
A distribution containing prebuilts for all platforms as well as those
containing no prebuilts (appropriate for Linux) can be generated by
issuing "python setup.py sdist --prebuilts."
When running in remote server mode (-s), the value of the -s option
specifies which port remote clients must connect to for the job
transmission protocol. The servers for individual suite runs, however,
would be placed at dynamically assigned ports, with port numbers
transmitted to clients through job parameters.
This does not work well when servers sit behind firewalls with restricted
UDP port ranges. This patch introduces the -p option, allowing ports for
server runs to remain fixed at the given port (which, of course, must
not conflict with the port number supplied with -s).
The new module test_perf.py can be used to characterize protocol
performance over a particular network link. Two stream protocols
(TCP and SSL) and two datagram protocols (UDP and DTLS) are available
for relative comparison.
The module will run servers in its process, and will spawn clients either
into separate processes, or, depending on command line options, will
expect one or more remote clients to connect to it. In the latter case,
jobs will be sent to such clients via a shared queue whenever the user
selects a test suite.
Stress testing under packet loss conditions revealed that that the
OpenSSL library's compression feature needed to be explicitly disabled
for DTLS: it evidently operates at the stream layer as opposed to the
datagram layer, and packet loss would result in corruption among the
packets that were successfully received, authenticated, and decrypted.
Several performance improvements are included in this patch.
Thread locking callbacks into the interpreter impose considerable overhead.
The standard library's ssl module registers its own thread locking and id
callback functions. Registration of the latter can be detected by a call to
the OpenSSL library. In this case, do not set the Python callback function
and therefore keep using the more efficient ssl module functions.
On a 64-bit OS, pointer return values needed to be marked as c_void_p instead
of a user-defined type, which would result in the transfer of 32 bits only.
In order to still return an instance of the user-defined type to the caller,
imported functions are now marked with the return type, and the return
value is converted to that type by a new error checking function used only
with imported functions that create and return user-defined types.
On 64-bit Linux, the long type becomes 8 bytes, whereas the int type remains
4 bytes. The various sockaddr_* fields therefore needed to be changed from
long to int, as did the type signatures of the packed string to array
conversion functions.
On an Ubuntu server installation, it was found that the name "localhost"
does not resolve to an ipv6 address. A name search has therefore been added
to the unit test driver, along with an ip number fallback.
Tested on Ubuntu Server 12.04.1 LTS 64-bit.
Regression tested on Ubuntu 12.04.1 LTS 32-bit.
This change introduces a demux that uses the kernel's network stack for UDP
datagram-to-socket assignment based on packet source address (as opposed to the
forwarding strategy of the routing demux). The osnet demux is used by default
on non-Windows platforms. When possible, use of the osnet demux is preferred
over the routing demux, since it can be expected to perform better.
The unit test suite has been extended to run all tests first with the demux
selected by default for the current platform, and then with the routing demux,
if the latter differs from the former. Tests were already being run twice, first
with IPv4 and then with IPv6, and thus we now run each test four times on
Linux, twice on Windows.
All unit tests pass with both demux types.
With this change all unit tests pass on the Linux platform (tested on
Ubuntu 12.04.1 LTS). demux/__init__.py has been adjusted temporarily so as
to load the routing demux on Linux until the osnet demux is ready.
Testing on Linux exposed an issue where comparison of the ssl object value from
cookie callbacks failed to compare equal to the value stored in the
SSLConnection callback object. This was because the callback function signature
of c_void_p for this parameter produced a 64-bit value if the 32nd bit was set
(as opposed to producing a negative integer 32-bit value). Changing the
signature from c_void_p to c_int for this parameter fixes the issue.
Prior to this change, the unit tests' echo servers' connections would sometimes
linger beyond server termination. A timeout mechanism is now implemented, which
will terminate a connection and clean up its resources when the timeout is
reached. For the purpose of unit testing, test echo servers now assert that
all connections have been terminated when the servers are closed.
For threaded echo servers, the timeout mechanism involves the use of sockets
with timeouts instead of blocking sockets. This required an implementation with
the proper handling of timeout sockets at the sslconnection level.
The new module tlock provides the locking callback function the OpenSSL library
requires for the case where multiple threads enter it concurrently. tlock is
now automatically initialized by sslconnection's module startup code, and does
nothing if the executing Python environment does not provide threading.
Note that this does introduce some overhead. For example, during the transfer
of CERTFILE by test_socketserver, about 300 locking callbacks are received.
The unit test suite was previously being run with IPv4 protocol addresses only.
With this change, we run the entire test suite twice: first with v4 addresses,
and then with v6 addresses, for all client and server-side sockets.
Ray
mcfreis
Jayson Vantuyl