CERT Airs Serious Flaws in OpenSSL Protocol

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Security experts have detected four serious vulnerabilities in OpenSSL, an open source implementation of the Secure
Sockets Layer (SSL v2/v3) and its successor, Transport Layer Security (TLS v1), according to the Carnegie Mellon
Software Engineering Institute (CERT).

The flaws, commonly known as buffer overflows, could be used by a malicious perpetrator to execute code on a system, or simply to
inflict denial-of-service attacks. Versions OpenSSL that CERT considers susceptible to a breach include
OpenSSL prior to 0.9.6e, up to and including pre-release 0.9.7-beta2, as well as OpenSSL pre-release 0.9.7-beta2 and prior with
Kerberos-enabled SSLeay library.

SSL and TLS protocols are used to provide a secure connection between a client and a server for higher level protocols,
such as HTTP. Developed by Netscape, SSL gained the support of Microsoft and other Internet client/server developers and became the
de facto standard until evolving into TLS. OpenSSL is an open-source derivative of SSL and TLS. OpenSSL stands apart from the other
two in that it also functions as a cryptography library.

The vulnerabilities were enough to make one security company issue a cautionary public statement:

“The wide proliferation of servers that utilize SSL to protect encrypted sessions, may make these vulnerabilities significant,” said Bruce Murphy, CEO of Vigilinx. “At this point, we are not aware of any damage that has been inflicted due to this vulnerability, but we believe that the potential for damage is high and a proactive response is warranted.”

The flaws include:

VU#102795 — OpenSSL servers contain a buffer overflow during the SSLv2 handshake process. This can be exploited by a client
using a malformed key during the handshake process with an SSL server connection
VU#258555 — OpenSSL clients contain a buffer overflow during the SSLv3 handshake process. A malicious server can exploit this
by sending a large session ID to the client during the handshake process.
VU#561275 — OpenSSL servers with Kerberos enabled contain an exploitable buffer overflow during the SSLv3 handshake
process. Users may exploit this if a malicious client sends a malformed key during the SSLv3 handshake process with the server
VU#308891 — OpenSSL contains multiple buffers overflows in buffers that are used to hold ASCII representations of integers

CERT Tuesday also warned of holes involving malformed ASN.1 encodings in OpenSSL. Those affected include SSL or TLS
applications, as well as S/MIME, PKCS#7, and certificate creation routines. Wit this flaw, the ASN.1 library has various encoding
errors that allow malformed certificate encodings to be parsed incorrectly, leaving it open to denial-of-serviceissues.

CERT recommends that those affected by the flaws upgrade to version 0.9.6e of OpenSSL. Patches for the flaw are available from OpenSSL.org are available: For the OpenSSL 0.9.6d patch go here.

Sites running OpenSSL pre-release version 0.9.7-beta2 may wish to upgrade to 0.9.7-beta3. Combined patches for OpenSSL 0.9.7 beta 2
may be found here.

After either applying the patches above or upgrading to 0.9.6e, CERT advises users to recompile all applications using OpenSSL to
support SSL or TLS services, and restart services or systems to punt vulnerable code.

CERT was made aware of the holes by: VU#102795, discovered by A.L. Digital Ltd and John McDonald of Neohapsis; VU#258555, VU#561275,
VU#308891, discovered by A.L. Digital Ltd; and VU#748355, discovered by Adi Stav and James Yonan independently.