Protocol state fuzzing of TLS implementations

Joeri De Ruiter, Erik Poll

Research output: Chapter in Book/Report/Conference proceedingConference contribution

97 Citations (Scopus)
181 Downloads (Pure)

Abstract

We describe a largely automated and systematic analysis of TLS implementations by what we call ‘protocol state fuzzing’: we use state machine learning to infer state machines from protocol implementations, using only blackbox testing, and then inspect the inferred state machines to look for spurious behaviour which might be an indication of flaws in the program logic. For detecting the presence of spurious behaviour the approach is almost fully automatic: we automatically obtain state machines and any spurious behaviour is then trivial to see. Detecting whether the spurious behaviour introduces exploitable security weaknesses does require manual investigation. Still, we take the point of view that any spurious functionality in a security protocol implementation is dangerous and should be removed.

We analysed both server- and client-side implementations with a test harness that supports several key exchange algorithms and the option of client certificate authentication. We show that this approach can catch an interesting class of implementation flaws that is apparently common in security protocol implementations: in three of the TLS implementations analysed new security flaws were found (in GnuTLS, the Java Secure Socket Extension, and OpenSSL). This shows that protocol state fuzzing is a useful technique to systematically analyse security protocol implementations. As our analysis of different TLS implementations resulted in different and unique state machines for each one, the technique can also be used for fingerprinting TLS implementations.
Original languageEnglish
Title of host publication24th USENIX Security Symposium (USENIX Security 15)
PublisherUSENIX
Pages193-206
Number of pages14
ISBN (Electronic)978-1-931971-232
Publication statusPublished - 2015
EventUsenix Security Symposium - Washington, United States
Duration: 10 Aug 201514 Aug 2015

Conference

ConferenceUsenix Security Symposium
Country/TerritoryUnited States
CityWashington
Period10/08/1514/08/15

Fingerprint

Dive into the research topics of 'Protocol state fuzzing of TLS implementations'. Together they form a unique fingerprint.

Cite this