Have you ever wondered how HTTPS works? In my early days of programming, this topic seemed very complicated to me. The basic idea that HTTPS is essentially HTTP running over SSL/TLS didn't help much. How does TLS work and what do the server and the client send to each other at the TCP level? I revisited this question years later, and I hope you'll find my discoveries interesting.
What is HTTP?
Before we dive into HTTPS, let's briefly review HTTP. HTTP is a protocol used by web browsers and web servers to communicate with each other. The client sends a request to the server, and the server responds with a response. The requests and responses are text-based. For example, here's a simple HTTP request:
GET / HTTP/1.1\r\n
Host: example.com\r\n
\r\nThe \r and \n are escape sequences indicating carriage return and newline, respectively. This request can be represented in a hex dump as follows:
00000000 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a |GET / HTTP/1.1..|
00000010 48 6f 73 74 3a 20 65 78 61 6d 70 6c 65 2e 63 6f |Host: example.co|
00000020 6d 0d 0a 0d 0a |m....|
00000025The server responds with an HTTP response, like this:
HTTP/1.1 200 OK\r\n
Age: 424871\r\n
Cache-Control: max-age=604800\r\n
Content-Type: text/html; charset=UTF-8\r\n
Date: Tue, 21 May 2024 16:44:39 GMT\r\n
Etag: "3147526947+ident"\r\n
Expires: Tue, 28 May 2024 16:44:39 GMT\r\n
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT\r\n
Server: ECAcc (dcd/7D0D)\r\n
Vary: Accept-Encoding\r\n
X-Cache: HIT\r\n
Content-Length: 1256\r\n
\r\n
<!doctype html>\n
<html>\n
<head>\n
<title>Example Domain</title>\n
...The important thing to note is that HTTP is a human-readable protocol, meaning you don't need special tools to understand it. You can use telnet or nc to interact with a web server directly. If you want to learn more about HTTP, you can refer to RFC 2616.
Transport Layer Security (TLS)
The TLS protocol provides privacy and data integrity between two communicating applications. It is described in RFC 5246. The TLS protocol has two layers: the TLS Record Protocol and the TLS Handshake Protocol. The TLS Record Protocol encapsulates various higher-level protocols, such as HTTP, into a secure connection. The TLS Handshake Protocol is responsible for the key exchange and authentication between the client and the server.
Cipher Suites
A cipher suite is a set of cryptographic algorithms used in the TLS protocol. A typical cipher suite consists of the following:
- Key exchange algorithm
- Authentication algorithm
- Encryption algorithm
- Message authentication code (MAC) algorithm
In this post, we'll use the cipher suite TLS_RSA_WITH_AES_128_CBC_SHA, which uses RSA for key exchange and authentication, AES-128-CBC for encryption, and SHA-1 for the MAC algorithm.
Textbook RSA
RSA is a type of cryptographic algorithm that uses two different keys: a public key and a private key. The public key is used for encryption and can be shared with anyone, while the private key is used for decryption and is kept secret.
In RSA, the public key is the pair of integers $(e, n)$, and the private key is the pair $(d, n)$. These keys have a special property: for any message $m$ (a natural number less than $n$), the following equation holds true: $$(m^e)^d \equiv m \pmod{n}$$
Before we go into the details of RSA, let's review some important concepts and algorithms:
Euler's theorem: If $n$ and $a$ are coprime (they have no common factors other than 1), then $a^{\phi(n)} \equiv 1 \pmod{n}$, where $\phi(n)$ is Euler's totient function, which counts the positive integers up to $n$ that are coprime with $n$.
For example, $\phi(15) = 8$, because the numbers 1, 2, 4, 7, 8, 11, 13, and 14 are coprime with 15. Let's pick an $a$ that is coprime with 15, such as $a = 7$. Then $7^8 \bmod 15 = 5764801 \bmod 15 = 1$.
If $n$ is a product of two primes $p$ and $q$, then $\phi(n) = (p - 1)(q - 1)$. For example, $\phi(15) = (3 - 1)(5 - 1) = 8$. You can read more about Euler's theorem and Euler's totient function on Wikipedia.
Bézout's identity: For integers $a$ and $b$, there exist integers $x$ and $y$ such that $$ax + by = \gcd(a, b)$$
Extended Euclidean algorithm: This algorithm finds the greatest common divisor (GCD) of two numbers and the coefficients of Bézout's identity.
You can learn more about Bézout's identity and the extended Euclidean algorithm on Wikipedia.
Now, let's create a pair of RSA keys. This process starts with selecting two prime numbers, $p$ and $q$. For this example, let's use $p = 23$ and $q = 17$. Next, we calculate $n$: $$n = pq = 391$$ and $\phi(n) = (p-1)(q-1) = 352$. We need to find an integer $e$ such that $1 < e < \phi(n)$ and $e$ is coprime with $\phi(n)$. Let's choose $e$: $$e = 3$$ Next, we need to find a number $d$ that satisfies the equation $de \equiv 1 \pmod{\phi(n)}$. In other words, we need to find $d'$ and $k$ that satisfy: $$d'e = 1 + k\phi(n)$$ Such a $d'$ will allow us to decrypt messages: $$m^{d'e} = m^{1 + k\phi(n)} = m \cdot m^{k\phi(n)} = m \cdot (m^{\phi(n)})^k \equiv m \pmod{n}$$ Rearranging the equation for $d'$, we get: $$d'e - k\phi(n) = 1$$ This equation is in the form of Bézout’s identity, which tells us that for $e$ and $\phi(n)$, there exist integers $d'$ and $-k$ such that $$d'e + (-k)\phi(n) = \gcd(e, \phi(n))$$ Applying the extended Euclidean algorithm to $\phi(n) = 352$ and $e = 3$, we obtain: $$\begin{align*} r_0 &= 352, s_0 = 1, t_0 = 0, \\ r_1 &= 3, s_1 = 0, t_1 = 1. \end{align*}$$ Then, we iteratively calculate until $r_i = 0$: $$\begin{align*} q_1 &= \lfloor r_0 / r_1 \rfloor = \lfloor 352 / 3 \rfloor = 117, \\ r_2 &= r_0 - q_1r_1 = 352 - 117 \times 3 = 352 - 117 \times 3 = 1, \\ s_2 &= s_0 - q_1s_1 = 1 - 117 \times 0 = 1, \\ t_2 &= t_0 - q_1t_1 = 0 - 117 \times 1 = -117, \\ q_2 &= \lfloor r_1 / r_2 \rfloor = \lfloor 3 / 1 \rfloor = 3, \\ r_3 &= r_1 - q_2r_2 = 3 - 3 \times 1 = 0. \end{align*}$$ Therefore, $s_2\phi(n) + t_2e = 1$, or $k = -s_2 = -1$ and $d' = t_2 = -117$ : $$-117 \times 3 + 1 \times 352 = 1$$ As we need a positive number and we work modulo 352, we can add 352 to -117 to get $$d = -117 + 352 = 235$$
Our public key is $(e, n) = (3, 391)$, and our private key is $(d, n) = (235, 391)$. Now we can encrypt and decrypt messages using these keys.
Let's say our message is $m = 42$. To encrypt it, we calculate $c = m^e \bmod n = 42^3 \bmod 391 = 74088 \bmod 391 = 189$. To decrypt it, we calculate $m = c^d \bmod n = 189^{235} \bmod 391 = 42$.
Real RSA
The textbook RSA it is vulnerable to various attacks. For example, if you have a ciphertext $$c = m^e \bmod n$$ you can create a new ciphertext $$c' = (c \cdot 2^e) \bmod n = (m^e \cdot 2^e) \bmod n = (2m)^e \bmod n$$ In other words, you can make predictable changes to the message without knowing the private key.
TLS v1.2 uses RSAES-PKCS1-v1_5 from RFC 3447. It pads the messag M as follows:
- Define mLen as the length of the message M in bytes, $k$ as the length of the modulus $n$ in bytes.
- If mLen > $k$ - 11, output "message too long" and stop.
- Generate a random string of non-zero bytes PS of length $k$ - mLen - 3.
- Concatenate {0x00, 0x02}, PS, {0x00}, M. The resulting string is the padded message EM, and its length is $k$ bytes.
Our textbook RSA example uses a small modulus $n = 391$, which is only 9 bits (2 bytes) long. Any message will result into "message too long". Let's pretend that our n is 512 bits (64 bytes) long and pad the message M:
M = {0x48, 0x64, 0x6c, ... 0x64, 0x21} /* Hello, world! */The message "Hello, world!" is 13 bytes long. We need to generate a random string PS of 64 - 13 - 3 = 48 bytes:
PS = {0xa0, 0xb4, 0x35, ..., 0x7d} /* 48 random bytes */The padded message will look like this:
00000000 00 02 a0 b4 35 47 59 ad 9c 2c c5 e8 dd e7 49 d9 |....5GY..,....I.|
00000010 21 a2 32 59 a0 d4 dd 9d 44 9e 3c 8c cb 6a bf 2f |!.2Y....D.<..j./|
00000020 f0 e0 05 70 9f f0 80 24 e1 a2 6c c9 ef ba 8f 38 |...p...$..l....8|
00000030 86 7d 00 48 65 6c 6c 6f 2c 20 77 6f 72 6c 64 21 |.}.Hello, world!|
00000040This padded message EM is then converted to an integer 0x0002a0b4..6421 and encrypted using the textbook RSA algorithm. The decryption process is the reverse of the encryption process: the ciphertext is decrypted using the private key, and the padding is removed to obtain the original message.
Now let's return to the TLS protocol.
TLS Record Protocol
On the TCP level, the client and the server exchange TLSPlaintext and TLSCiphertext records. The TLSPlaintext structure is defined as follows:
struct {
ContentType type;
ProtocolVersion version;
uint16 length;
opaque fragment[TLSPlaintext.length];
} TLSPlaintext;
enum {
change_cipher_spec(20), alert(21), handshake(22),
application_data(23), (255)
} ContentType;
struct {
uint8 major;
uint8 minor;
} ProtocolVersion;TLSCiphertext structure is similar to TLSPlaintext, but it contains encrypted data:struct {
ContentType type;
ProtocolVersion version;
uint16 length;
select (SecurityParameters.cipher_type) {
case stream: GenericStreamCipher;
case block: GenericBlockCipher;
case aead: GenericAEADCipher;
} fragment;
} TLSCiphertext;ProtocolVersion represents the version of the TLS protocol, and for TLS 1.2, it is {3, 3}. This might seem weird at first, but it makes more sense when you see how the versions have evolved:
SSL 3.0: {3, 0} (defined at RFC 6101)
TLS 1.0: {3, 1} (defined at RFC 2246)
TLS 1.1: {3, 2} (defined at RFC 4346)
TLS 1.2: {3, 3} (defined at RFC 5246)
The field type allows distinguishing between different types of records. For example, application_data is used to encapsulate higher-level protocols, such as HTTP, and alert indicates an error condition independent of the application protocol.
For an application data fragment Hello, world! (13 bytes), the record would look like this:
TLSPlaintext tls_plaintext = {
type: 0x17, /* application_data(23) */
version: {3, 1}, /* TLS v1.0 */
length: 0x0d, /* 13 bytes */
fragment: {0x48, 0x65, 0x6c, 0x6c, ..., 0x21} /* Hello, world! */
};00000000 17 03 01 00 0d 48 65 6c 6c 6f 2c 20 77 6f 72 6c |.....Hello, worl|
00000010 64 21 |d!|
00000012TLS Handshake Protocol
A typical TLS handshake process involves the following steps:
Certificate
ServerKeyExchange
ServerHelloDone Client->>Server: ClientKeyExchange
ChangeCipherSpec
Finished Server->>Client: ChangeCipherSpec
Finished
Every message in the handshake protocol uses a Handshake structure (encapsulated within the TLSPlaintext structure). The Handshake structure is defined as follows:
struct {
HandshakeType msg_type; /* handshake type */
uint24 length; /* bytes in message */
select (HandshakeType) {
case hello_request: HelloRequest;
case client_hello: ClientHello;
case server_hello: ServerHello;
case certificate: Certificate;
case server_key_exchange: ServerKeyExchange;
case certificate_request: CertificateRequest;
case server_hello_done: ServerHelloDone;
case certificate_verify: CertificateVerify;
case client_key_exchange: ClientKeyExchange;
case finished: Finished;
} body;
} Handshake;
enum {
hello_request(0), client_hello(1), server_hello(2),
certificate(11), server_key_exchange (12),
certificate_request(13), server_hello_done(14),
certificate_verify(15), client_key_exchange(16),
finished(20), (255)
} HandshakeType;ClientHello
The client initiates the TLS handshake by sending a ClientHello message to the server, with the following structure:
struct {
ProtocolVersion client_version;
Random random;
SessionID session_id;
CipherSuite cipher_suites<2..2^16-2>;
CompressionMethod compression_methods<1..2^8-1>;
select (extensions_present) {
case false:
struct {};
case true:
Extension extensions<0..2^16-1>;
};
} ClientHello;
struct {
uint32 gmt_unix_time;
opaque random_bytes[28];
} Random;
opaque SessionID<0..32>;
uint8 CipherSuite[2];
enum { null(0), (255) } CompressionMethod;
struct {
ExtensionType extension_type;
opaque extension_data<0..2^16-1>;
} Extension;
enum {
signature_algorithms(13), (65535)
} ExtensionType;There’s a lot to unpack here, so let’s break it down.
First, let’s address the unusual syntax with angle brackets, such as SessionID<0..32>. This notation indicates the length of the field in bytes. When the length is variable, the length is encoded before the field using enough bytes to represent the maximum possible length. Let's illustrate this with an example:
uint16 foo<0..800>; // zero to 400 16-bit unsigned integersSuppose we want to encode foo = {0x0001, 0x0002, 0x0003}. Since the maximum length is 800, we need two bytes to represent the length. In our example, with 3 uint16 values totaling 6 bytes, foo should be encoded as
00000000 00 06 00 01 00 02 00 03 |........|
00000008opaque declares a byte without any interpretation. For example, opaque random_bytes[28] means 28 bytes.
Now let's take a look at the ClientHello fields.
The first field is client_version, which should be the highest version of the TLS protocol supported by the client. The values are the same as for the ProtocolVersion structure.
The random field is a 32-byte value. TLS 1.2 specifies that the first 4 bytes should contain the current Unix time, but notes that the clocks do not need to be set correctly. Later it was found that exposing the system time could be used to fingerprint the client, hence clients use random values instead. In TLS 1.3, the random field is defined as just 32 bytes of random data.
The session_id field is used to resume a previous session. I won't cover it in this post and will use an empty session ID to indicate that we want to establish a new session.
The cipher_suites field is a list of the cryptographic algorithms supported by the client, ordered by the client's preference. A cipher suite is a set of cryptographic algorithms used in the TLS protocol. The cipher suite values are assigned by IANA. The value for TLS_RSA_WITH_AES_128_CBC_SHA is {0x00, 0x2F}.
The compression_methods field is a list of compression methods supported by the client. We'll use the null method, which means no compression.
The extensions field is used to add additional features to the ClientHello message. We won't use any extensions.
Let's encode a ClientHello message with the following values:
ClientHello client_hello_msg = {
client_version: {3, 3}, /* TLS v1.2 */
random: {
gmt_unix_time: 0x00010203,
random_bytes: {0x04, 0x05, ..., 0x1f}
},
session_id: {
/* session_id length: 0 bytes */
},
cipher_suites: {
/* cipher_suites length: 2 bytes */
{0x00, 0x2F} /* TLS_RSA_WITH_AES_128_CBC_SHA */
},
compression_methods: {
/* compression_methods length: 1 byte */
null /* 0 */
}
};00000000 03 03 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d |................|
00000010 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d |................|
00000020 1e 1f 00 00 02 00 2f 01 00 |....../..|
00000029Now let's encapsulate the ClientHello message into a Handshake structure:
Handshake client_hello_handshake = {
msg_type: 0x01, /* client_hello(1) */
length: 0x29, /* length of client_hello_msg in bytes */
body: {0x03, 0x03, 0x00, 0x01, ..., 0x00} /* client_hello_msg */
};00000000 01 00 00 29 03 03 00 01 02 03 04 05 06 07 08 09 |...)............|
00000010 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 |................|
00000020 1a 1b 1c 1d 1e 1f 00 00 02 00 2f 01 00 |........../..|
0000002dFinally, let's encapsulate the Handshake message into a TLSPlaintext structure:
TLSPlaintext client_hello_record = {
type: 0x16, /* handshake(22) */
version: {3, 3}, /* TLS v1.2 */
length: 0x2d, /* length of client_hello_handshake in bytes */
fragment: {0x01, 0x00, 0x00, 0x29, ..., 0x00} /* client_hello_handshake */
};00000000 16 03 03 00 2d 01 00 00 29 03 03 00 01 02 03 04 |....-...).......|
00000010 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 |................|
00000020 15 16 17 18 19 1a 1b 1c 1d 1e 1f 00 00 02 00 2f |.............../|
00000030 01 00 |..|
00000032
This is what the client can send to the server. You can try it with nc:
printf '\x16\x03\x03\x00\x2d\x01\x00\x00\x29\x03\x03\x00\x01\x02\x03\x04\x05\x06'\
'\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a'\
'\x1b\x1c\x1d\x1e\x1f\x00\x00\x02\x00\x2f\x01\x00' | nc example.com 443 | hexdump -CIf the client supports older versions, it may send any {3, x} version in TLSPlaintext.version and the highest supported version in ClientHello.client_version, but we'll use {3, 3} everywhere to make it simple. RFC 8446 (TLS 1.3) goes into more detail about backwards compatibility.
ServerHello
The server responds to the ClientHello message with a ServerHello message, which has the following structure:
struct {
ProtocolVersion server_version;
Random random;
SessionID session_id;
CipherSuite cipher_suite;
CompressionMethod compression_method;
select (extensions_present) {
case false:
struct {};
case true:
Extension extensions<0..2^16-1>;
};
} ServerHello;Here is a real example of a ServerHello from example.com:
00000000 16 03 03 00 4a 02 00 00 46 03 03 fe c0 e1 70 4e |....J...F.....pN|
00000010 8f ea 8c 8d 71 5c 82 c0 1a 9a ab 87 66 72 9e 03 |....q\......fr..|
00000020 9a ef d1 44 4f 57 4e 47 52 44 01 20 2f 62 57 88 |...DOWNGRD. /bW.|
00000030 6e 7f 5b 15 80 6f 7b 3d 1d 5b 7b ae 6c 6d 21 2a |n.[..o{=.[{.lm!*|
00000040 e1 73 ca 77 5d 91 8c 2e 63 b8 d7 69 00 2f 00 |.s.w]...c..i./.|
TLSPlaintext server_hello_record = {
type: 0x16, /* handshake(2) */
version: {3, 3}, /* TLS v1.2 */
length: 0x4a, /* 74 */
fragment: {
msg_type: 0x02, /* server_hello(2) */
length: 0x46, /* 70 */
body: {
server_version: {3, 3}, /* TLS v1.2 */
random: {
gmt_unix_time: 0xfec0e170,
random_bytes: {0x4e, 0x8f, ..., 0x01}
},
session_id: {
/* session_id length: 0x20 bytes */
0x2f, 0x62, ..., 0x69
},
cipher_suite: {0x00, 0x2f}, /* TLS_RSA_WITH_AES_128_CBC_SHA */
compression_method: 0x00 /* null */
}
}
};As we had only one cipher suite in the ClientHello message and one compression method, and the server supports them, it responded with the same values.
Certificate
The next message from the server is the Certificate message, which contains the server's certificate. The structure is as follows:
opaque ASN.1Cert<0..2^24-1>;
struct {
ASN.1Cert certificate_list<0..2^24-1>;
} Certificate;The server's certificate must be the first in the list, followed by intermediate certificates.
Let's continue analyzing the example.com response:
00000000 16 03 03 0c 4b 0b 00 0c 47 00 0c 44 00 07 72 30 |....K...G..D..r0|
00000010 82 07 6e 30 82 06 56 a0 03 02 01 02 02 10 07 5b |..n0..V........[|
...
00000160 01 01 05 00 03 82 01 0f 00 30 82 01 0a 02 82 01 |.........0......|
00000170 01 00 86 85 0f bb 0e f9 ca 5f d9 f5 e0 0a 32 2c |........._....2,|
00000180 33 d9 aa 0e 07 29 a8 2f 08 ad 78 bd c2 06 bf f7 |3....)./..x.....|
...
00000250 48 e8 3f 4e 19 9a bf 9e 46 aa 32 93 ff a5 b2 5a |H.?N....F.2....Z|
00000260 b4 b1 2f 1e 69 84 92 1d b0 b9 8d af f2 31 6c 95 |../.i........1l.|
00000270 86 f3 02 03 01 00 01 a3 82 03 f2 30 82 03 ee 30 |...........0...0|
00000280 1f 06 03 55 1d 23 04 18 30 16 80 14 74 85 80 c0 |...U.#..0...t...|
...
00000770 b9 0b f6 b2 8b cc b5 55 33 66 ba 33 c2 c4 f0 a2 |.......U3f.3....|
00000780 e9 00 04 cc 30 82 04 c8 30 82 03 b0 a0 03 02 01 |....0...0.......|
00000790 02 02 10 0c f5 bd 06 2b 56 02 f4 7a b8 50 2c 23 |.......+V..z.P,#|
...
00000c30 59 92 23 0d 24 2a 95 25 4c ca a1 91 e6 d4 b7 ac |Y.#.$*.%L.......|
00000c40 87 74 b3 f1 6d a3 99 db f9 d5 bd 84 40 9f 07 98 |.t..m.......@...|
00000c50TLSPlaintext certificate_record = {
type: 0x16, /* handshake(22) */
version: {3, 3}, /* TLS v1.2 */
length: 0x0c4b, /* 3147 */
fragment: {
msg_type: 0x0b, /* certificate(11) */
length: 0x0c47, /* 3143 */
body: {
certificate_list: {
/* certificate_list length: 0x0c44 */
{
/* certificate length: 0x0772 */
0x30, 0x82, 0x07, 0x6e, 0x30, ..., 0x02, 0x82, 0x01,
0x01, 0x00, 0x86, 0x85, 0x0f, ..., 0x86, 0xf3, 0x02,
0x03, 0x01, 0x00, 0x01, 0xa3, ..., 0xa2, 0xe9
},
{
/* certificate length: 0x04cc */
0x30, 0x82, 0x04, 0xc8, 0x30, ..., 0x07, 0x98
}
}
}
}
}The certificates are encoded in the X.509 format. As we negotiated a cipher suite that uses RSA for key exchange, the server responded with a certificate containing its RSA public key:
modulus = 0x86850fbb0ef9ca5fd9f5e00a322c33d9aa0e0729a82f08ad78bdc206bff72d2ba6a7273d53a64cc34bb2277720d6c15449b808daf970a961f6b2499d6957dafb6d2434722e47f0043f9db15be2bc66315932e6a97ebfd4b0d464f56bca7bff725b5e9ad83fd406b2f3c8dc8f665a468466a8181579a708ce053cfb3989ef6dfa4e71527bb7e4a0a49c96c0613da40a704dc38ecd6eb3326cf2c7440904dda055fd23a52078b2855ed83bad17ff85c5b9748d33b9b8576eb5bc6965db0b3c925599f473b46424ca674c2899ccdc673d79c7169c2be6abaaaa357237f6812a48e83f4e199abf9e46aa3293ffa5b25ab4b12f1e6984921db0b98daff2316c9586f3 (256 bytes, 2048 bits)
publicExponent = 65537 (0x010001)
ServerHelloDone
To indicate that the server is done sending messages to support the key exchange, the server sends a ServerHelloDone message, which is an empty message:
struct { } ServerHelloDone;It's received as:
00000000 16 03 03 00 04 0e 00 00 00 |.........|
00000009TLSPlaintext server_hello_done_record = {
type: 0x16, /* handshake(22) */
version: {3, 3}, /* TLS v1.2 */
length: 4
fragment: {
msg_type: 0x0e, /* server_hello_done(14) */
length: 0
body: {}
}
}ClientKeyExchange
After receiving the server's RSA public key, the client generates a pre-master secret, encrypts it with the server's RSA public key, and sends it to the server. The pre-master secret is a 48-byte value that is used to generate the master secret, which is used to generate the session keys.
A pre-master secret has the following structure:
struct {
ProtocolVersion client_version;
opaque random[46];
} PreMasterSecret;So, let's create our pre-master secret:
PreMasterSecret pre_master_secret = {
client_version: {3, 3}, /* TLS v1.2 */
random: {
0x00, 0x01, 0x02, ..., 0x2c, 0x2d
}
}00000000 03 03 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d |................|
00000010 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d |................|
00000020 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d |.. !"#$%&'()*+,-|
00000030Next we need to pad the pre-master secret to the length of the RSA modulus. For simplicity, let's assume our random generator is broken and it always returns 0x55:
00000000 00 02 55 55 55 55 55 55 55 55 55 55 55 55 55 55 |..UUUUUUUUUUUUUU|
00000010 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 |UUUUUUUUUUUUUUUU|
*
000000c0 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 00 |UUUUUUUUUUUUUUU.|
000000d0 03 03 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d |................|
000000e0 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d |................|
000000f0 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d |.. !"#$%&'()*+,-|
00000100pre_master_secret_msg = 0x255555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555000303000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2dNow we encrypt the pre-master secret using the server's RSA public key:
pre_master_secret_encrypted = pow(pre_master_secret_msg, publicExponent) % modulus = 0x24969bfed1de98f2ad40f7a0d50c8fe2e8e534f5f3d91c25a6b23d0b01d9bb5de146f4ecf1e70451a3e9a663c9e2ef732646a70135cb9e2f4e65ad43273b117d66b6619278e820ec074d866ce0c1aed6ebeeb29e88ccff3a50c4cee42c909cc78e49231aa790bc34d9e4a8e78e0a8ac2aada0d7d6a6d57b9d6bbe4edc782c1f166564b2822cf3f66f845d664a9e228b06b6dfe047651322cb77512ecc7e9b0a8a73b9e2ea127e14f24cc6d23892a60189c665422fd89890d2c1bc9c6a9fc948582c461e36a82be3b78332e1272e2611d9b15a5d32e2b1e00a722008212c994ece5e25aa9676ca156af8de0b0e8041276168c1ee9535793c3b8c2f5670dbdcf27Finally, the client sends the encrypted pre-master secret to the server in a ClientKeyExchange message:
struct {
select (KeyExchangeAlgorithm) {
case rsa:
EncryptedPreMasterSecret;
case dhe_dss:
case dhe_rsa:
case dh_dss:
case dh_rsa:
case dh_anon:
ClientDiffieHellmanPublic;
} exchange_keys;
} ClientKeyExchange;
struct {
opaque encrypted_pre_master_secret<0..2^16-1>;
} EncryptedPreMasterSecret;
TLSPlaintext client_key_exchange_handshake_record = {
type: 0x16, /* handshake(22) */
version: {3, 3}, /* TLS v1.2 */
length: 0x0106, /* 262 */
fragment: {
msg_type: 0x10, /* client_key_exchange(16) */
length: 0x0102, /* 258 */
body: {
exchange_keys: {
rsa: {
encrypted_pre_master_secret: {
/* encrypted_pre_master_secret length: 0x0100 */
0x24, 0x96, 0x9b, ..., 0x27
}
}
}
}
}
}00000000 16 03 03 01 06 10 00 01 02 01 00 24 96 9b fe d1 |...........$....|
00000010 de 98 f2 ad 40 f7 a0 d5 0c 8f e2 e8 e5 34 f5 f3 |....@........4..
...
000000f0 6c a1 56 af 8d e0 b0 e8 04 12 76 16 8c 1e e9 53 |l.V.......v....S|
00000100 57 93 c3 b8 c2 f5 67 0d bd cf 27 |W.....g...'|
0000010bChangeCipherSpec
The next step is to calculate the master secret and the session keys, and then switch to ciphertext. To do that, we need to introduce a pseudo-random function (PRF):
A(0) = seed
A(i) = HMAC_<hash>(secret, A(i-1))
P_<hash>(secret, seed) = HMAC_<hash>(secret, A(1) + seed) +
HMAC_<hash>(secret, A(2) + seed) +
HMAC_<hash>(secret, A(3) + seed) + ...
PRF(secret, label, seed) = P_<hash>(secret, label + seed)In this definition, + denotes concatenation, and hash in our case is SHA-256. HMAC is defined by RFC 2104, HMAC_SHA256 for keys less than 512 bits (64 bytes) long looks like this:
REPEAT(b, 0) = {}
REPEAT(b, n) = {b} + REPEAT(b, n - 1)
PAD(key, n) = key + REPEAT(0x00, n - len(key))
K_IPAD(key) = PAD(key, 64) XOR REPEAT(0x36, 64)
K_OPAD(key) = PAD(key, 64) XOR REPEAT(0x5c, 64)
HMAC_SHA256(key, text) = SHA256(K_OPAD(key) +
SHA256(K_IPAD(key) +
text))For longer keys, the key is first hashed with SHA-256.
The master secret is always 48 bytes long and is calculated as follows:
master_secret = PRF(pre_master_secret,
"master secret",
ClientHello.random + ServerHello.random)[0..47]As P_SHA256 output is 32 bytes long, it's enough to calculate two iterations:
ms_seed = "master secret" + ClientHello.random + ServerHello.random
A(1) = HMAC_SHA256(pre_master_secret, ms_seed)
A(2) = HMAC_SHA256(pre_master_secret, A(1))
master_secret = (HMAC_SHA256(pre_master_secret, A(1) + ms_seed) +
HMAC_SHA256(pre_master_secret, A(2) + ms_seed))[0..47]Let's calculate the master secret:
pre_master_secret = {0x03, 0x03, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11,
0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b,
0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21, 0x22, 0x23, 0x24, 0x25,
0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d}
ms_seed = "master secret" + ClientHello.random + ServerHello.random
= "master secret" +
{0x00, 0x01, ..., 0x1f} +
{0xfe, 0xc0, ..., 0x01}
= {0x6d, 0x61, 0x73, 0x74, 0x65, 0x72, 0x20, 0x73, 0x65, 0x63, 0x72,
0x65, 0x74, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13,
0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e,
0x1f, 0xfe, 0xc0, 0xe1, 0x70, 0x4e, 0x8f, 0xea, 0x8c, 0x8d, 0x71,
0x5c, 0x82, 0xc0, 0x1a, 0x9a, 0xab, 0x87, 0x66, 0x72, 0x9e, 0x03,
0x9a, 0xef, 0xd1, 0x44, 0x4f, 0x57, 0x4e, 0x47, 0x52, 0x44, 0x01}
PAD(pre_master_secret, 64)
= {0x03, 0x03, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09,
0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15,
0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21,
0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00}
K_IPAD(pre_master_secret) = PAD(pre_master_secret, 64) XOR REPEAT(0x36, 64)
= {0x35, 0x35, 0x36, 0x37, 0x34, 0x35, 0x32, 0x33, 0x30, 0x31, 0x3e, 0x3f,
0x3c, 0x3d, 0x3a, 0x3b, 0x38, 0x39, 0x26, 0x27, 0x24, 0x25, 0x22, 0x23,
0x20, 0x21, 0x2e, 0x2f, 0x2c, 0x2d, 0x2a, 0x2b, 0x28, 0x29, 0x16, 0x17,
0x14, 0x15, 0x12, 0x13, 0x10, 0x11, 0x1e, 0x1f, 0x1c, 0x1d, 0x1a, 0x1b,
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
0x36, 0x36, 0x36, 0x36}
K_OPAD(pre_master_secret) = PAD(pre_master_secret, 64) XOR REPEAT(0x5c, 64)
= {0x5f, 0x5f, 0x5c, 0x5d, 0x5e, 0x5f, 0x58, 0x59, 0x5a, 0x5b, 0x54, 0x55,
0x56, 0x57, 0x50, 0x51, 0x52, 0x53, 0x4c, 0x4d, 0x4e, 0x4f, 0x48, 0x49,
0x4a, 0x4b, 0x44, 0x45, 0x46, 0x47, 0x40, 0x41, 0x42, 0x43, 0x7c, 0x7d,
0x7e, 0x7f, 0x78, 0x79, 0x7a, 0x7b, 0x74, 0x75, 0x76, 0x77, 0x70, 0x71,
0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c, 0x5c,
0x5c, 0x5c, 0x5c, 0x5c}
SHA256(K_IPAD(pre_master_secret) + ms_seed)
= SHA256({0x35, 0x35, ..., 0x36, 0x6d, 0x61, ..., 0x01})
= {0x52, 0x2a, 0x76, 0xda, 0x66, 0xfa, 0x54, 0x14, 0xfe, 0x86, 0xd9, 0xdb,
0x55, 0x71, 0x84, 0x42, 0x2a, 0x3a, 0x5b, 0x1d, 0x69, 0x18, 0x40, 0x83,
0xfa, 0x90, 0xb0, 0x4d, 0x33, 0xcd, 0x7a, 0x47}
A(1) = HMAC_SHA256(pre_master_secret, ms_seed)
= SHA256(K_OPAD(pre_master_secret) +
SHA256(K_IPAD(pre_master_secret) + ms_seed))
= SHA256({0x5f, 0x5f, ..., 0x5c, 0x52, 0x2a, ..., 0x47})
= {0x13, 0x96, 0x14, 0x48, 0x79, 0x5d, 0x37, 0xf7, 0x13, 0xae, 0x57, 0x70,
0x42, 0x8e, 0x0d, 0xe2, 0x40, 0xa5, 0xbc, 0x95, 0x93, 0xa4, 0xa4, 0x6e,
0x68, 0x37, 0xf6, 0x41, 0x9a, 0x42, 0xb7, 0xf4}
SHA256(K_IPAD(pre_master_secret) + A(1))
= {0x67, 0xb1, ..., 0x4b}
A(2) = HMAC_SHA256(pre_master_secret, A(1))
= SHA256(K_OPAD(pre_master_secret) +
SHA256(K_IPAD(pre_master_secret) + A(1)))
= SHA256({0x5f, 0x5f, ..., 0x5c, 0x67, 0xb1, ..., 0x4b})
= {0x87, 0xea, 0x5d, 0xaf, 0x25, 0x36, 0xdd, 0xdc, 0xa3, 0x38, 0x48, 0x85,
0xb7, 0x63, 0x0d, 0x29, 0xb3, 0x75, 0x30, 0xa1, 0x5d, 0x37, 0x15, 0x30,
0x56, 0xe3, 0xdc, 0x9c, 0x82, 0xfc, 0xf7, 0xf3}
HMAC_SHA256(pre_master_secret, A(1) + ms_seed)
= {0x1b, 0xf6, 0x98, 0x50, 0xb4, 0x22, 0x77, 0x34, 0x74, 0x17, 0xa9, 0xda,
0xd8, 0x6e, 0xbf, 0xe1, 0xbe, 0x90, 0x66, 0xad, 0xf4, 0x37, 0xe9, 0x0a,
0x67, 0xe2, 0x92, 0x01, 0xdf, 0x08, 0xb4, 0x35}
HMAC_SHA256(pre_master_secret, A(2) + ms_seed)
= {0xb9, 0x21, 0xed, 0x7d, 0x7a, 0x45, 0x7d, 0x35, 0x1c, 0x3c, 0x06, 0xfa,
0x18, 0x1c, 0x37, 0x6d, 0x7d, 0x25, 0x0b, 0x13, 0x8a, 0xc2, 0x28, 0xf7,
0x87, 0xa0, 0x00, 0x0a, 0x55, 0xae, 0x2e, 0xca}
master_secret = (HMAC_SHA256(pre_master_secret, A(1) + ms_seed) +
HMAC_SHA256(pre_master_secret, A(2) + ms_seed))[0..47]
= {0x1b, 0xf6, 0x98, 0x50, 0xb4, 0x22, 0x77, 0x34, 0x74, 0x17, 0xa9, 0xda,
0xd8, 0x6e, 0xbf, 0xe1, 0xbe, 0x90, 0x66, 0xad, 0xf4, 0x37, 0xe9, 0x0a,
0x67, 0xe2, 0x92, 0x01, 0xdf, 0x08, 0xb4, 0x35, 0xb9, 0x21, 0xed, 0x7d,
0x7a, 0x45, 0x7d, 0x35, 0x1c, 0x3c, 0x06, 0xfa, 0x18, 0x1c, 0x37, 0x6d}
Next, we need to calculate the session keys. The session keys are calculated by creating a key block:
key_block = PRF(master_secret,
"key expansion",
ServerHello.random + ClientHello.random);And partitioning it into the several components:
client_write_MAC_key[mac_key_length]
server_write_MAC_key[mac_key_length]
client_write_key[enc_key_length]
server_write_key[enc_key_length]
client_write_IV[fixed_iv_length]
server_write_IV[fixed_iv_length]RFC 5246 gives the following table for the cipher suites:
| Cipher | Type | Key Material | IV Size | Block Size |
|---|---|---|---|---|
| NULL | Stream | 0 | 0 | N/A |
| RC4_128 | Stream | 16 | 0 | N/A |
| 3DES_EDE_CBC | Block | 24 | 8 | 8 |
| AES_128_CBC | Block | 16 | 16 | 16 |
| AES_256_CBC | Block | 32 | 16 | 16 |
And for the MACs:
| MAC | Algorithm | MAC Length | MAC Key Length |
|---|---|---|---|
| NULL | N/A | 0 | 0 |
| MD5 | HMAC-MD5 | 16 | 16 |
| SHA | HMAC-SHA1 | 20 | 20 |
| SHA256 | HMAC-SHA256 | 32 | 32 |
In our case, TLS_RSA_WITH_AES_128_CBC_SHA, the MAC key length is 20 bytes, the encryption key length is 16 bytes, and the fixed IV length is 16 bytes. We won't need these IVs, so we need the first 20+20+16+16 = 72 bytes of the key block, i.e. we need 3 iterations of the PRF:
ke_seed = "key expansion" + ServerHello.random + ClientHello.random
= "key expansion" +
{0xfe, 0xc0, ..., 0x01} +
{0x00, 0x01, ..., 0x1f}
= {0x6b, 0x65, 0x79, 0x20, 0x65, 0x78, 0x70, 0x61, 0x6e, 0x73, 0x69,
0x6f, 0x6e, 0xfe, 0xc0, 0xe1, 0x70, 0x4e, 0x8f, 0xea, 0x8c, 0x8d,
0x71, 0x5c, 0x82, 0xc0, 0x1a, 0x9a, 0xab, 0x87, 0x66, 0x72, 0x9e,
0x03, 0x9a, 0xef, 0xd1, 0x44, 0x4f, 0x57, 0x4e, 0x47, 0x52, 0x44,
0x01, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09,
0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14,
0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f}
A(1) = HMAC_SHA256(master_secret, ke_seed)
= {0xeb, 0x70, 0xe4, 0xcb, 0x7f, 0x05, 0x83, 0x90, 0xf6, 0xd5, 0x2f, 0xaa,
0x92, 0xdd, 0x69, 0xaa, 0xe8, 0x03, 0xff, 0x3b, 0x7a, 0xca, 0x28, 0xd9,
0xb1, 0x08, 0xe7, 0x1a, 0x70, 0x4c, 0xbe, 0xfe}
A(2) = HMAC_SHA256(master_secret, A(1))
= {0x74, 0x58, 0x65, 0xc3, 0x3b, 0x8d, 0xe5, 0x70, 0x5b, 0x0f, 0x6a, 0x82,
0x0d, 0x1d, 0x05, 0x8e, 0x6e, 0xe9, 0xdc, 0xd7, 0xc7, 0x20, 0xe8, 0xef,
0x51, 0xb1, 0x56, 0xc6, 0x8d, 0x72, 0x0f, 0x47}
A(3) = HMAC_SHA256(master_secret, A(2))
= {0xe4, 0x6f, 0x28, 0x13, 0x9a, 0xbf, 0xd0, 0xca, 0x3b, 0xd1, 0x8e, 0xa6,
0xa1, 0x1d, 0x68, 0xe2, 0xec, 0x90, 0x6e, 0x74, 0xee, 0x2b, 0x27, 0x39,
0x79, 0x19, 0xce, 0xc5, 0xab, 0xc1, 0x7b, 0x2d}
key_block = HMAC_SHA256(master_secret, A(1) + ke_seed) +
HMAC_SHA256(master_secret, A(2) + ke_seed) +
HMAC_SHA256(master_secret, A(3) + ke_seed)
= {0x4d, 0x36, 0x7f, 0x54, 0xa7, 0xd6, 0x9d, 0x9b, 0xc4, 0xa1, 0xf0, 0xf8,
0x53, 0x0c, 0x63, 0x86, 0x34, 0x5f, 0x16, 0x02, 0x3b, 0x43, 0xdf, 0x4f,
0xe0, 0x55, 0x77, 0x6f, 0xdb, 0xbb, 0x39, 0x1d}
+ {0x22, 0x1b, 0x24, 0x61, 0x8e, 0x71, 0x84, 0x36, 0x96, 0x49, 0x56, 0xd1,
0x98, 0xb6, 0x8c, 0xef, 0xa7, 0xf9, 0x32, 0x7d, 0x09, 0x92, 0xcd, 0x14,
0x96, 0x18, 0x7c, 0xe6, 0x7f, 0xd1, 0x6a, 0xe0}
+ {0xa3, 0xb8, 0xec, 0x27, 0xd2, 0x73, 0xcc, 0x35, 0xe1, 0x04, 0x92, 0x20,
0x2c, 0x7c, 0x67, 0xba, 0x42, 0xcc, 0x20, 0xa1, 0x64, 0x1d, 0xa9, 0xb2,
0x71, 0x42, 0xbf, 0x74, 0xed, 0xfd, 0x2a, 0x60}From the key block, we can extract the session keys:
client_write_MAC_key = {0x4d, 0x36, 0x7f, 0x54, 0xa7, 0xd6, 0x9d, 0x9b, 0xc4, 0xa1,
0xf0, 0xf8, 0x53, 0x0c, 0x63, 0x86, 0x34, 0x5f, 0x16, 0x02}
server_write_MAC_key = {0x3b, 0x43, 0xdf, 0x4f, 0xe0, 0x55, 0x77, 0x6f, 0xdb, 0xbb,
0x39, 0x1d, 0x22, 0x1b, 0x24, 0x61, 0x8e, 0x71, 0x84, 0x36}
client_write_key = {0x96, 0x49, 0x56, 0xd1, 0x98, 0xb6, 0x8c, 0xef, 0xa7, 0xf9,
0x32, 0x7d, 0x09, 0x92, 0xcd, 0x14}
server_write_key = {0x96, 0x18, 0x7c, 0xe6, 0x7f, 0xd1, 0x6a, 0xe0, 0xa3, 0xb8,
0xec, 0x27, 0xd2, 0x73, 0xcc, 0x35}We are ready to switch to ciphertext. To indicate that, we send a ChangeCipherSpec message, which is defined as follows:
struct {
enum { change_cipher_spec(1), (255) } type;
} ChangeCipherSpec;It is not a handshake message, so it is encapsulated directly into a TLSPlaintext record:
TLSPlaintext change_cipher_spec_record = {
type: 0x14, /* change_cipher_spec(20) */
version: {3, 3}, /* TLS v1.2 */
length: 1,
fragment: {
0x01 /* change_cipher_spec(1) */
}
}00000000 14 03 03 00 01 01 |......|
00000006Starting from this record, the client and the server will encrypt and decrypt the data using the session keys.
Advanced Encryption Standard (AES)
We won't go into the details of the AES encryption algorithm, but there are a few key concepts to understand.
AES is a block cipher, which means that it encrypts data in blocks. The block size is 128 bits (16 bytes). It takes 16 bytes of plaintext as input and outputs 16 bytes of ciphertext. Since the plaintext may not be a multiple of the block size, we need to pad it.
If we simply split the plaintext into blocks and encrypt them separately, we would have a problem: the same blocks would encrypt to the same ciphertext, which could reveal patterns in the plaintext. To avoid this, we need to use a different mode of operation. In our case, we will use the Cipher Block Chaining (CBC) mode.
In CBC mode, the plaintext is XORed with the previous ciphertext block before encryption. The first block is XORed with the Initialization Vector (IV). The IV is a random value that is sent in the clear at the beginning of the communication and should be unpredictable and unique for each message.
To encrypt a message, we need a padded message that is a multiple of 16 bytes, an IV (16 random bytes), and a key (for AES-128, this is 128 bits or 16 bytes of secret data). The ciphertext will be the same size as the padded message.
To decrypt the message, we need the ciphertext, the IV, and the key.
The TLSCiphertext uses the following structure for the encrypted data:
struct {
opaque IV[record_iv_length];
block-ciphered struct {
opaque content[length];
opaque MAC[mac_length];
uint8 padding[padding_length];
uint8 padding_length;
};
} GenericBlockCipher;Let's say our content is "Hello, world!" (13 bytes), and the MAC is 20 bytes (represented as XX in this example). The padding_length takes 1 byte. The total length is 13 + 20 + 1 = 34 bytes. We need to pad it with 14 bytes to make it a multiple of 16 bytes. The padding should use the same value as padding_length. Therefore, our plaintext block will look like this:
00000000 48 65 6c 6c 6f 2c 20 77 6f 72 6c 64 21 XX XX XX |Hello, world!...|
00000010 XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX |................|
00000020 XX 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e 0e |................|
00000030To make a TLSCiphertext record, we need to encrypt this block with the session key and prepend it with the IV.
Finished
The next message that the client sends is the Finished message. It is the first message that is encrypted with the session keys. The Finished message is defined as follows:
struct {
opaque verify_data[verify_data_length];
} Finished;The client calculates the verify_data field as follows:
verify_data = PRF(master_secret,
"client finished",
hash(handshake_messages))[0..verify_data_length-1];where the hash is the hash function used as the basis for the PRF, which is SHA-256 in our case. The verify_data_length depends on the cipher suite. In our case, it is 12 bytes.
The handshake_messages are all the messages sent or received during the handshake, up to but not including this message:
handshake_messages = {0x01, 0x00, 0x00, 0x29, ..., 0x00} /* ClientHello */
+ {0x02, 0x00, 0x00, 0x46, ..., 0x00} /* ServerHello */
+ {0x0b, 0x00, 0x0c, 0x47, ..., 0x98} /* Certificate */
+ {0x0e, 0x00, 0x00, 0x00} /* ServerHelloDone */
+ {0x10, 0x00, 0x01, 0x02, ..., 0x27} /* ClientKeyExchange */
SHA256(handshake_messages) = {0x2c, 0xe5, 0x34, 0xf7, 0x2a, 0x97, 0xa6, 0x7e, 0x17,
0x36, 0x4e, 0x8c, 0x9f, 0xc5, 0x23, 0x7e, 0x58, 0xd9,
0xad, 0x3c, 0xfc, 0x0e, 0xef, 0xa5, 0x19, 0x75, 0xc3,
0xf0, 0x71, 0x99, 0x30, 0x73}The verify_data is calculated as follows:
verify_data = PRF({0x1b, 0xf6, 0x98, ..., 0x6d},
"client finished",
{0x2c, 0xe5, 0x34, ..., 0x73})[0..11]
= {0x2e, 0x5a, 0x3c, 0xd1, 0x40, 0x99, 0x6b, 0xd6, 0x7a, 0x70, 0x3d, 0x4e}Then we need to create our Finished message:
Handshake finished_handshake = {
msg_type: 0x14, /* Finished */
length: 12,
body: {
verify_data: {0x2e, 0x5a, 0x3c, 0xd1, 0x40, 0x99, 0x6b, 0xd6, 0x7a, 0x70,
0x3d, 0x4e}
}
}00000000 14 00 00 0c 2e 5a 3c d1 40 99 6b d6 7a 70 3d 4e |.....Z<.@.k.zp=N|
00000010This would be content in a TLSCiphertext record. Now we need to add the MAC and the padding.
The MAC is calculated as follows:
MAC(MAC_write_key, seq_num +
TLSPlaintext.type +
TLSPlaintext.version +
TLSPlaintext.length +
TLSPlaintext.fragment);(It is actually TLSCompressed, but compression is out of scope for this article.)
MAC is the MAC algorithm defined in the cipher suite, which is HMAC-SHA1 in our case. The seq_num is the sequence number of the encrypted message, with the type uint64.
This is our first encrypted message, so the sequence number is 0. Type is handshake(22), version is {3, 3} (TLS v1.2), and the fragment is 16 bytes long.
seq_num_0 = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}
finished_handshake_mac = HMAC_SHA1(client_write_MAC_key,
seq_num_0 + {0x16} + {3, 3} + {0x00, 0x10}
+ {0x14, 0x00, 0x00, 0x0c, ..., 0x4e})
= HMAC_SHA1({0x4d, 0x36, 0x7f, 0x54, ..., 0x02},
{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x16, 0x03, 0x03, 0x00, 0x10, 0x14, 0x00, 0x00,
0x0c, 0x2e, 0x5a, 0x3c, 0xd1, 0x40, 0x99, 0x6b,
0xd6, 0x7a, 0x70, 0x3d, 0x4e})
= {0xc9, 0x19, 0x59, 0x37, 0x47, 0x9e, 0x02, 0x4b, 0x5f,
0x8a, 0x7e, 0x3c, 0x86, 0x8d, 0xcc, 0xe3, 0x69, 0x8a,
0xf7, 0x7e}The fragment is 16 bytes long, the MAC is 20 bytes long, and an extra byte for padding length. This makes the total length of the message 16 + 20 + 1 = 37 bytes. We need to pad it with 11 bytes to make it a multiple of 16 bytes. It should be padded with the same value as the padding length. Therefore, our plaintext block will look like this:
finished_handshake_block_plaintext = {
content: {0x14, 0x00, ..., 0x4e}, /* finished_handshake */
MAC: {0xc9, 0x19, ..., 0x7e}, /* finished_handshake_mac */
padding: REPEAT(11, 11),
padding_length: 0x0b /* 11 */
}00000000 14 00 00 0c 2e 5a 3c d1 40 99 6b d6 7a 70 3d 4e |.....Z<.@.k.zp=N|
00000010 c9 19 59 37 47 9e 02 4b 5f 8a 7e 3c 86 8d cc e3 |..Y7G..K_.~<....|
00000020 69 8a f7 7e 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b |i..~............|
00000030Now we need to generate a random IV:
finished_handshake_IV = {0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf}And encrypt the block:
finished_handshake_block_ciphertext
= AES_128_CBC_encrypt(client_write_key,
finished_handshake_IV,
finished_handshake_block_plaintext)
= AES_128_CBC_encrypt({0x96, 0x49, 0x56, 0xd1, 0x98, 0xb6, 0x8c, 0xef,
0xa7, 0xf9, 0x32, 0x7d, 0x09, 0x92, 0xcd, 0x14},
{0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf},
{0x14, 0x00, 0x00, 0x0c, 0x2e, 0x5a, 0x3c, 0xd1,
0x40, 0x99, 0x6b, 0xd6, 0x7a, 0x70, 0x3d, 0x4e,
0xc9, 0x19, 0x59, 0x37, 0x47, 0x9e, 0x02, 0x4b,
0x5f, 0x8a, 0x7e, 0x3c, 0x86, 0x8d, 0xcc, 0xe3,
0x69, 0x8a, 0xf7, 0x7e, 0x0b, 0x0b, 0x0b, 0x0b,
0x0b, 0x0b, 0x0b, 0x0b, 0x0b, 0x0b, 0x0b, 0x0b})
= {0x31, 0xa0, 0x50, 0xc2, 0x6d, 0x0d, 0x0a, 0xdd,
0x55, 0x57, 0x1d, 0xfa, 0xda, 0x18, 0x90, 0xaa,
0x31, 0x16, 0xcf, 0x5f, 0x4b, 0xad, 0x65, 0x9e,
0xbc, 0x4f, 0x8d, 0x55, 0x87, 0xaf, 0xd7, 0x05,
0x41, 0x97, 0x66, 0x41, 0x90, 0x40, 0x40, 0x72,
0x9a, 0x3c, 0x90, 0x7c, 0xa8, 0xd1, 0xb2, 0x72}Now we can create a TLSCiphertext record, which will be sent to the server:
TLSCiphertext finished_handshake_record = {
type: 0x16, /* handshake(22) */
version: {3, 3}, /* TLS v1.2 */
length: 0x40, /* 64 */
fragment: {
IV: {0xa0, 0xa1, ..., 0xaf},
block-ciphered: {0x31, 0xa0, ..., 0x72}
}
}00000000 16 03 03 00 40 a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa |....@...........|
00000010 ab ac ad ae af 31 a0 50 c2 6d 0d 0a dd 55 57 1d |.....1.P.m...UW.|
00000020 fa da 18 90 aa 31 16 cf 5f 4b ad 65 9e bc 4f 8d |.....1.._K.e..O.|
00000030 55 87 af d7 05 41 97 66 41 90 40 40 72 9a 3c 90 |U....A.fA.@@r.<.|
00000040 7c a8 d1 b2 72 ||...r|
00000045And that's it! We have created our first encrypted message.
ChangeCipherSpec and Finished on the server side
The server should decrypt the Finished message and verify the verify_data field. If it is correct, the server should also send a ChangeCipherSpec message and its own Finished message.
The server calculates the verify_data field almost the same way as the client:
verify_data = PRF(master_secret,
"server finished",
hash(handshake_messages))[0..verify_data_length-1];Note, that the server will have client's Finished message in the handshake_messages as well.
Once client has the Finished message from the server, and verified that the verify_data is correct, the handshake is complete!
Application Data
Once the handshake is complete, the client and the server can start sending application data. The application data is encrypted the same way as the Finished message.
http_request = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"
http_request_len = {0x00, 0x25} /* 37 */
seq_num_1 = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}
http_request_mac = HMAC_SHA1(client_write_MAC_key,
seq_num_1 + {0x17} + {3, 3} +
+ http_request_len + http_request)
TLSCiphertext application_record = {
type: 0x17, /* application_data(23) */
version: {3, 3}, /* TLS v1.2 */
length: 0x50, /* 80 */
fragment: {
IV: random_IV,
block-ciphered: {
content: http_request,
MAC: http_request_mac,
padding: {0x06, 0x06, 0x06, 0x06, 0x06, 0x06},
padding_length: 6
}
}
}Python implementation
Do you prefer to read code? You can find a Python implementation of everything described in this article in this gist.