Internet Engineering

02

HTTP

Introduction · Cookie · Proxy & Cache · Authentication

Fall 2026 · Amirkabir University of Technology
@1995parham

Introduction

  • HTTP is the transfer protocol for web applications
    • H T T P: Hyper Text Transfer Protocol
    • HTTP/1.0 (RFC 1945), HTTP/1.1 (RFC 9112), HTTP/2 (RFC 9113), HTTP/3 (RFC 9114)
    • Semantics and caching are version-independent: RFC 9110 (Semantics), RFC 9111 (Caching)
    • In fact, it can be used to transfer everything (not only hyper text)
      • Text Documents e.g. HTML, XML, JSON, etc.
      • Multimedia e.g. JPG, GIF, MP4, MKV, etc.
      • Application e.g. PDF, ZIP, etc.

Introduction (Cont.)

  • HTTP uses the client/server paradigm
    • HTTP server provides resource
    • HTTP client (usually web browser) gets resource
  • But not pure client/server communication
    • Proxies
    • Caches
    • ...

Introduction (Cont.)

  • HTTP is an application layer protocol
  • HTTP assumes reliable communication
    • over TCP
    • default (server) port: 80
    • client port is chosen randomly per connection
  • HTTP is Stateless
    • Server does not keep history/state of client
    • High performance & Low Complexity
    • Problematic in some applications (sessions)
      • Cookies
      • JSON Web Tokens

Data Resources to Transfer

  • HTTP is the protocol to transfer data between server and client (usually from server to client)
  • Which data?
    • It can be anything
    • In web, usually, it is a resource/object on server
  • Each resource must be identified/located uniquely
    • URI (Uniform Resource Identifier)
    • URL (Uniform Resource Locator)
  • URIs identify and URLs locate; however, locators are also identifiers, so
  • every URL is also a URI, but there are URIs which are not URLs.

URI in Action

  • A Uniform Resource Name (URN) is a URI that identifies a resource by name in a particular namespace.
  • International Standard Book Number (ISBN) system

URL

<protocol(scheme)> :// <user> : <pass> @ <host> : < port> / <path> ? <query> # <frag>

  • https://aut.ac.ir/page/1/AUT
  • https://www.bing.com/search?q=Hello+World&form=QBLH&sp=-1&pq=&sc=0-0&qs=n&sk=&cvid=7F2B496642F94D5F95989756B4FF60EE
  • file:///home/parham/Documents/Git/parham/dotfiles
  • https://archive.org/details/texts
  • ftp://speedtest.tele2.net

URL (Cont.)

  • Scheme: The application layer protocol
  • HTTP: The web protocol
  • HTTPS: Secure HTTP
  • FTP: File Transfer Protocol
  • File: Access to a local file
  • javascript: Run javascript code
  • mailto: Send mail to given address
  • etc.

URL (Cont.)

  • Path: The path of the object on host filesystem
  • E.g. web server root directory is /var/www/
    • http://www.example.com/1.html/var/www/1.html
    • http://www.example.com/1/2/3.jpg/var/www/1/2/3.jpg
    • http://www.example.com/1/2/../3.jpg/var/www/1/3.jpg 🤨

URL (Cont.)

  • Query: A mechanism to pass information from client to active pages or forms
    • Fill information in a university registration form
    • Ask Bing! to search a phrase
  • Starts with "?"
  • name=value format
  • "&" is the border between multiple parameters
  • Try this out, by clicking me

HTTPBin

A simple HTTP Request & Response Service.

URL (Cont.)

URL (Cont.)

  • Domain names are case insensitive according to RFC 4343.
  • The rest of URL is sent to the server via the GET method and etc. This may be case sensitive or not.

URL (Cont.)

  • URL is encoded by client before transmission
  • How: Each byte is divided into two 4-bit group, hexadecimal of the 4-bits are prefixed by %
    • ~126%7E
  • What & Why?
    • Non-ASCII (e.g., Persian Characters, Emoji)
    • Reserved character when are not used for special role
    • Unsafe character, e.g. space, %, ...

    https://ganj.irandoc.ac.ir/api/v1/search/main?keywords=hellow%20world

URL in Action

  • User asks the browser to retrieve a resource
  • Browser finds the ip address of <host> (DNS lookup)
  • Browser creates a TCP connection to the IP address and the <port>
  • Browser sends http requests through the connection
  • Browser gets the response and processes it

URL in Action (Cont.)


# ce.aut.ac.ir resolves to 185.211.88.129
# connectify0 is the network interface

sudo tcpdump --interface connectify0 --number -n -v 'port 80 and dst host 185.211.88.129'
  

1  07:47:41.159469 IP (tos 0x0, ttl 64, id 59255, offset 0, flags [DF], proto TCP (6), length 60)
10.202.0.2.55340 > 185.211.88.129.80: Flags [S], cksum 0x5a35 (correct), seq 440330126, win 59040, options [mss 14760,sackOK,TS val 954835682 ecr 0,nop,wscale 7], length 0

2  07:47:41.159904 IP (tos 0x0, ttl 64, id 59256, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xf7e0 (correct), ack 35014, win 462, options [nop,nop,TS val 954835682 ecr 15081515], length 0

3  07:47:41.160032 IP (tos 0x0, ttl 64, id 59257, offset 0, flags [DF], proto TCP (6), length 137)
10.202.0.2.55340 > 185.211.88.129.80: Flags [P.], cksum 0x282f (correct), seq 0:85, ack 1, win 462, options [nop,nop,TS val 954835682 ecr 15081515], length 85: HTTP, length: 85
    GET /~students/ HTTP/1.1
    Host: ce.aut.ac.ir
    User-Agent: curl/8.3.0
    Accept: */*

4  07:47:41.218732 IP (tos 0x0, ttl 64, id 59258, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xe649 (correct), ack 4334, win 429, options [nop,nop,TS val 954835741 ecr 15081574], length 0

5  07:47:41.219358 IP (tos 0x0, ttl 64, id 59259, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xd5c5 (correct), ack 8592, win 397, options [nop,nop,TS val 954835742 ecr 15081575], length 0

6  07:47:41.224808 IP (tos 0x0, ttl 64, id 59260, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xcb33 (correct), ack 11288, win 397, options [nop,nop,TS val 954835747 ecr 15081580], length 0

7  07:47:41.226004 IP (tos 0x0, ttl 64, id 59261, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xc0a9 (correct), ack 13984, win 397, options [nop,nop,TS val 954835748 ecr 15081581], length 0

8  07:47:41.226548 IP (tos 0x0, ttl 64, id 59262, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xb61f (correct), ack 16680, win 397, options [nop,nop,TS val 954835749 ecr 15081582], length 0

9  07:47:41.227124 IP (tos 0x0, ttl 64, id 59263, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xb332 (correct), ack 17427, win 397, options [nop,nop,TS val 954835750 ecr 15081583], length 0

10  07:47:41.227371 IP (tos 0x0, ttl 64, id 59264, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [F.], cksum 0xb331 (correct), seq 85, ack 17427, win 397, options [nop,nop,TS val 954835750 ecr 15081583], length 0

11  07:47:41.227485 IP (tos 0x0, ttl 64, id 59265, offset 0, flags [DF], proto TCP (6), length 52)
10.202.0.2.55340 > 185.211.88.129.80: Flags [.], cksum 0xb330 (correct), ack 17428, win 397, options [nop,nop,TS val 954835750 ecr 15081583], length 0
  

How does HTTP work? Transactions

  • HTTP data transfer is a collection of transactions
  • Each transaction is composed of 2 HTTP messages
  • Requests are identified by methods
    • Method: The action that client asks from server
  • Responses are identified by status codes
    • Status: The result of the requested action

HTTP Transactions

http-transactions

HTTP Transactions in Web

  • (Typically) each web page contains multiple resources
    • The main skeleton HTML page
    • Some linked materials: figures, videos, JS, CSS, etc.
  • Displaying a web page by a browser
    • Get the HTML page (first transaction)
    • Try to display the page (rendering)
    • Other resources are linked to the page
    • Get the resources (subsequent transactions)

HTTP Transactions in Web (Cont.)

  • HTTP Transactions & TCP Connections
    1. Non-persistent
      • A new TCP connection per object
      • Network overhead + Connection establish delay + Resource intensive
      • Parallel connections speed up browsing
    2. Persistent
      • Get multiple objects using a single TCP connection
      • No extra processing & networking overhead
      • Poor performance if implemented in serial manner
      • Pipeline requests speed up browsing (HTTP/1.1)

HTTP/2 for a Faster Web

HTTP Pipelining

  • HTTP Pipelining is not difficult to deploy, it is impossible.
  • it still allowed a single large or slow response to block all others that followed.

HTTP/2 Multiplexing

  • Multiplexing allows multiple request-response messages to be in flight over a single HTTP/2 connection, at the same time.

HTTP/2 for a Faster Web (Cont.)

http2-transactions

HTTP Transactions in Web: Hands on

  • Get a HTML page from a server
  • Capture the packets
  • Investigate the transactions

HTTP Messages

  • HTTP is text-based protocol
    • Human readable headers
    • The header is composed of some lines
message-structure

HTTP Messages (Cont.)

  • E.g. HTTP request message

GET /index.html HTTP/1.1
Host: www.aut.ac.ir
User-Agent: Mozilla/36.0
Accept-Language: en-us
Connection: keep-alive
    

Method<sp>Path<sp>Version<CRLF>
Header-Field:Header-Value<CRLF>
...
Header-Field:Header-Value<CRLF>
<CRLF>
Entity-Body
    
  • E.g. HTTP response message

HTTP/1.1 200 OK
Date: Sun, 02 Oct 2018 20:30:40
Server: Apache/2.2.2
Last-Modified: Mon, 03 May 2017 10:20:22
Connection: keep-alive
Content-Length: 3000

data data data ...
    

Version<sp>Code<sp>Reason<CRLF>
Header-Field:Header-Value<CRLF>
...
Header-Field:Header-Value<CRLF>
<CRLF>
Entity-Body
    

HTTP Methods

  • Methods are actions that client asks from server to do on the specified resource (given by the path parameter)
  • Which actions?
    • Basic data communication operations
      • Read a resource from the server
      • Send data to the server
      • Create, replace or delete a resource on the server
    • Debugging and troubleshooting
      • Get information about a resource
      • Check what the server has got from a client
      • Get the list of operations which can be applied on a resource

Safe & Idempotent [RFC 9110]

  • Safe: the request does not change anything on the server. The client is only asking a question
    • GET, HEAD, OPTIONS, TRACE
    • POST is not safe, even though it only sends data
  • Idempotent: doing it twice leaves the server in the same state as doing it once
    • the safe methods, plus PUT and DELETE
    • POST and PATCH are not

Why It Matters

  • A client, proxy or cache may retry an idempotent request after a timeout. It cannot safely retry a POST — that is why you see do not press refresh on payment pages
  • Only safe responses may be prefetched. A GET that deletes something will be deleted by a crawler
  • Every safe method is idempotent; the reverse is not true

HTTP Methods (Cont.)

  • GET: Retrieve resource from server
  • HEAD: Similar to GET but the resource itself is not retrieved, just the HTTP response header
    • Useful for debugging or some other applications
  • POST: Submit data to be processed by the specified resource
    • Data itself is enveloped in message body

HTTP Methods (Cont.)

  • DELETE: Remove the resource
    • Not popular in web, can be used in other applications
  • PUT: Add message body as the specified resource to server
  • PATCH: A PATCH request is considered a set of instructions on how to modify a resource. Contrast this with PUT; which is a complete representation of a resource.
  • TRACE: Server echoes back the received message
    • For troubleshooting & debugging
  • OPTIONS: Request the list of supported methods by server on the resource

HTTP Responses

  • The message for the result/response of the requested action
  • Which responses?
    • Basic responses
      • Success
      • Failure
        • Bad client request
        • Server problem
        • ...
    • Others
      • E.g., Redirection to other resources

HTTP Responses (Cont.)

  • 2xx (Successful responses)
    • 200: OK
    • 201: Created
    • 204: No Content
  • 4xx (Client errors)
    • 400: Bad Request
    • 401: Unauthorized (Authorization required)
    • 403: Forbidden
    • 404: Not Found
    • 405: Method Not Allowed

HTTP Responses (Cont.)

  • 5xx (Server errors)
    • 500: Internal Server Error
    • 501: Not Implemented
    • 503: Service Unavailable
  • 3xx (Redirects)
    • 301: Moved Permanently
    • 302: Found
      • redirect status response code indicates that the resource requested has been temporarily moved to the URL given by the Location header
    • 303: See Other
    • 307: Temporary Redirect
      • Resource has been moved, Redirection
      • Location header contains the new location of resource
      • Unlike 302, the client must not change the method
    • 304: Not Modified
    • 308: Permanent Redirect
  • 302, 303 and 307 are all temporary; they differ in what the client does with the method

Redirects: Where, and How?

A redirect says look over there. The harder question is what the client sends when it gets there — the same method and body, or a fresh GET?

CodeLifetimeMethod on the second request
301 Moved Permanentlypermanentin practice becomes GET
302 Foundtemporaryin practice becomes GET
303 See Othertemporaryalways GET, by specification
307 Temporary Redirecttemporarypreserved, body and all
308 Permanent Redirectpermanentpreserved, body and all
  • Note the wording for 301 and 302: in practice. That is the whole story

Why Would the Method Change at All?

  • The original specifications said a client must not change the method after a 301 or 302 without asking the user
  • Early browsers turned POST into GET anyway, silently — and the web was written against the browsers, not the document
  • By RFC 7231 the specification gave up and described reality: for historical reasons, a user agent MAY change the request method
  • So two unambiguous codes were added on either side: 303 means definitely switch to GET, 307 and 308 mean definitely do not

The Shape of It

301 and 302 are the ambiguous ones. After a POST, never use them — say what you mean with 303 or 307.

the 3xx family, grouped by whether the method is preserved

The Good Reason: POST / Redirect / GET

  • After a plain POST, the address bar still holds the URL you posted to
  • Press refresh and the browser offers to send the form again — one order becomes two
  • Answer: do not answer a POST. Redirect it

POST /orders HTTP/1.1
Host: shop.example
Content-Type: application/x-www-form-urlencoded

item=book&qty=1
    

HTTP/1.1 303 See Other
Location: /orders/1042
    

GET /orders/1042 HTTP/1.1
Host: shop.example
    
  • The browser now sits on the receipt. Refresh re-fetches it, and the order is placed exactly once
  • Turning the POST into a GET is not a wart here — it is the point

When You Must Keep the Method

  • An API endpoint moves. The client should re-send the same request, not a GET at the new address

POST /api/v1/orders HTTP/1.1
Host: api.example
Content-Type: application/json

{"item": "book"}
    

HTTP/1.1 308 Permanent Redirect
Location: /api/v2/orders
    

POST /api/v2/orders HTTP/1.1
Host: api.example
Content-Type: application/json

{"item": "book"}
    
  • With a 301 here the body would be lost and v2 would receive a meaningless GET

See It On the Wire

  • A server that answers every /rNNN with that status and Location: /target
  • /target simply reports the method and body it received

curl -L -d 'name=parham' http://127.0.0.1:8232/r303
    
  • -d implies POST. Do not add -X POST — that forces the method on the second request too and hides the effect

Result


301  →  GET  /target   body=''
302  →  GET  /target   body=''
303  →  GET  /target   body=''
307  →  POST /target   body='name=parham'
308  →  POST /target   body='name=parham'
    
  • 301, 302 and 303 all arrived as GET, and the body is gone — nothing warns you
  • 307 and 308 arrived intact
  • curl can be told to keep the method with --post301 --post302, which is exactly the ambiguity those two codes carry

Rules of Thumb

  • Answering a form POST? → 303, pointing at the result
  • Moved an endpoint and the method matters? → 308 permanent, 307 temporary
  • Moved a page a browser only ever GETs? → 301 is fine, and caches will remember it
  • 301 is sticky: browsers cache it hard. Get it wrong on a URL and visitors keep going to the old target long after you fix the server

1xx: Interim Responses

  • A 1xx is not the answer. It means message received, still working — another response is coming
  • So one request can produce two or more responses
  • They never carry a body: a status line and headers, then a blank line
  • You rarely see them because client libraries read past them and hand you only the final response
CodeMeaningStill used?
100 Continuego ahead and send the bodyyes, for big uploads
101 Switching Protocolsthis connection is now something elseyes, WebSocket
102 ProcessingWebDAV keep-alivedeprecated
103 Early Hintsstart fetching these while I thinkyes, and growing

100 Continue

  • You are about to POST a 2 GB file. The server may reject it anyway — wrong credentials, wrong content type, too large
  • Uploading two gigabytes in order to be told 401 is a poor use of everyone's evening
  • So the client sends the headers only, adds Expect: 100-continue, and waits
    • 100 Continue → go ahead, send the body
    • a 4xx → the body is never sent at all
  • A client must not wait forever: if nothing comes back, send the body anyway. Some old servers simply do not answer

100 Continue on the Wire


$ curl -v -X POST -H 'Expect: 100-continue' --data-binary @big.txt \
       http://127.0.0.1:8241/upload

> POST /upload HTTP/1.1
> Host: 127.0.0.1:8241
> Expect: 100-continue
> Content-Length: 3000
>
< HTTP/1.1 100 Continue          <- interim: the body has not been sent yet
<
< HTTP/1.1 200 OK               <- final, after the body went up
< Content-Length: 15
    
  • Two responses, one request. Notice the blank line after the 100 — it is a complete message with no body
  • curl adds the header by itself only for bodies over 1 MB; below that it just sends the body

101 Switching Protocols

  • Sent in response to an Upgrade request header, naming the protocol the server is switching to
  • After it, this is no longer HTTP. The TCP connection carries the new protocol, and there is no third response
  • This is how every WebSocket starts — we come back to it later in the lecture

GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
    

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
    

103 Early Hints

  • A server often knows what the page will need long before it knows what the page is — the database query is still running, but the CSS is always the same
  • 103 lets it say so immediately, with Link: rel=preload headers, and send the real response later
  • The browser spends the server's thinking time downloading instead of waiting

103 on the Wire


$ curl -v http://127.0.0.1:8241/hints

> GET /hints HTTP/1.1
> Host: 127.0.0.1:8241
>
< HTTP/1.1 103 Early Hints
< Link: </style.css>; rel=preload; as=style
< Link: </app.js>; rel=preload; as=script
<
                                 <- server is still working here
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 27
    
  • This is what replaced HTTP/2 Server Push: the server suggests, and the browser decides — so nothing is pushed that the cache already holds

Two Things That Bite

  • HTTP/1.0 clients do not understand 1xx. A server must not send one to a 1.0 client — it would be read as the final response, and the real one as garbage
  • Intermediaries may swallow them. An old proxy that does not recognise 103 can drop it, so it must only ever be an optimisation — never carry information the page needs
  • In HTTP/2 and HTTP/3 there is no status line; a 1xx is simply a HEADERS frame that does not end the stream

HTTP Messages Hands on

  • Connect to a web server
    • Telnet can create TCP socket
  • Play with the server by sending HTTP methods and checking the responses

HTTP Headers

  • Headers are additional information that is sent by client to server and vice versa
    • Most (almost all) are optional
  • Which headers?
    • Information about client
    • Information about server
    • Information about the requested resource
    • Information about the response
    • Security/Authentication
    • ...

HTTP Headers

  • General headers
    • Appear both on request & response messages
  • Request headers
    • Information about request
  • Response headers
    • Information about response
  • Entity headers
    • Information about body (size, ...)
  • Extension headers
    • New headers (not standard)

General Headers

  • Date: Date & Time that message is created
  • Connection: Close or Keep-Alive
    • Close: Non-persistent connection
    • Keep-Alive: Persistent connection
  • Via: Information about the intermediate nodes between two sides
    • Proxy servers

Request Headers

  • Host: The name of the server (required, why?)
  • Referer: URL that contains requested URL
  • Information about the client
    • User-Agent: The client program
    • Accept: The acceptable media types
    • Accept-Encoding: The acceptable encoding
    • Accept-Language: The acceptable language

Request Headers (Cont.)

  • Range: Specific range (in byte) of resource
  • Authorization: Response to the authenticate
    • Will be discussed later
  • Cookie: To return back the cookies
    • Will be discussed later
  • If-Modified-Since: Request is processed if the objected is modified since the specified time.
    • Used in Web Caching
    • Will be discussed later

Response Header

  • Server: Information about server
  • WWW-Authenticate: Used to specify authentication parameters by server
  • Proxy-Authenticate: Used to specify authentication parameters by proxy
  • Set-Cookie: To send a cookie to client
  • Location: The location of entity to redirect client

Response Header (Cont.)

  • Last-Modified: The date and time of last modification of entity
  • Content-Range: Range of this entity in the entire resource
  • Expires: The date and time at which the entity will expire

Entity Headers

  • Content-Length: The length of body (in byte)
  • Content-Type: The type of entity
    • MIME types: text/xml, image/gif
  • Allow: The allowed request methods can be performed on the entity
    • This is in response of OPTIONS method

from https://avatars1.githubusercontent.com/u/8181240?v=4

a GitHub avatar, served as image/png

Extension Headers

  • Custom proprietary headers have historically been used with an X- prefix, but this convention was deprecated in June 2012
  • implementation-specific and private-use parameters could at least incorporate the organization's name
    • ExampleInc-foo
    • VND.ExampleInc.foo (vnd stands for vendor)
  • or primary domain name
    • com.example.foo
    • http://example.com/foo

HTTP Tools

Noodle

  • a terminal REST client: no browser, no account, works over ssh
  • a request is a YAML file and a collection is a directory, so they live next to the code they call and go through git and code review like everything else
  • environments keep the parts that change per machine — $base_url, tokens — out of the request itself
  • imports what you already have: OpenAPI, Postman, Insomnia
  • the same collections run from the command line, which is what makes them usable in CI

A Request is a File

demo/example.yml


name: Example
method: GET
url: https://example.com
timeout: 30000
followRedirects: true
maxRedirects: 5
    

and the same file runs without the TUI


$ noodle collection run demo
✓ GET example  200 OK  491.524791ms
  https://example.com

Summary: 1 passed, 0 failed, 491.524791ms
    
the Noodle TUI: a collection tree on the left, a PATCH request with a
      JSON body in the middle, and the 200 OK response below it

Server-sent events (SSE)

  • Traditionally, a web page has to send a request to the server to receive new data; that is, the page requests data from the server
  • With server-sent events, it's possible for a server to send new data to a web page at any time, by pushing messages to the web page.

Server-Sent Events (SSE) is a server push technology enabling a client to receive automatic updates from a server via an HTTP connection, and describes how servers can initiate data transmission towards clients once an initial client connection has been established.

They are commonly used to send message updates or continuous data streams to a browser client and designed to enhance native, cross-browser streaming through a JavaScript API called EventSource, through which a client requests a particular URL in order to receive an event stream.

The EventSource API is standardized as part of HTML5 by the WHATWG. The media type for SSE is text/event-stream.

HTTP/2 Server Push ⚰️

  • HTTP/2 Server Push let a server send resources to a client before the client asked for them, to save a round trip.
  • In practice servers pushed things the browser already had cached, so it often made pages slower. Chrome removed support in 2022, and it was never really deployed over HTTP/3.
  • The replacement is 103 Early Hints: the server answers early with Link: </style.css>; rel=preload and lets the browser decide what to fetch.

WebSocket Protocol [RFC 6455]

  • HTTP is a request-response protocol
    • Client must initiate every communication
    • Server cannot push data without a request
    • Each request creates overhead (headers, connection setup)
  • WebSocket provides full-duplex, bidirectional communication over a single TCP connection
  • Both client and server can send messages at any time

WebSocket vs SSE vs HTTP Polling

WebSocketSSEPolling
DirectionBidirectionalServer → ClientClient → Server
Protocolws:// or wss://HTTPHTTP
OverheadLowLowHigh
Use CaseChat, GamesNotificationsSimple updates

WebSocket Handshake

  • WebSocket connection starts as an HTTP request with an Upgrade header
  • Server responds with 101 Switching Protocols
  • After handshake, protocol switches from HTTP to WebSocket

Client Request


GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
    

Server Response


HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    

WebSocket in JavaScript


// Create WebSocket connection
const socket = new WebSocket('wss://example.com/socket');

// Connection opened
socket.addEventListener('open', (event) => {
  socket.send('Hello Server!');
});

// Listen for messages
socket.addEventListener('message', (event) => {
  console.log('Message from server:', event.data);
});

// Handle errors
socket.addEventListener('error', (error) => {
  console.error('WebSocket error:', error);
});

// Connection closed
socket.addEventListener('close', (event) => {
  console.log('Connection closed:', event.code, event.reason);
});
    

Sending Data


// Send text
socket.send('Hello!');

// Send JSON
socket.send(JSON.stringify({ type: 'message', data: 'Hello!' }));

// Check connection state
if (socket.readyState === WebSocket.OPEN) {
  socket.send('Connection is open');
}

// Close connection
socket.close(1000, 'Normal closure');
    
  • readyState: CONNECTING (0), OPEN (1), CLOSING (2), CLOSED (3)

WebSocket Use Cases

  • Real-time chat applications
  • Live notifications and alerts
  • Multiplayer games
  • Collaborative editing (Google Docs)
  • Live sports scores and stock tickers
  • IoT device communication

Use WebSocket when you need low-latency, bidirectional communication. For server-to-client only, consider SSE as a simpler alternative.

WebSocket Security

  • Always use wss:// (WebSocket Secure) in production
    • Encrypts data using TLS (same as HTTPS)
  • Origin checking: Server should validate the Origin header
  • Authentication: Send auth token in first message or via URL parameter
  • WebSocket is not subject to same-origin policy
    • Server must implement its own origin validation

Stateless Problem

  • HTTP is a stateless protocol
    • The server does not remember its client between requests
    • Two requests from the same person look exactly alike
  • So how do we recognise a returning visitor?
  • Could we use the client's IP address?
    • Many clients, one address: behind NAT, a whole university leaves through a single public IP
    • One client, many addresses: DHCP hands out a different address tomorrow, and a phone changes address every time it moves between networks
    • An address identifies a connection, not a person
  • So the server has to ask the client to carry the identity on its behalf

Solution of Stateless Problem: Cookie [RFC 6265]

  • Cookie: are information (e.g. unique identifiers) sent by server to user (browser) which are returned back to server
  • How it works
    • Server asks client to remember the information
      • Set-Cookie header in response message
      • Set-Cookie: <cookie-name>= <cookie-value>
    • Client gives back the information to server in every request
      • Cookie header in request messages
      • Cookie: <cookie-name>= <cookie-value>; <cookie-name>= <cookie-value>
    • Server customizes responses according to the cookie

Two Kinds, One Difference: Lifetime

  • Session cookie: no Expires and no Max-Age. The browser drops it when it closes
  • Persistent cookie: carries one of them, and survives a restart until that time passes
  • Session here means the browser session, not your application's login session — a login can be built on either

What Goes In a Cookie?

  • An opaque, unguessable identifier — and nothing else
  • The state itself stays on the server, in a session store

Set-Cookie: session=P4kQ2m9xR7tLv3Nb; Path=/; HttpOnly; Secure; SameSite=Lax
    
  • The value means nothing to the client, and nothing to whoever steals it once the server has forgotten it
  • Revoking a login is then one delete on the server, and the cookie in the browser becomes worthless

Why Not Put the Data Itself In?


Set-Cookie: user=parham; role=admin
    
  • The client can edit itrole=admin is one DevTools click away
  • It rides along on every single request, forever
  • You cannot revoke it: there is no way to reach into a browser and change your mind
  • If claims really must travel in the cookie, they have to be signed — which is what a JWT is, and we come back to it under Authentication

Cookies in Action


GET /cookies/set?name=parham&family=alvani HTTP/1.1
Host: httpbin.org

    

HTTP/1.1 302 FOUND
Date: Mon, 07 Sep 2020 05:19:50 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 223
Connection: keep-alive
Server: gunicorn/19.9.0
Location: /cookies
Set-Cookie: name=parham; Path=/
Set-Cookie: family=alvani; Path=/
    

GET /cookies HTTP/1.1
Host: httpbin.org
Cookie: name=parham; family=alvani
    

HTTP/1.1 200 OK
Date: Mon, 07 Sep 2020 05:23:53 GMT
Content-Type: application/json
Content-Length: 58
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "cookies": {
    "family": "alvani",
    "name": "parham"
  }
}
    

Limits

  • Limitation
    • Cannot be used to store large data
    • At least 4096 bytes per cookie
    • At least 50 cookies per domain
    • At least 3000 cookies in total
    • These are the minimums a browser must support, from RFC 6265. The older 300/20 figures come from RFC 2965 and no longer apply
  • The server never has to ask for a cookie back
    • The browser attaches the matching ones by itself, every request
    • Convenient — and the reason CSRF exists at all

Who Controls a Cookie?

  • Client can control cookies
    • Disable cookies: no cookie is saved & used
    • View & Delete cookies
  • Server can control cookies by its attributes
    • Expiration time
    • Domain
    • Path
    • Security
    • ...

Cookies Attributes

  • Expire & Max-Age: The life time of the cookie
    • Expire: An absolute time to delete cookie
    • Max-Age: The maximum life time (sec) of cookie
    • If either is present, the cookie is persistent
    • If neither is, it is a session cookie
    • Send a past time (or negative) to delete a cookie
  • Secure: Cookie is sent only if channel is secure
    • Specially useful for login sessions cookies
    • A cookie with the Secure attribute is sent to the server only with an encrypted request over the HTTPS protocol, never with unsecured HTTP (except on localhost), and therefore can't easily be accessed by a man-in-the-middle attacker.
  • HttpOnly: the cookie is sent on HTTP requests but is invisible to JavaScript
    • It does not appear in document.cookie
    • Nothing to do with http versus https — that is the Secure attribute above
    • This is what stops an XSS payload from stealing the session cookie
  • SameSite allows you to declare if your cookie should be restricted to a first-party or same-site context
    • Lax: Cookies are not sent on normal cross-site subrequests (for example to load images or frames into a third party site), but are sent when a user is navigating to the origin site (i.e. when following a link). This is the default in modern browsers.
    • Strict: Cookies will only be sent in a first-party context and not be sent along with requests initiated by third party websites.
    • None: Cookies will be sent in all contexts, i.e in responses to both first-party and cross-origin requests. If SameSite=None is set, the cookie Secure attribute must also be set (or the cookie will be blocked).

Cookies Attributes: Domain & Path

  • Domain & Path determine the scope of the cookie
    • For which path and domain, the cookie is saved & returned back by browser
  • If Domain is omitted, defaults to the host of the current document URL.
    • Browser returns back the cookie for the domain and not sub-domains
  • If Path is omitted, defaults to the path of the current document URL.
    • Browser returns back the cookie for the path and also for all sub-paths
  • If present then browser checks validity
    • If they are valid then Browser returns back the cookie for that domain & that path and also for all sub-domains and sub-paths

Cookies Attributes: Domain & Path

  • Validity check by major browsers
    • Contrary to earlier specifications, a leading dot (.example.com) is ignored — write example.com
    • Don’t accept for other domains than the base domain
    • Don’t accept cookies for sub-domains
    • Accept cookies for higher domains
      • Except public suffixes, e.g. .com or .ac.ir — otherwise one site could set a cookie for every university in Iran
      • Browsers use the Public Suffix List to decide where that boundary is
    • Accept cookies for other (sub or higher) paths
      • A path that must exist in the requested URL, or the browser won't send the Cookie header.
which cookies a browser returns for each path
how a Domain attribute widens a cookie to sub-domains

Cookies Attributes: Domain & Path: Hands on

  • Ask a server to set two cookies, scoped to different paths
  • Then request two different paths, and see which cookies the browser attaches

curl -D - -c jar.txt http://127.0.0.1:8210/login
    

The Server Sets Them


HTTP/1.0 200 OK
Set-Cookie: session=abc123; Path=/
Set-Cookie: admin_token=xyz789; Path=/admin
    

Result


$ curl -b jar.txt http://127.0.0.1:8210/
path=/ cookies=session=abc123

$ curl -b jar.txt http://127.0.0.1:8210/admin/panel
path=/admin/panel cookies=admin_token=xyz789; session=abc123
    
  • session goes everywhere, because its path is /
  • admin_token is sent only under /admin, and the server never even sees it elsewhere
  • Path is about scoping, not security: any script on the origin can still read it unless the cookie is HttpOnly

Cookie Prefixes

  • A cookie's name can carry a rule the browser enforces when it is set
    • __Secure-: accepted only if it was set with Secure, over HTTPS
    • __Host-: the same, plus Path=/ and no Domain attribute at all

Set-Cookie: __Host-session=P4kQ2m9xR7tLv3Nb; Path=/; Secure; HttpOnly; SameSite=Lax
  
  • Remember two slides ago: a sub-domain can set a cookie on the parent domain. __Host- is what stops evil.example.com from handing you a cookie that looks like yours
  • The rule travels in the name, so no proxy, framework or refactor can quietly drop it

Third-Party Cookies

  • A cookie set by a domain other than the one in the address bar — an ad, a font, an embedded widget
  • That third party sees the same cookie on every site that embeds it, which is how cross-site tracking was built
  • Safari and Firefox block them by default. Chrome's plan to remove them has been delayed and rescoped several times — but building anything new on them is building on sand
  • This is the whole reason SameSite=None; Secure exists: it is how you opt in to a cross-site cookie, and it is exactly what is going away

CSRF, in One Slide

  • The browser attaches your cookie to any request to that origin, no matter who caused it
  • So a form on evil.example can POST to bank.example, and it arrives authenticated
  • SameSite is the first defence, a per-session token is the second
a request from an attacker's page to the bank, carrying the victim's cookie

Covered properly in the Web Security lecture.

Warning!

Neither Strict nor Lax are a complete solution for your site's security. Cookies are sent as part of the user's request and you should treat them the same as any other user input. That means sanitizing and validating the input. Never use a cookie to store data you consider a server-side secret.

Single Sign-on (SSO)

  • One login, many applications: you authenticate once at a central identity provider, and every service trusts the result
  • The cookie lives on the provider's domain, so each application still establishes its own session afterwards — that hand-off is the interesting part
  • e.g. CAS Enterprise Single Sign-On, and the OAuth 2.0 / OIDC flows in the Web Security lecture

Proxy

  • Proxies sit between client and server
  • Act as server for client
  • Act as client for server
proxy

Forward Proxies

  • A forward proxy, or gateway, or just "proxy" provides proxy services to a client or a group of clients.

Reverse Proxies

  • As the name implies, a reverse proxy does the opposite of what a forward proxy does:
  • A forward proxy acts in behalf of clients (or requesting hosts), a reverse proxy acts in behalf of servers.
  • Forward proxies can hide the identities of clients whereas reverse proxies can hide the identities of servers.
  • Reverse proxies have several use cases, a few are:
    • Load balancing: distribute the load to several web servers,
    • Cache static content: offload the web servers by caching static content like pictures,
    • Compression: compress and optimize content to speed up load time.
reverse-proxy

HTTP Proxy Applications

  • Authentication
    • Client side: Authenticate clients before they access web
    • Server side: Authenticate clients before access the server
  • Accounting: Log client activities
  • Security: Analyze request before sending it to server
    • Integrated in modern firewalls
  • Filtering: Limit access to specified contents
  • Anonymizer: Anonymous web browsing
  • Caching (more details in the following slide)

Caching

  • Caching: save a copy of a resource and use it instead of requesting server
  • Browser has its own local caches
  • Cache server is special proxy for caching
  • Benefits
    • Reduce redundant data transfer
    • Reduce network bottleneck
    • Reduce load on server
    • Reduce delay
browser cache, proxy cache and gateway cache along one request path

Caching Algorithm

  • If the object is not cached, it is got from server, saved in cache, and sent to client
  • Else, if object is in cache
    • Cache server must return only fresh objects
    • Freshness check
  • Objects life-time specified by server
  • The Cache-Control HTTP/1.1 general-header field is used to specify directives for caching mechanisms in both requests and responses.

Expiration

the maximum amount of time a resource will be considered fresh.


Cache-Control: max-age=<seconds>
    

No caching

The cache should not store anything about the client request or server response.


Cache-Control: no-store
    

Cache but revalidate

A cache will send the request to the origin server for validation before releasing a cached copy.


Cache-Control: no-cache
    
  • If requested object is not expired
    • Cache server gives it to client

Caching Algorithm (Cont.)

  • If requested object is expired
    • Its freshness must be checked
  • Freshness is checked by conditional request
    • If-Modified-Since: current last-modified time
    • If-None-Match: the server will send back the requested resource, with a 200 status, only if it doesn't have an ETag matching the given ones.
      • The ETag HTTP response header is an identifier for a specific version of a resource.
  • Server responses
    • 304 Not modified response + new expire time
      • Cached copy is valid until the specified time
    • 200 OK
      • Server provides a new version of the object
      • Cache server updates cached copy
the life of a cached response: fresh, then stale, then revalidated

Authentication vs Authorization

  • Authentication is the verification of the credentials of the connection attempt.
  • This process consists of sending the credentials from the remote access client to the remote access server in an either plaintext or encrypted form by using an authentication protocol.
  • Authorization is the verification that the connection attempt is allowed.
  • Authorization occurs after successful authentication.
  • In short: authentication answers who are you?, authorization answers what may you do?

HTTP Authentication

  • All resources are not public in web; e.g.,
    • Financial documents, Customer information, ...
  • HTTP has two (similar) authentications
    • Basic: Base64 encoded user:pass [The username itself cannot contain a colon]
    • Digest: Plain username + Digest of pass
  • Steps are the same
      1. Client-side app (browser) request resource from server
      2. Server refuses with 401 Unauthorized
      3. Client-side app ask Username & Password from user
      4. Client send Username & Password to server
      5. Server authenticates and allows.
    • Authentication information are sent by every request until end of current session

Base64

  • Base64 is a group of binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation.
three bytes encoded as four base64 characters

An additional pad character is allocated which may be used to force the encoded output into an integer multiple of 4 characters (or equivalently when the unencoded binary text is not a multiple of 3 bytes) ; these padding characters must then be discarded when decoding but still allow the calculation of the effective length of the unencoded text, when its input binary length would not be not a multiple of 3 bytes (the last non-pad character is normally encoded so that the last 6-bit block it represents will be zero-padded on its least significant bits, at most two pad characters may occur at the end of the encoded stream).

If unpadded strings are concatenated, it's impossible to recover the original data because information about the number of odd bytes at the end of each individual sequence is lost

HTTP Authentication (Cont.)

basic-auth-in-action

HTTP Authentication (Cont.)


GET /basic-auth/admin/admin HTTP/1.1
Host: httpbin.org
Authorization: Basic YWRtaW46YWRtaW4=

  

HTTP/1.1 200 OK
Date: Mon, 07 Sep 2020 14:14:25 GMT
Content-Type: application/json
Content-Length: 48
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "authenticated": true,
  "user": "admin"
}
  

Digest Authentication

  • Basic authentication is insecure
    • Password is sent in base64 encoding
    • Attacker can easily find it
  • Digest authentication: Don’t send password
    • Send its digest (hash)
  • Digest/hash function
    • One way function, irreversible
  • Attacker cannot find password 😌
  • But! Replay attack 😞
    • Attacker resends the same digest then he will be authenticated
    • Use Nonce
  • Digest authentication uses nonce and digest together

Digest Authentication (Cont.)

  • Client requests a private resource
  • Server creates a nonce (the server only issues a new nonce
    for each 401 response)

WWW-Authenticate: Digest realm="testrealm@host.com",
                        qop="auth,auth-int",
                        nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
                        opaque="5ccc069c403ebaf9f0171e9517f40e41"
    
  • Client computes digest of password & nonce
  • 
    HA1 = MD5(username:realm:password)
    HA2 = MD5(method:digestURI)
    response = MD5(HA1:nonce:HA2)
          
    
    Authorization: Digest username="Mufasa",
                         realm="testrealm@host.com",
                         nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
                         uri="/dir/index.html",
                         qop=auth,
                         nc=00000001,
                         cnonce="0a4f113b",
                         response="6629fae49393a05397450978507c4ef1",
                         opaque="5ccc069c403ebaf9f0171e9517f40e41"
          
  • Server looks up the password of username and computes
    hash(pass, nonce)
  • RFC 7616 adds SHA-256 and deprecates MD5; the example above is the classic MD5 one from RFC 2617
  • In practice you will almost never meet Digest today — it is TLS plus a token instead

In the Wild

  • There is no need for basic/digest authentication in our new world we can send plaintext passwords over secure HTTP.
  • In both ways, we need a solution for stateful connection to not repeat the authentication procedure for every request.
courses.aut.ac.ir
moodle-cookies moodle-request moodle-post

Session Authentication

  • The answer to do not repeat the procedure for every request is the cookie we met earlier
  • Check the password once, then hand out an opaque session identifier
  • The browser attaches it automatically from then on — that is the whole point of a cookie
  1. POST /login with the credentials
  2. Set-Cookie: session=… on the response
  3. Every later request carries Cookie:
  4. The server looks the identifier up in its session store

Logging In


$ curl -sS -D - -o /dev/null http://127.0.0.1:8211/me
HTTP/1.1 401 Unauthorized

$ curl -sS -D - -c jar.txt -d 'user=parham&pass=s3cret' \
       http://127.0.0.1:8211/login
HTTP/1.1 303 See Other
Location: /me
Set-Cookie: session=5r1dBZIscs2oNeux; Path=/; HttpOnly; SameSite=Lax
    
  • 303: POST / Redirect / GET again — the password must not sit in a reloadable request
  • The password appears once, in a body, over TLS. After this it never travels again

Every Request After


GET /me HTTP/1.1
Host: 127.0.0.1:8211
Cookie: session=5r1dBZIscs2oNeux

HTTP/1.1 200 OK

hello parham
    
  • session=5r1dBZIscs2oNeux means nothing on its own — the name parham lives in the server's session store
  • HttpOnly keeps an XSS payload from reading it; SameSite keeps another site from riding it

Logging Out is a Server-Side Delete


$ curl -sS -D - -b jar.txt -X POST http://127.0.0.1:8211/logout
HTTP/1.1 200 OK
Set-Cookie: session=; Path=/; Max-Age=0

$ curl -sS -b jar.txt http://127.0.0.1:8211/me   # jar still has the cookie
unknown session
    
  • The Max-Age=0 cookie is only a courtesy to the browser
  • What actually ends the login is the delete in the session store — a stolen copy of the identifier is worthless the moment the server forgets it

Session vs Token

Session cookieBearer token
StateOn the serverIn the token
Sent byThe browser, automaticallyYour code, explicitly
RevokeOne deleteWait for expiry
CSRFExposed, needs SameSiteImmune
FitsA site with a browser in front of itAPIs, mobile, service to service

Bearer Authentication

  • Bearer authentication (also called token authentication) is an HTTP authentication scheme that involves security tokens called bearer tokens
  • The name Bearer authentication can be understood as give access to the bearer of this token.

JWT

  • JSON Web Tokens are an open, industry standard RFC 7519 method for representing claims securely between two parties.

Sample Token


eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IlBhcmhhbSBBbHZhbmkiLCJpYXQiOjE1MTYyMzkwMjIsInByb2plY3QiOiJhbiBhd2Vzb21lIHByb2plY3QifQ.
gWWHu5Ps_F6lbqJRBXkNjEk_-0QdLhN9l2MNjWOcj90
    

{
  "alg": "HS256",
  "typ": "JWT"
}

{
  "sub": "1234567890",
  "name": "Parham Alvani",
  "iat": 1516239022,
  "project": "an awesome project"
}
    

Snapp! Token


{
  "alg": "RS512",
  "kid": "z8a4l4oOFEqgehRYDBZP+fprPnLDLmabkslOxVVpLNE",
  "typ": "JWT"
}
{
  "aud": [
    "passenger"
  ],
  "email": "parhamalvani@gmail.com",
  "exp": 1646469738,
  "iat": 1645260138,
  "iss": 1,
  "jti": "2NFKm5FfEey65wIArBQAz289hDgf/E0gjnyXrNCM0v4",
  "sid": "25JzmlUBAwtMfQvT7qmOalw5M7p",
  "sub": "KpQxO5glyv04Ad1"
}
    

Security

  • Digest authentication protect password only
  • Data is completely insecure
  • No mechanism in HTTP to protect data
  • HTTP over TLS (HTTPS) is the solution
    • An encrypted tunnel between client & server
    • Send HTTP traffic through the tunnel
HTTP carried inside an encrypted TLS tunnel

Seeing the Tunnel

  • Put a tap on the path — a relay that prints every byte it forwards, which is all a proxy or a router ever is
  • Then send the same request to port 80 and to port 443

$ curl --connect-to example.com:80:127.0.0.1:9080   http://example.com/
$ curl --connect-to example.com:443:127.0.0.1:9443 https://example.com/
    

Port 80: It Is Just Text


C->S  47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a  |GET / HTTP/1.1..|
C->S  48 6f 73 74 3a 20 65 78 61 6d 70 6c 65 2e 63 6f  |Host: example.co|
C->S  6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63  |m..User-Agent: c|
C->S  75 72 6c 2f 38 2e 37 2e 31 0d 0a 41 63 63 65 70  |url/8.7.1..Accep|
C->S  74 3a 20 2a 2f 2a 0d 0a 43 6f 6e 6e 65 63 74 69  |t: */*..Connecti|
C->S  6f 6e 3a 20 63 6c 6f 73 65 0d 0a 0d 0a           |on: close....|
S->C  48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d  |HTTP/1.1 200 OK.|
S->C  0a 44 61 74 65 3a 20 4d 6f 6e 2c 20 31 37 20 41  |.Date: Mon, 17 A|
    
  • No decoding needed — the path, the headers and the cookies are right there, and anyone on the path can change them too

Port 443: The Same Request


C->S  17 03 03 00 6e b8 bc 67 ba 90 7c 40 2c e2 bb 5a  |....n..g..|@,..Z|
C->S  c0 1d 7b 7f 27 49 9e aa 81 81 cc 7f 10 d6 a1 40  |..{.'I.........@|
C->S  06 41 a7 a3 9f 94 ab 22 a9 40 4c 03 f2 25 4a b2  |.A.....".@L..%J.|
C->S  ca 54 00 a3 38 89 bb cb 10 bf 2e a1 a8 24 03 94  |.T..8........$..|
C->S  8c f2 24 6b 71 19 40 f4 3b 24 d6 66 fe 1a e7 5c  |..$kq.@.;$.f...\|
C->S  49 9d 2a 71 63 6d d4 b6 6a 53 30 33 bc ee b5 77  |I.*qcm..jS03...w|
C->S  cf a2 30 c9 06 ff 4d 81 0c 8d d1 7e 5b 1c b4 54  |..0...M....~[..T|
C->S  7e 53 45                                         |~SE|
    
  • 17 = application data, 03 03 = record version, 00 6e = 110 bytes follow
  • That is the whole HTTP request — the record header is the only thing the tap can still parse

What TLS Does Not Hide


C->S  16 03 01 01 39 01 00 01 35 03 03 0f 96 b0 2f 41  |....9...5...../A|
      ...  cipher suites, key share, 220 bytes  ...
C->S  0e 00 00 0b 65 78 61 6d 70 6c 65 2e 63 6f 6d 00  |....example.com.|
    
  • The ClientHello is sent before any key exists, so the SNI hostname goes out in the clear
  • So do the destination IP, and the size and timing of every message — enough to guess which page you loaded
  • Encrypted Client Hello (ECH) is the fix for the hostname; the rest is the price of using a network

References 📚

Fork me on GitHub