Parsing HTTP

• An HTTP/1.1 message is a request line, then header lines, then CRLF CRLF, then an optional body. The parser is a state machine over a byte stream; it is fed whatever arrives and must be able to stop mid-token and resume.
• Body framing is decided by rules, in order: certain statuses and methods have no body; Transfer-Encoding: chunked means the body is a sequence of length-prefixed chunks terminated by a zero-length chunk; otherwise Content-Length gives the exact byte count; otherwise there is no body.
• If both Content-Length and Transfer-Encoding: chunked are present, the specification says chunked wins and the Content-Length must be ignored or the message rejected. Disagreement about this rule between two hops is the classic smuggling primitive.
• Header names are case-insensitive, so servers normalise them — typically to lower case. Values are byte strings; a header may legitimately appear multiple times, and how those are combined is a per-header rule rather than a global one.
• The URL is parsed separately: path, query string, percent-decoding, and normalisation of . and .. segments. Whether decoding happens before or after routing and authorization checks decides whether %2e%2e%2f is a traversal (Transport Validation).
• The parser enforces the limits that exist before identity does: maximum request-line length, maximum header size, maximum header count, and a timeout on how long the headers may take to arrive.
• HTTP/2 replaces all of the above with binary frames and HPACK-compressed headers, so "parsing" becomes framing plus decompression — and introduces its own resource concerns, since a small compressed header block can expand into a large one.

state: START -> REQUEST_LINE -> HEADERS -> (BODY_LENGTH | BODY_CHUNKED | NO_BODY) -> DONE

feed(bytes):
  buffer += bytes
  if len(buffer) > limit_for(state): reject(431 or 400)   # budget per state

  while progress_possible:
    START/REQUEST_LINE: need a full CRLF-terminated line, else return and wait
    HEADERS:            each CRLF line is a header; blank line ends the block
                        -> decide framing here, ONCE, and reject if ambiguous
    BODY_LENGTH:        consume exactly Content-Length bytes
    BODY_CHUNKED:       read size line (hex), then that many bytes, then CRLF;
                        a 0-size chunk ends the body
    DONE:               hand the request up; anything LEFT in the buffer is
                        the start of the NEXT request on this connection

timeout in START/REQUEST_LINE/HEADERS -> close, do not wait forever
error in any state -> close the connection; do not try to resynchronise

• Use your runtime's HTTP parser. Correct, hostile-input-safe HTTP parsing is a large body of work, and a hand-rolled parser reachable from the internet is a liability rather than an optimisation.
• Set explicit limits rather than inheriting defaults: header size, header count, request-line length and the header-read timeout. Know the numbers, because they are the only defence that runs before authentication.
• Make the proxy and the origin agree. Same limits, same body-framing behaviour, same treatment of ambiguous requests — divergence is the smuggling surface (The Request Lifecycle).
• Reject ambiguity rather than guessing. A request with both Content-Length and Transfer-Encoding, or with two conflicting Content-Length headers, should be a 400, not an interpretation.
• Read headers case-insensitively through the framework's accessor, never by indexing a raw object with a capitalised key.
• Treat the parsed result as untrusted structured data: parsing proves the shape, not the meaning (The Three Validations).