bhttp: Binary Representation of HTTP Messages

Today we're releasing confidentsecurity/bhttp, a Go implementation of RFC 9292 Binary Representation of HTTP Messages, or BHTTP for short.

Our library's features include:

• Encoding and decoding to/from *http.Request and *http.Response types.
• Indeterminate-length and known-length BHTTP messages
• Encoding/decoding logic that behaves like `net/http` but can be overwritten when required
• Trailers
• Padding

What is BHTTP?

BHTTP is a low-overhead encoding that describes the semantics of HTTP messages. It won't capture the nitty-gritty details of your HTTP requests and responses, but it will leave their meaning intact.

It's designed for use cases that require HTTP request/responses to be passed outside the regular HTTP protocol.

For example, Oblivious HTTP (OHTTP) uses BHTTP to encode requests/responses before passing them around as encrypted messages.

Side note: If that piqued your interest, check out our other package, twoway, which implements exactly this kind of encrypted messaging.

Encoding and decoding

confidentsecurity/bhttp allows you to encode regular net/http requests and responses to BHTTP.

To begin encoding an *http.Request or *http.Response, you will first need to have an encoder.

The zero values of all encoders and decoders in confidentsecurity/bhttp implement sensible defaults, so the following code is enough to create a request encoder:

encoder := &bhttp.RequestEncoder{}

If you then call encoder.EncodeRequest with a request, you will get back an io.Reader or error in return:

// encode a request to BHTTP
// (req is a *http.Request)
msg, err := encoder.EncodeRequest(req)
if err != nil {
    // ...
}

// msg is an io.Reader

Since the message is an io.Reader it doesn't matter whether it's a known-length or indeterminate-length (streaming) message. Both can be treated the same way.

Decoding is just as easy. You construct a decoder, provide it with an io.Reader and receive an *http.Request.

decoder := &bhttp.RequestDecoder{}

// need to provide a context for the request.
decReq, err := decoder.DecodeRequest(context.Background(), msg)
if err != nil {
    // ...
}

// decReq is an *http.Request that can be used as normal.

Response encoding/decoding works similarly:

encoder := &bhttp.ResponseEncoder{}

// resp is a *http.Response//
msg, err := encoder.EncodeResponse(resp)
if err != nil {
    // ...
}

decoder := &bhttp.ResponseDecoder{}
decResp, err := decoder.DecodeResponse(context.Background(), msg)
if err != nil {
    // ...
}

// decResp is a *http.Response

Known-Length vs. Indeterminate-Length Encoding

BHTTP supports two kinds of messages:

• Known-length messages: Messages of which the length is known in advance.
• Indeterminate-length messages: Messages that end when an "end-of-message" indicator is encountered. Intended for streaming use cases.

This distinction is independent of the original HTTP request or response.

For example, an HTTP request with a Content-Length header can be encoded as an indeterminate-length BHTTP message, even though the length is actually known and set in the header.

Similarly, an HTTP response without a Content-Length header can be encoded as a known-length BHTTP message (it will require the full response to be read into memory to determine the length).

So how does confidentsecurity/bhttp decide what kind of BHTTP message to use?

By default it will use the same heuristics that net/http uses when it decides whether a request or response should use Transfer-Encoding: chunked.

Side note: Transfer-Encoding: chunked is not used by BHTTP and will be ignored by confidentsecurity/bhttp.

If you always want to encode the same kind of message, the package includes convenience functions that do just that:

• NewKnownLengthRequestEncoder
• NewIndeterminateLengthRequestEncoder
• NewKnownLengthResponseEncoder
• NewIndeterminateLengthResponseEncoder

If none of the provided options suit your needs, you can provide a custom MapFunc to the encoders; this allows you to fully customize the way a *http.Request or *http.Response is mapped to a BHTTP message.

Trailers

Trailers are fully supported on both requests and responses; they work just like in net/http.

1. The .Trailer field on the *http.Request or *http.Response needs to be initialized to a map with keys.
2. The values for the trailer headers should be set before the request/response Body returns io.EOF.

Internally, confidentsecurity/bhttp will:

• Write the required Trailer header to the BHTTP message.
• Write the header values to the BHTTP message after the body returns io.EOF.

Padding

BHTTP supports padding by zeroes as specified in the RFC. This is especially useful when you want to encrypt the encoded result and hide its true length.

In confidentsecurity/bhttp you can enable padding by setting the PadToMultipleOf encoder field to a value greater than 1. This will then pad the resulting output with zeroes until the total message length reaches a multiple of that value.

Wrap-Up

If you're working on an issue where you need to pass HTTP messages around outside of the regular channels, be sure to check out confidentsecurity/bhttp.