Message framing and protocol parsing. Protocol GClasses sit between a transport (C_TCP) and the application logic, translating raw bytes into structured events.
Source: kernel/c/root-linux/src/c_prot_http_cl.c, c_prot_http_sr.c,
c_prot_tcp4h.c, c_prot_raw.c, c_websocket.c
C_PROT_HTTP_CL¶
HTTP client protocol — sends requests and parses responses.
Can deliver the body incrementally (raw_body_data) or as a complete
message.
| Property | Value |
|---|---|
| States | ST_DISCONNECTED, ST_CONNECTED |
| Input events | EV_CONNECTED, EV_STOPPED (disconnected). EV_SEND_MESSAGE, EV_RX_DATA, EV_TIMEOUT, EV_TX_READY, EV_DISCONNECTED, EV_DROP (connected) |
| Output events | EV_ON_MESSAGE, EV_ON_HEADER, EV_ON_OPEN, EV_ON_CLOSE |
Key attributes¶
| Attribute | Type | Description |
|---|---|---|
url | string | HTTP endpoint URL. |
timeout_inactivity | integer | Inactivity timeout in milliseconds (60000). |
cert_pem | string | TLS certificate (PEM). |
crypto | json | TLS config forwarded to the bottom C_TCP for https:// URLs. Default {"ssl_use_system_ca": true, "ssl_verify_mode": "required"} — verifies the server cert against the system CA. Override with ssl_trusted_certificate for a private CA, or ssl_allow_insecure_client: true to skip verification (MITM risk). |
raw_body_data | bool | TRUE to deliver the head in EV_ON_HEADER and the body in pieces in EV_ON_MESSAGE (__pbf__, __pbf_size__). The last EV_ON_MESSAGE, without __pbf__, says the body is whole. Each response of a connection has the headers of its own. |
max_body_size | integer | Longest body of a response kept, in bytes (default 0: half the max block of the yuno. A bigger value is capped to that half, so the buffer that grows to hold it is always one block that can be allocated). With raw_body_data the body is not held: 0 is no limit. See A response refused below. |
max_header_size | integer | Longest head of a response (status text, header names and values, and 64 bytes for each header), in bytes (default 65536, and 0 is that too. Never more than half the max block). See A response refused below. |
max_tx_queue | integer | Requests waiting in the transport from which a server that writes nothing is one that does not read (default 64, 0: no limit). See A server that does not read below. |
timeout_tx_stall | integer | Milliseconds with max_tx_queue requests waiting and no write ended (default 30000, 0: never). See A server that does not read below. |
subscriber | pointer | Gobj receiving output events. |
A negative limit is taken as its default, with an ERROR at the creation (“A limit of C_PROT_HTTP_CL is negative: its default taken”).
An interim response (1xx, such as 100 Continue or 103 Early Hints) is
not published: the final response that follows it on the connection is the
EV_ON_MESSAGE of the request. The first one of a connection is said in a DEBUG
log, and the parser counts all of them (interim_dropped). A 101 Switching Protocols is published, as
it is the answer to an upgrade. The connection is keep-alive, and nothing
pairs an answer with its request, so up to 7.31.29 a published 1xx was taken
as the answer, and each later answer went to the request after its own.
Sending a request: EV_SEND_MESSAGE¶
Sent to the client once it is connected (after its EV_ON_OPEN). The kw:
| Key | Default | Description |
|---|---|---|
method | "GET" | The HTTP method. |
resource | "/" | The path of the request. |
headers | — | A dict of headers, written over the defaults (User-Agent, Connection, Accept, Host, and Content-Type: application/json when there is data). |
data | — | The body. Any json value (7.27.0+ also a bare string or number), sent compact. |
query | "" | With Content-Type: application/x-www-form-urlencoded, the form as text, sent as it is (already encoded). When it is empty, the form is built from data (a dict: k1=v1&k2=v2, each key and value encoded). |
With application/x-www-form-urlencoded the form is the body, except in a
GET, which carries it as the query of the resource (/path?k1=v1). A GET
with no form has no ?. A form built from data is form-urlencoded, each
key and each value: the unreserved bytes of RFC 3986 (A-Z a-z 0-9 - . _ ~)
as they are, a space as +, any other byte as %XX. So a +, %, & or
= of a value is that byte for the server, never a space, an escape or the
start of another field: pass the values as they are, never encoded.
{"user": "user+1@x.com", "pass": "p a%41&b=c"} goes as
user=user%2B1%40x.com&pass=p+a%2541%26b%3Dc. Every byte of a string goes,
a NUL too (%00). An empty string goes as key= (an empty password is a
password). A number or a boolean goes as its json text ({"n": 5, "r": 1.5, "t": true} is n=5&r=1.5&t=true). A null, an object or an array has no
text in a form: the request is not sent, with ONE error (“A value of a form
must be a string, a number or a boolean”), and the action answers -1. The same
applies to a data that is not a dict. The form is built in a buffer of its exact
size, so a value as long as the max block allows goes (6 MB with the default
max block of 16 MB). A form with no fields ("data": {}) goes with
Content-Length: 0. Up to 7.29.3 the values were
written as they are, so a value with & added fields to the form. A query
text is the caller’s, and goes as it is. Up to 7.26.9 a GET lost the last
character of its query.
/* GET /users?first=0&max=50 */
gobj_send_event(gobj_http, EV_SEND_MESSAGE, json_pack("{s:s, s:s, s:{s:s}, s:{s:s, s:s}}",
"method", "GET",
"resource", "/users",
"headers", "Content-Type", "application/x-www-form-urlencoded",
"data", "first", "0", "max", "50"
), gobj);
/* POST /token, the form as text */
gobj_send_event(gobj_http, EV_SEND_MESSAGE, json_pack("{s:s, s:s, s:s, s:{s:s}}",
"method", "POST",
"resource", "/token",
"query", "grant_type=refresh_token&refresh_token=...",
"headers", "Content-Type", "application/x-www-form-urlencoded"
), gobj);
/* PUT /actions with a json array body */
gobj_send_event(gobj_http, EV_SEND_MESSAGE, json_pack("{s:s, s:s, s:[s]}",
"method", "PUT",
"resource", "/actions",
"data", "UPDATE_PASSWORD"
), gobj);A request that cannot be built (a part over the maximum block, a data that
cannot be dumped) is not sent: the action logs an ERROR and answers -1.
A server that does not read. The queue of the transport moves only when a
write ends, in the event loop, so a burst of requests sent in one turn fills
it whether the server reads or not: its length says nothing of the server.
What says it is that no write ends. Each request looks: with max_tx_queue
requests waiting a wait of timeout_tx_stall milliseconds starts, any write
that ends starts it again, and the request that finds it over with nothing
written is refused (the action answers -1), with ONE warning (“HTTP server
does not read the requests, request refused and connection dropped”), and
the connection is dropped. So the requests of a turn are never refused. The
check runs when a request is sent: with none coming, timeout_inactivity
ends the connection. While the connection is dropped every request sent is
refused the same way (-1, and one warning per connection, “HTTP request sent
while the connection is dropped, refused, the first one of the connection”): a
server that never reads can hold the close for as long as the kernel takes to
give up, and if accepted, those requests go nowhere. Up to 7.29.3 the action took them
with 0.
A slow server can be taken as one that does not read. A write ends when the
kernel has taken the last of it, and the kernel wakes a blocked write only when
a large part of the send buffer is free. So a server that reads, but slower
than about send buffer / (2 x timeout_tx_stall), ends no write in that
time and is cut. Linux grows the send buffer up to net.ipv4.tcp_wmem, 4 MB by
default, so at the default of 30 s the minimum speed is about
4 MB / (2 x 30 s) = 68 KB/s. Raise timeout_tx_stall for a server slower
than that, or set 0.
/* a client that gives a stuck server one second */
hgobj http = gobj_create("idp", C_PROT_HTTP_CL, json_pack("{s:s, s:i, s:i}",
"url", "https://idp.example.com:8443",
"max_tx_queue", 16,
"timeout_tx_stall", 1000
), gobj);A response: EV_ON_MESSAGE¶
The kw carries response_status_code, headers (a dict, the names in upper
case: RETRY-AFTER), body_size (the bytes of the body), and the body: a
body json for application/json, else a gbuffer. An empty body is
"body": "" with body_size 0, which tells it apart from the JSON string
"" (body_size 2).
A response refused. A response whose body passes max_body_size, or
whose head passes max_header_size, is not read to its end. Nor is one that
breaks the protocol after its status line (a CR alone in a header, for one).
The client logs ONE warning (“HTTP body larger than its limit, message refused”,
“HTTP headers larger than their limit, message refused”, or “Protocol
violation: non-HTTP data received”), publishes EV_ON_MESSAGE with no
body nor gbuffer and one of "body_too_large": <limit>,
"headers_too_large": <limit>, "protocol_error": "<reason>" (the request
waiting for it is answered), and drops the connection. The client never
sends a request again by itself. Its headers hold what came, each a string.
Bytes that are not a response (garbage alone, or after a whole response) give
the same warning and drop, and no EV_ON_MESSAGE: nothing answers, and
the request waiting has EV_ON_CLOSE. With raw_body_data nothing is
published either, in any of these cases: a last EV_ON_MESSAGE, if published, says the
body is whole, so the subscriber takes EV_ON_CLOSE without it for the
failure. Once the connection is dropped (EV_DROP, the inactivity timeout,
a response refused) nothing more of what was received is published, and a
request sent before it closes is refused (-1, “HTTP request sent while the
connection is dropped, refused, the first one of the connection”). A write
in flight can keep it open for a while. Up to 7.30.14 a refused response left
the connection open to requests: they went to the transport, which threw them
away (“tcp data sent while the connection closes, dropped”), and the action
did not answer -1. Up to 7.29.3 a body over the max block logged an ERROR with a stack for every
piece that arrived, and was handed up cut. A head had no limit, and took a
time that grew with the square of its size (an 8 MB header, seconds). A
broken response answered nothing.
if(kw_get_int(gobj, kw, "body_too_large", 0, 0) > 0 ||
kw_get_int(gobj, kw, "headers_too_large", 0, 0) > 0 ||
kw_has_key(kw, "protocol_error")) {
/* refuse the answer: it was not read */
}C_PROT_HTTP_SR¶
HTTP server protocol — parses incoming HTTP requests and produces structured events for the application.
| Property | Value |
|---|---|
| States | ST_STOPPED, ST_DISCONNECTED, ST_CONNECTED |
| Input events | EV_SEND_MESSAGE, EV_CONNECTED, EV_DISCONNECTED, EV_RX_DATA, EV_TX_READY |
| Output events | EV_ON_MESSAGE, EV_ON_HEADER, EV_ON_OPEN, EV_ON_CLOSE |
Key attributes¶
| Attribute | Type | Description |
|---|---|---|
timeout_inactivity | integer | Inactivity timeout in seconds. |
max_header_size | integer | Longest head of a request (target, header names and values, and 64 bytes for each header), in bytes (default 65536, and 0 is that too. Never more than half the max block). |
max_body_size | integer | Longest body of a request, in bytes (default 0: half the max block. A bigger value is capped to that half). |
max_tx_queue | integer | Answers owed (not sent yet by the application, or waiting in the transport) from which what the peer sends is held, not parsed (default 64, 0: no limit). See A peer that does not read below. |
timeout_tx_stall | integer | Milliseconds with max_tx_queue answers waiting in the transport and no write seen to end (default 30000, 0: never). See A peer that does not read below. |
max_rx_held | integer | Most bytes held behind max_tx_queue answers owed (default 1048576, 0: no limit). |
timeout_linger | integer | Milliseconds a refused request has to go after its answer (default 2000, 0: dropped at once). |
subscriber | pointer | Gobj receiving output events. |
A negative limit is taken as its default, with an ERROR at the creation (“A limit of C_PROT_HTTP_SR is negative: its default taken”). The timers of this gclass are C_TIMERs, accurate to the second: timeout_linger is a minimum, and can last one second more. timeout_tx_stall is a minimum too, and can last about two seconds more: a write ends with no event, so the wait looks at the bytes written every second, and it can see the last write one second late and end one second late. With timeout_tx_stall 3000 the connection is dropped from 3 to 5 seconds after the last write.
A request refused. A request whose head passes max_header_size is
answered 431 Request Header Fields Too Large, one whose body passes
max_body_size 413 Content Too Large (both with Connection: close and no
body), with ONE warning (“HTTP headers larger than their limit, message
refused”, “HTTP body larger than its limit, message refused”), and no
EV_ON_MESSAGE. HTTP answers go in the order of the requests, so with an
application that answers later the refusal waits for the answers of the
requests before it (it goes behind the last one, and an application that never
answers leaves it to timeout_inactivity). The connection then lingers
timeout_linger milliseconds from the refusal, the bytes the peer still
sends read and thrown away, and is dropped: a socket closed with bytes unread
resets the connection, and the peer can lose the answer. A peer that keeps
sending does not lengthen the wait. A peer that breaks the protocol is dropped
at once, with ONE warning. Up to 7.29.3 a head had no limit (an 8 MB header
took seconds, the time growing with the square of its size), and a refused
body was dropped with no answer.
A peer that does not read. The queue of the transport moves only when a
write ends, in the event loop, so while one read is parsed no answer leaves,
whether the peer reads or not: 100 requests pipelined in one send put 100
answers in the queue of a peer that is reading them. Its length says nothing
of the peer, and it decides only this: with max_tx_queue answers owed,
those waiting in the transport and those the application has not sent yet,
what the peer sends is held, not parsed, and it is parsed read by read
when there is room again. So a pipeline of any length is served, in its order,
in pieces, and the answers owed never pass max_tx_queue by more than what
one read holds, however late the application answers. A peer is dropped, with
ONE warning, in two cases:
No write ended in
timeout_tx_stallmilliseconds withmax_tx_queueanswers in the transport (“HTTP peer does not read its answers, connection dropped”). Any write that ends starts the wait again. The timer watches it, so a peer that went silent is dropped too. An application that takes its time says nothing of the peer: answers it has not sent do not count here.What is held passes
max_rx_heldbytes (“HTTP requests held behind the answers owed passed max_rx_held, connection dropped”). This one looks at the bytes, not at the peer, and the warning does not say who is slow: a peer that pipelines more thanmax_rx_heldbytes ahead of its answers is dropped even if it reads them, and so is one whose application takes its time, because the bytes have to be somewhere. The warning carriescur_tx_queue, the answers waiting in the transport:0means that the application is the slow one. Raise it for a client that does that.
A peer that reads slowly can count as one that does not read. A write ends
when the kernel has taken the last of it, and the kernel wakes a blocked write
only when a large part of the send buffer is free. So a peer that reads, but
slower than about send buffer / (2 x timeout_tx_stall), with
max_tx_queue answers waiting, ends no write in that time and is dropped.
Linux grows the send buffer up to net.ipv4.tcp_wmem, 4 MB by default, so the
minimum speed is about:
timeout_tx_stall | Minimum speed with a send buffer of 4 MB |
|---|---|
30000 (default) | 4 MB / (2 x 30 s) = 68 KB/s, about 550 kbit/s |
120000 | 17 KB/s |
0 | no minimum: only timeout_inactivity and max_rx_held end the connection |
An application that answers later gets at most max_tx_queue requests ahead
of its answers, plus what one read holds: the next requests wait until it has
answered. So it must answer every request it gets, with EV_SEND_MESSAGE: one
that leaves max_tx_queue requests of a connection without an answer holds that
connection until timeout_inactivity. Up to 7.29.3 nothing was held. And if
the requests held pass max_rx_held while it takes its time, the peer is
dropped: size max_rx_held for that as well. The answer it sends to a peer
that stopped reading looks at the same wait.
/* a server for peers that may pipeline up to 8 MB of requests, and an
application that answers at most 64 of them at a time */
{
"name": "http-1",
"gclass": "C_PROT_HTTP_SR",
"kw": {
"max_tx_queue": 64,
"timeout_tx_stall": 10000,
"max_rx_held": 8388608
},
"children": [{"name": "http-1", "gclass": "C_TCP"}]
}A connection dropped. Once the application drops the connection
(EV_DROP), or it closes, nothing more of what was received is delivered,
not even the requests pipelined in the same read after the one it dropped
at, even with a write in flight, which makes the disconnection come later.
Up to 7.29.3 they were delivered to the gobj already disconnected: an ERROR
with a stack (“Event NOT DEFINED in state”) for each one.
An answer that cannot be built. HTTP answers go in the order of the
requests, so an answer that cannot be built -- a body that cannot be dumped
(a string that is not UTF-8, a body over the max block), or an answer too big
for a gbuffer -- leaves a 500 Internal Server Error in its place, with
Content-Length: 0: the action logs an ERROR (“An answer cannot be built (a
body that cannot be dumped, or no memory), answered 500 in its place”) and
answers -1, and the next answer still goes to the next request. If not even
the 500 can be built, the connection is dropped. Up to 7.30.14, a body that
the action failed to dump sent nothing and was counted as sent all the same, so the
peer took the next answer as the answer of that request. An answer too big for
a gbuffer crashed the yuno.
json_t *kw = json_pack("{s:s, s:s, s:o}",
"code", "200 OK",
"headers", "",
"body", json_stringn_nocheck("\xff", 1) // not UTF-8: cannot be dumped
);
if(gobj_send_event(channel, EV_SEND_MESSAGE, kw, gobj) < 0) {
/* the peer got a 500 for this request */
}/* a server of 16 KB heads, 1 MB bodies */
{
"name": "http-1",
"gclass": "C_PROT_HTTP_SR",
"kw": {
"max_header_size": 16384,
"max_body_size": 1048576
},
"children": [{"name": "http-1", "gclass": "C_TCP"}]
}tests/c/c_prot_http_sr (test1): 600 requests in one write to an
application that drops at the 3rd deliver 3, with a write in flight too. A
head over the limit gets 431, a body over it 413, each with one warning, also
behind an answer that comes later. A peer that reads gets 2000 pipelined
answers in order, and a slow one is not dropped. A peer that never reads is
held at max_tx_queue and dropped. With an application that answers later,
10000 pipelined requests stay under max_tx_queue plus one read, a peer that
reads gets every answer in order, and one that does not is dropped from 4 to 6
seconds after its last write with timeout_tx_stall 4000. A last answer that
brings the transport to max_tx_queue starts the wait by itself. A refused
request held behind the answers owed is answered once, after them, even when
the application answers them seconds later. A peer that keeps sending is not
dropped at timeout_inactivity. An answer that cannot be built (a body that
is not UTF-8, headers that fill the max block) leaves a 500 in its place.
C_PROT_TCP4H¶
Frame protocol with a 4-byte binary header containing the payload length. Works in both client and server modes.
| Property | Value |
|---|---|
| States | ST_DISCONNECTED, ST_WAIT_HANDSHAKE, ST_WAIT_FRAME_HEADER, ST_WAIT_PAYLOAD |
| Input events | EV_SEND_MESSAGE, EV_CONNECTED, EV_DISCONNECTED, EV_RX_DATA, EV_TX_READY, EV_DROP, EV_TIMEOUT |
| Output events | EV_ON_MESSAGE, EV_ON_OPEN, EV_ON_CLOSE |
Key attributes¶
| Attribute | Type | Description |
|---|---|---|
url | string | Connection URL. |
iamServer | bool | TRUE for server mode. |
max_pkt_size | integer | Largest payload of a frame, in bytes (default 0: what a gbuffer holds, the max block of the yuno less one byte. A bigger value is capped to that). A frame announced bigger closes the connection, with a warning (“tcp4h head too long”, with len and max). Up to 7.25.22 the default was the max block itself, and a frame of exactly that length never ended. |
timeout_handshake | integer | Handshake timeout in milliseconds (default 30000). |
timeout_payload | integer | Milliseconds to receive the rest of a frame whose header came (default 5000). When it runs out, the connection is closed. |
timeout_close | integer | Milliseconds a closing client waits for the server to drop the connection before it drops it itself (default 3000). |
cert_pem | string | TLS certificate (PEM). |
The length of a frame is the peer’s word, in the 4-byte header (big-endian, counting the header itself) before any of its payload. The payload buffer starts at 4 KB and grows with what arrives, up to what the frame needs. Up to 7.25.21, the whole length was reserved at the header, so a peer that wrote only headers announcing the max block reserved it each time, with no payload sent.
uint32_t len = htonl(payload_len + 4); // the header counts itself
gbuffer_append(gbuf, &len, sizeof(len));
gbuffer_append(gbuf, payload, payload_len);tests/c/c_prot_tcp4h (test1): a frame of 60000 bytes in three parts
arrives whole, and the header of a frame of 10 MB with no payload does not
grow the yuno’s memory.
C_PROT_RAW¶
Raw pass-through protocol — forwards data without adding or removing headers. Useful for transparent proxying.
| Property | Value |
|---|---|
| States | ST_STOPPED, ST_DISCONNECTED, ST_CONNECTED |
| Input events | EV_SEND_MESSAGE, EV_CONNECTED, EV_DISCONNECTED, EV_RX_DATA, EV_TX_READY |
| Output events | EV_ON_MESSAGE, EV_ON_OPEN, EV_ON_CLOSE |
Key attributes¶
| Attribute | Type | Description |
|---|---|---|
subscriber | pointer | Gobj receiving output events. |
C_WEBSOCKET¶
WebSocket protocol (RFC 6455) — supports both client and server roles with frame masking, ping/pong, and graceful close handshake.
| Property | Value |
|---|---|
| States | ST_DISCONNECTED, ST_WAIT_HANDSHAKE, ST_WAIT_FRAME_HEADER, ST_WAIT_PAYLOAD |
| Input events | EV_SEND_MESSAGE, EV_CONNECTED, EV_DISCONNECTED, EV_RX_DATA, EV_TX_READY, EV_DROP, EV_TIMEOUT |
| Output events | EV_ON_MESSAGE, EV_ON_OPEN, EV_ON_CLOSE |
Key attributes¶
| Attribute | Type | Description |
|---|---|---|
url | string | WebSocket URL (ws:// or wss://). |
iamServer | bool | TRUE for server mode. |
cert_pem | string | TLS certificate (PEM). |
timeout_handshake | integer | Milliseconds from the connection to the end of the handshake (default 30000), server and client. A part of the handshake does not stop it: when it runs out, a warning (“Timeout waiting websocket handshake”) and the connection is dropped. Up to 7.30.15 every read stopped it and nothing armed it again, so a peer that sent one byte (G) held its connection, and a channel of the server’s pool, for ever. |
timeout_payload | integer | Milliseconds to receive the rest of a frame whose header came (default 5000). When it runs out, the connection is closed. |
timeout_close | integer | Milliseconds a closing client waits for the server to drop the connection before it drops it itself (default 3000). |
pingT | integer | Ping interval in milliseconds (0 = disabled). |
max_payload_size | integer | Largest payload of a frame, in bytes (default 0: what a gbuffer holds, the max block of the yuno less one byte. A bigger value is capped to that -- up to 7.25.22 the default was the max block itself, and a frame of exactly that length never ended). A frame announced bigger is refused: a warning (“Websocket frame bigger than the max payload, connection closed”, with frame_length) and the connection closed with 1009. |
The length of a frame is the peer’s word, written in its header before any of
its payload. The payload buffer starts at 4 KB and grows with what arrives, up
to what the frame needs (a frame of exactly max_payload_size included: up to
7.25.21 it never ended, a gbuffer holding one byte less than its max). Up to
7.25.21, the whole length was reserved at the header, so a few connections that
each announced the max block reserved it each with a few bytes sent.
The head of the handshake (request or response) is limited to 64 KB, its
target or status text and its header names and values
(ghttp_parser): a longer one is refused with ONE
warning (“HTTP headers larger than their limit, message refused”) and the
connection is dropped. Up to 7.29.3 it had no limit.
{
"name": "input",
"gclass": "C_WEBSOCKET",
"kw": {
"iamServer": true,
"max_payload_size": 1048576,
"timeout_handshake": 10000
}
}Writing a frame¶
EV_SEND_MESSAGE takes a gbuffer and writes it as one text frame:
gbuffer_t *gbuf = gbuffer_create(256, 256);
gbuffer_append_string(gbuf, "{\"event\": \"EV_HELLO\"}");
json_t *kw = json_pack("{s:I}",
"gbuffer", (json_int_t)(uintptr_t)gbuf
);
gobj_send_event(gobj_websocket, EV_SEND_MESSAGE, kw, gobj);A server writes a frame with up to 64 KB of payload with ONE EV_TX_DATA
to its C_TCP: the header and the payload are copied into one gbuffer. A
bigger frame is written as before: its header, then the gbuffer of the payload
itself, not copied. C_TCP counts every EV_TX_DATA in txMsgs, so a server
counts one message per frame of up to 64 KB, where it counted two. Up to
7.27.7 the header always went alone, and the peer often read it alone.
A client masks every frame (RFC 6455 §5.3) with a key from getrandom(),
taken 64 keys at a time. Up to 7.27.7 the key came from srand(time(0)) and
rand() at every frame: every frame of the same second had the same key, which
the clock made predictable, and the process’s own rand() sequence was reset at
every frame. If getrandom() fails, an ERROR is logged and the frame gets a
key from a weak generator.
Closing¶
ws_close() -- a timeout, a protocol error, a received Close -- moves the
gobj to ST_DISCONNECTED at once and sends its Close frame. A server drops
the TCP connection there and then. A client keeps it, as RFC 6455 asks:
the server closes the TCP connection (7.1.1), and an endpoint that sent a
Close can go on receiving (5.5.1). The client drops it itself when
timeout_close runs out (warning “Timeout waiting websocket
disconnected”).
A frame the peer must not send closes the connection (1002) with ONE
warning (MSGSET_PROTOCOL, with opcode, fin, rsv, frame_length,
peername, and a trace dump of the payload capped at 256 bytes):
| The frame | The warning |
|---|---|
| an RSV bit set, in any frame (no extension is agreed) | “Websocket frame with RSV bits and no extension, connection closed” |
| a continuation (final or not) with no message begun | “Websocket continuation frame with no message begun, connection closed” |
| a text or binary frame (final or not) inside a fragmented message | “Websocket data frame inside a fragmented message, connection closed” |
| a control frame (ping, pong, close) not final | “Websocket control frame fragmented, connection closed” |
a text frame that is not utf-8 (closed with 1007) | “Websocket text frame not utf-8, connection closed” |
| a ping of 126 bytes or more | “Websocket control frame of 126 bytes or more, connection closed” |
| a frame header that cannot be decoded | “Websocket frame header not decoded, connection closed” |
Up to 7.30.15 these closes left no log, and the RSV bits of a frame that is
not final were not looked at. For example, a text frame hi with RSV1 set,
as a server writes it (not masked):
c1 02 68 69 FIN + RSV1 + text, length 2, "hi" -> the warning, and a Close 1002A message is never carried from one connection to the next: the fragments of a message begun and a payload half read are dropped when a connection opens and when it closes. Up to 7.30.15 a fragment begun stayed in the gobj, and on a server’s static channel the next peer that sent a final continuation was delivered the bytes of the peer before it:
peer A: 01 88 <key> "A-secret" (text, not final), then leaves
peer B: 80 81 <key> "b" (continuation, final), same channel
-> 7.30.15: "A-secretb" delivered to B's session
-> now: the continuation warning above, and a Close 1002What arrives in between -- the rest of the frame the client gave up on, the
server’s own Close -- belongs to a session that is over: it is discarded, and
nothing is published (the debug trace level logs it). Up to 7.25.10
ST_DISCONNECTED did not declare EV_RX_DATA, and each such read logged
“Event NOT DEFINED in state”. A client too busy to read a frame in time
showed it. A client configured to give a slow peer more time:
{
"name": "output",
"gclass": "C_WEBSOCKET",
"kw": {
"timeout_payload": 10000,
"timeout_close": 5000
}
}A C_WEBSOCKET server answers one subprotocol, or none (RFC 6455 §4.2.2):
the first value of Sec-WebSocket-Protocol. Nothing validates it yet. A value
that is no token, or longer than NAME_MAX, gets no subprotocol and one
warning: it is never cut.
Sec-WebSocket-Protocol: chat, superchat -> Sec-Websocket-Protocol: chat
Sec-WebSocket-Protocol: bad value, chat -> (none, and a warning)The longest value answered is NAME_MAX bytes (255): one of 256 gets none.
A Sec-WebSocket-Key that is not 24 characters (16 bytes in base64, RFC 6455
§4.2.1) is answered 400 Bad Request, with the warning “Websocket upgrade
request refused”. Up to 7.30.14, one over 64 characters was cut where its
accept is computed, and answered a wrong accept.
The handshake a client accepts¶
A C_WEBSOCKET client sends a Sec-WebSocket-Key of 16 random bytes in
base64 (from getrandom(), the generator of its masking keys. Up to 7.30.14,
it was the clock, in hex). It opens the session only with a 101 that
upgrades to websocket and proves the server read its request (RFC 6455
§4.1): Upgrade: websocket and a Connection that holds upgrade (tokens,
any case), and a Sec-WebSocket-Accept that is the SHA-1 of the key it sent
and the GUID 258EAFA5-E914-47DA-95CA-C5AB0DC85B11, in base64. A client
offers no subprotocol and asks for no extension, so a 101 that names one
is refused too. Frames the server writes behind its 101, in the same read,
are delivered (up to 7.30.14 they were dropped). A refused handshake
logs ONE warning (MSGSET_PROTOCOL, with status, accept, protocol and
a dump of the read, capped at 256 bytes), publishes no EV_ON_OPEN, and drops
the connection. The transport reconnects as after any drop. The example of
RFC 6455 §1.3:
GET /chat HTTP/1.1 HTTP/1.1 101 Switching Protocols
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== -> Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=The 101 | The client |
|---|---|
no Upgrade: websocket, or no upgrade in Connection | warning “Websocket handshake refused: no Upgrade: websocket, or no Connection: upgrade” |
no Sec-WebSocket-Accept, or a wrong one | warning “Websocket handshake refused: Sec-WebSocket-Accept missing or wrong” |
a Sec-WebSocket-Protocol | warning “Websocket handshake refused: a subprotocol that was not offered” |
a Sec-WebSocket-Extensions | warning “Websocket handshake refused: an extension that was not asked for” |
any status but 101 | warning “NO 101 HTTP Response”, with the status |
The values of the headers are read without the OWS around them (RFC 9110
§5.5, see ghttp_parser): Upgrade: websocket is
websocket. Up to 7.30.15 the trailing OWS stayed in the value, and this
client refused such a legal 101 (and a server answered 400 to a key
followed by a space).
Up to 7.30.14 the client looked only at the 101 and opened the session with
any of them. A server that answered a wrong accept worked with a Yuneta
client and failed with a browser. It now fails with both. A C_WEBSOCKET
server, and a proxy that passes the upgrade through, answer the right one.
Tests: tests/c/c_websocket (test_websocket_test1, test_websocket_test4
for the subprotocol a server answers, test_websocket_test5 for the
handshake a client refuses, test_websocket_test6 for a server’s half
handshakes, test_websocket_test7 for what a client closes for and a 101
with OWS, and test_websocket_test8 for a message that must not cross from one
connection to the next).