dock
dock is the part every TCP server writes again. It listens, accepts, runs your handler in a task of its own for each client, and closes the socket when the handler returns, crashes or is stopped. The server never waits for one client. It does not parse a protocol, and it does not do HTTP (that is std/http).
import Dock from "pkg/dock/dock"
def echo(conn: Dock.Conn) -> Void
while true
case Dock.recv(conn, 4096)
error(_) -> return
ok(chunk) ->
if chunk.empty?
return
end
case Dock.send(conn, chunk)
error(_) -> return
ok(_) -> nil
end
end
end
end
def main() -> Void
server = Dock.start(Dock.Config(port: 0), &echo).unwrap!
client = TCP.connect("127.0.0.1", server.port, 1000).unwrap!
TCP.send(client, "hello", 1000).unwrap!
puts(String.from_bytes(TCP.recv_bytes(client, 16, 1000).unwrap!).unwrap!)
TCP.close(client)
Dock.stop(server).unwrap!
endhelloPort 0 picks a free port, and server.port is the real one: start returns once the port is open, so there is no ready callback to wait for.
API #
Dock.start(config: Config, handler: Fn[Conn, Void]) -> Result[Server]
Dock.stop(server: Server) -> Result[Void]
Dock.wait(server: Server) -> Result[Void]
Dock.recv(conn: Conn, max_bytes: Int = 65536) -> Result[Bytes]
Dock.send(conn: Conn, bytes: Bytes) -> Result[Void]
Dock.close(conn: Conn) -> Void
Dock.hand_over(conn: Conn) -> Void
Dock.stats(server: Server) -> Stats
Dock.watch(server: Server, task: Task[R]) -> Void
Dock.on_close(config: Config, hook: Fn[Conn, Void]) -> Config
Config(port: Int, host: String = "127.0.0.1", idle_ms: Int = 31536000000, max_outbox_bytes: Int = 8388608, max_connections: Int = 65536, grace_ms: Int = 0, on_close: Optional[Fn[Conn, Void]] = none)
Conn(id: Int, peer: String)
Server(port: Int)
Stats(open: Int, accepted: Int, refused: Int, dropped: Int, crashed: Int)startlistens and returns theServer, or the error of the listen (a port in use, say). The handler runs once for each client, in its own task, and gets aConn. When it returns, the socket is closed.stopstops accepting and frees the port, then cancels the handlers and closes every socket; withgrace_msabove 0 (at most one hour) it first closes each client asclosedoes (the queued bytes are written, then the socket closes, which ends a handler that is reading) and waits up to that long for the handlers to return. A client that is still there after the grace is cancelled, and a stop with grace 0 (the default) does not wait. It returns after that, or an error if the server crashed, and a second call gives the same result.waitblocks until the server ends: a program that only serves endsmainwith it. If the listener is closed under the loop, the server ends with an error thatwaitandstopreturn.recvgives empty bytes when the client closed. It gives chunks: where a request ends is for your protocol to decide, and a buffer that grows on each chunk (buffer + chunk) is quadratic on a big request, so cap the size of a request and cut it as the bytes arrive. Ifidle_mspasses, the socket is closed. Amax_bytesoutside 1 to 1,048,576 is an error that closes nothing.sendqueuesbytesin the outbox of the client and returns at once: it never waits for the client, so a task that serves many clients (a state task that answers with theConnit got in a message) is never stuck behind a slow one. It works from any task. Each message stays whole, and messages of one connection go out in the order the calls were made, also when several tasks call it. It returns an error when the client is gone, afterclose, or when the client was dropped. Queued bytes are lost when the handler crashes, and asendof another task that races aclosemay be lost.- The cap: a message is charged its bytes plus 64, but never more than
max_outbox_bytes, and it is accepted when the charges waiting before it are at most the cap. A client that does not read is dropped (its socket closed) once more than the cap waits, and a big reply followed by a short one is fine. So a client holds in memory at most the cap plus the last message that was let in, of any size: at most two messages when they are bigger than the cap. The cap is 65 to 68,157,310. closecloses the connection after the queued bytes were written, so a reply and the close of aQUITgo out in order. When the handler returns, the same happens, and the queue has 30 seconds to be written.hand_overtells dock that another task owns the connection now, so the handler may return without closing it. Call it in the handler, before theConnis given away and before returning. A connection that was handed over ends only atclose, when arecvsees the end of the client (empty bytes) or fails (the idle time passed), when its outbox passes the cap, when a send fails, or atstop; the client still counts as open andon_closeruns once, at that end. Nobody reads the socket after the handler returned: dock keeps no reader, because it would take bytes that the new owner wants. So the end of the client is seen by a task that callsrecv, and a handed over connection that nobody reads, sends to or closes stays open untilstop(there is no idle time for it), so an owner that never reads runs its own timeout and callsDock.close. Empty bytes fromrecv(the end of the client, also a half-close) close the connection. Arecvthat gets the end or an error closes the connection asclosedoes, so the queued bytes are still written first.max_connections(1 to 1,048,576) is the most connections open at once. One that comes over it is accepted and closed at once, without a handler, so the client sees the end of the connection; it is counted asrefused. Before that the loop waits up to 50 ms for the notice of a handler that is just ending, so a client that comes right after another left is not refused; after a wait that found none it does not wait again for 100 ms, so a flood of refused connections costs the loop at most a third of its time.statsreads the counters, from any task:open(handlers that have not ended),accepted(connections that got a handler),refused(closed overmax_connections),dropped(clients closed for passing the outbox cap) andcrashed(handlers that crashed).on_closeruns once for each connection that got a handler and ended: the handler returned, crashed, orstopcancelled it (for a handed over connection, the end is the one listed underhand_over). It gets theConnin a task of its own, after the socket is closed. The server only counts the callbacks that run, it does not wait for one while serving, so slow callbacks never hold the accept loop. A callback that crashes prints its diagnostic and does not affect the server. To tell another task, a callback may wait, so it retries itssenduntil the channel takes the notice (the example below): asendwith a timeout that fails is a notice lost. A stop waits up to one second for the callbacks that still run and then returns: those are left running, not cancelled, and a crash of a handler during the stop is not counted incrashed. A field of a struct cannot hold a block, soDock.on_close(config) do |conn| ... endsets it with a block that captures what it needs, as in the example below;Config(on_close: some(&f))takes a function.watchties the server to a task of yours, such as the one that owns the state: when that task ends, the server stops asstopdoes, within about 100 ms, andwaitandstopreturnerror("dock: the watched task crashed: ...")for a crash (for a cancel the error says it was cancelled; a task that returns normally also stops the server, andwaitreturns ok). It may be called for more than one task.ConnandServerare read-only values that dock gives you (startreturns aServer, the handler gets aConn): do not build one. They have more fields, which are private, so use the functions above, and do not callTCPon a client.- A crash of a handler prints its diagnostic and stack on the stderr of the VM, closes the socket, and the server goes on. A handler that wants to log its own errors handles the
Results it gets. - A handler is a function, or a block that captures what it needs. A server with one task that owns the state passes it a channel. That channel holds a number of messages, not bytes, so bound what a client can put in it: cap the size of a request that the handler reads, and do not read the next one while the channel is full (
sendwith a timeout of 0 tells it).
A server with one task that owns the state (a store) gets a connection from the handler with hand_over, tells it things later, and learns of its end with on_close; the server ends when the store crashes:
import Dock from "pkg/dock/dock"
enum Msg
Join(Dock.Conn)
Gone(Int)
end
# The handler gives the connection to the store and returns at once; without `hand_over` the socket would close here.
def serve(conn: Dock.Conn, inbox: Channel[Msg]) -> Void
Dock.hand_over(conn)
_ = inbox.send(Msg.Join(conn), 1000)
end
# The one task that owns the data: it keeps the connections it was given and writes to them when it likes.
def store(inbox: Channel[Msg]) -> Void
members: Map[Int, Dock.Conn] = {}
while true
case inbox.recv(60_000)
error(_) -> return
ok(msg) ->
case msg
Msg.Gone(id) -> members = members.delete(id)
Msg.Join(conn) ->
for other in members.values
_ = Dock.send(other, "joined #{conn.id}\r\n".bytes)
end
members = members.put(conn.id, conn)
end
end
end
end
# `on_close` runs in a task of its own, so it may wait: a notice that is dropped would leave the session in the store for
# ever. Retry until the inbox takes it or is closed.
def tell(inbox: Channel[Msg], msg: Msg) -> Void
while true
case inbox.send(msg, 1000)
ok(_) -> return
error(_) ->
if inbox.closed?
return
end
end
end
end
def main() -> Void
inbox: Channel[Msg] = Channel.new(1024)
keeper = Task.spawn(&store, inbox)
config = Dock.on_close(Dock.Config(port: 6380, max_connections: 10_000, grace_ms: 2000)) do |conn|
tell(inbox, Msg.Gone(conn.id))
end
started = Dock.start(config) do |conn| serve(conn, inbox) end
server = started.unwrap!
Dock.watch(server, keeper)
Dock.wait(server).unwrap!
endThe store here never reads the sockets, so it sees a client leave only when a send fails (Gone then comes from on_close). A store that wants the end at once gives each connection a reader task (Dock.recv in a loop, which closes the connection at the end of the client), or the handler keeps reading and does not hand over.
Why it is built this way #
- One task per client, started by a supervisor task that owns the listener. The supervisor never calls anything that waits for a client: it only accepts (100 ms at most), starts tasks and closes sockets.
- The supervisor owns the sockets, not the client tasks, so that it can close one when its handler crashed or was cancelled (a crash does not run the handler's last lines). It learns of the end of every task with
Task.monitor. The task closes the socket itself when the handler returns and the queue is written, so the client sees the end at once. A close is idempotent, so the second one does nothing. A lost notice would keep a socket until the server stops, and a full channel drops notices, so the channel holdsmax_connectionsnotices: the server never has more live tasks than that, which is also why a refused connection gets no task. The supervisor empties it between accepts. - Each client has an outbox and a writer task.
sendadds the bytes to a counter of the queued bytes and puts the message in a channel with a timeout of 0, so it never waits. The writer takes the messages that wait (up to 256 and about 1 MiB) and writes them in one send, which keeps a client that pipelines a million small replies from being dropped. It subtracts after each send, so the counter covers the send in progress. The channel holdsmax_outbox_bytes / 65 + 2messages, so the cap, not a full channel, is what drops a client. A write that takes 5 seconds closes the socket. Task.spawn_owningwould give the socket to the task, but it cannot also pass a shared channel (agentica#1341), which a handler that talks to a state task needs.- A receive that times out closes the socket, and there is no wait-forever form (agentica#1342). So
idle_msis a value of theConfig, one year by default. - A stop closes the listener first, so the port is free during the grace. With a grace the supervisor never blocks: it waits for the notices of the tasks (50 ms at a time) until they are all in or the time is over.
watchis aTask.monitorof your task into a channel that the supervisor reads on each turn of the accept loop (there is noselect), so it reacts within one accept timeout, 100 ms.- A timeout of an accept adds no delay to a client that comes next. An accept that fails at once (no free file descriptor) waits the rest of the 100 ms before the next try, so it cannot spin. A closed listener, which fails at once for ever, ends the server with an error.
Planned #
Tracked in #71: the first release, dock-0.1.0.