Events
jobs says what happens to a job with the telemetry package, as std/logger says it in text for a person. The calls of Jobs (enqueue, enqueue_all, fetch, ack, nack) emit an event after the backend answered, so every backend gets them, and Jobs.drain also runs each job in a span. Nothing is emitted for a call that failed: an event says what happened. With no handler attached a call costs a few dozen steps more (one table read, and no map is built), so a program that does not use telemetry pays nothing it can see. Once any handler is attached, even one for other events, each call costs a little more, about 100 steps for each check of whether a handler hears it.
An app attaches the handlers, after Telemetry.start() in its main, with the names it wants (["jobs"] for all of them). Use the same telemetry tag as jobs does, or the manifest has two sources for one name (PKG-SOURCE-CONFLICT).
| Event | Measurements | Metadata |
|---|---|---|
jobs.enqueue | count (jobs added), duplicate_count (jobs not added) | queue, backend |
jobs.fetch | count (jobs leased) | queue, backend |
jobs.ack | count (1) | queue, id, attempt, backend |
jobs.nack | count (1) | queue, id, attempt, backend, how (retry, release or dead) |
jobs.job.start | none | queue, id, attempt, backend, span_id |
jobs.job.stop | duration_ns | as start |
jobs.job.error | duration_ns | as start, and error_kind |
jobs.job.settle | count (1) | queue, id, attempt, backend, verdict (ack, retry or dead) |
jobs.enqueueis one event for each queue of a batch.countis the jobs that were added, andduplicate_countthe ones that were a duplicate of a unique key and so were not added: together they are the jobs that the caller offered.jobs.job.start|stop|errorandjobs.job.settlecome only fromJobs.drain: a consumer that callsfetch,ackandnackitself gets the four events of the calls, and no span of its own.- A fetch that leased nothing emits nothing: a consumer that polls would fill a handler with empty events. The event of a fetch that did lease has the number of jobs.
jobs.job.start|stop|errorare the span of the handler inJobs.drain. A handler that returns an error ends injobs.job.error, with theerror_kindof that error (jobs.discardforJobs.discard, anderrorfor a plain one). The time is the time of the handler, not of the ack.jobs.job.settleis what the drain decided after the handler:ack,retry(the job runs again) ordead(a discard, or a failure on the last attempt). It comes after thejobs.ackorjobs.nackthat carried it out.- The metadata has ids and names, never the payload or the text of an error. The
idis a string that the backend makes up, a good key for a trace and a bad label for a metric: keep it out of the labels of a reporter. The logger lines of a drain stay as they were, for people. - Events of one task arrive in order, and events of different tasks have no order between them. A handler is slow or fails without hurting the consumer (see the telemetry docs); the events it cannot take are counted by
Telemetry.dropped.
Read them in a test #
Testing.session of telemetry captures the events of a block and leaves no task running, so it fits in a test of an app. This one drains a queue and reads what jobs said.
import Jobs from "pkg/jobs/jobs"
import Telemetry from "pkg/telemetry/telemetry"
import Testing from "pkg/telemetry/testing"
def handle(job: Jobs.Job) -> Result[Void]
return Jobs.discard("not an address") if job.attempt == 1
ok()
end
def line(event: Telemetry.Event) -> String
extra = event.metadata.get("error_kind", event.metadata.get("verdict", event.metadata.get("how", "")))
return event.name if extra == ""
"#{event.name} #{extra}"
end
def drain_one(backend: Jobs.Backend, capture: Testing.Capture) -> Result[List[String]]
_ = Jobs.enqueue(backend, Jobs.NewJob(queue: "mail", payload: "nobody".bytes)).unwrap!
_ = Jobs.drain(backend, Jobs.Drain(queue: "mail"), &handle)
found = try Testing.events(capture)
ok(found.map(&line))
end
def main() -> Void
backend = Jobs.memory().unwrap!
for text in Testing.session(["jobs"], ->(capture) { drain_one(backend, capture) }).unwrap!
puts(text)
end
Jobs.close(backend, 1_000)
endjobs.enqueue
jobs.fetch
jobs.job.start
jobs.job.error jobs.discard
jobs.nack dead
jobs.job.settle dead