Native modules
A native module is C code written against the VM’s C API, the header agentica_native.h. An extern module "NAME" block declares its functions with Agentica types, and the program calls them like any def. One call can take and return lists, maps, optionals, results and the program’s own structs and enums, such as all the rows of a query, where an extern block needs a call per C value.
struct Summary
count: Int
mean: Float
end
enum Cell
Empty
Number(Float)
Text(String)
end
extern module "stats"
## The count and mean of the values; none for no values.
def summary(values: List[Float]) -> Optional[Summary]
## The lines of a CSV text after its header, as maps from the header's names to typed cells.
def rows(text: String) -> Result[List[Map[String, Cell]]]
def empty?(text: String) -> Bool
end
def main() -> Void
puts(summary([2.0, 4.0, 9.0]))
puts(rows("name,qty\nkiwi,3\n"))
endThe block #
NAMEis a library as in anexternblock: a name without/, an absolute path, or the[native.NAME]of the package’sagentica.toml.- Each line is a
deforcompute defwith no body, named as the module’s table names it,?and!included, or aresource Name.##docs work as for anydef, and a resource line takes the place of adefin the declaration order. The block is public as a whole, or private as a whole afterprivate. - These types cross:
Int,Float,Bool,String,Bytes,Voidas the result, aList,OptionalorResultof them, aMapwithInt,StringorByteskeys, aliases of these, and the structs and enums of the program that are not generic, whose fields and payloads cross too. A resource of the block crosses as the whole type of a parameter, and anywhere in a result but in a field or payload. Any other type isCHECK-NATIVE-TYPEat it, and so are two such types or resources of one bare name in one block (A.RowandB.Row): the module knows its types by that name. - The table names the C function and its types, so a
callbackorstructline, afree:after a resource, anoutparameter, a...and a C name after=arePARSE-EXTERN-MODULE, and aC.type isPARSE-EXTERN-TYPE. - A
defis a short call. Acompute defis a dirty call: it runs on the compute pool while the task waits, as acompute defof anexternblock does, and may block. A task cancelled during one does not stop it: the call runs to its end and its result is dropped. - A resource is a handle to C state that the module keeps, such as a connection. It belongs to the task that made it. The module’s free function frees it once: at
Native.close, at the end of that task, or at the end of the run, newest first. A call that holds it as an argument keeps it alive until the call returns, whatever closes it meanwhile.Native.closed?tells whether it is closed, and a call with a closed one fails withRUNTIME-NATIVE-CLOSED. - Any
--deny-*flag or--sandboxrefuses a program with a module block before it runs (RUNTIME-NATIVE-DENIED, exit 2), with or without a cached program, and nothing is built: abuildtable starts no compiler, andagentica compileunder such a flag refuses the module (PKG-NATIVE-BUILD).
Binding #
The first call of a function of the block opens the library, as an extern block does, and calls its agentica_native_init once per run. A library that does not open, has no entry point, or was built for another major version of the API (or a newer minor one) fails that call with RUNTIME-NATIVE-LOAD. Then the VM looks the function up by name in the module’s table. It must have the same signature, the same mode, and each struct or enum with the same name, fields or variants and types; otherwise the call fails with RUNTIME-NATIVE-SIGNATURE. A signature is written canonically: aliases expanded, user types by their bare names, ", " between parameters and type arguments, then " -> " and the result, as in (List[Float]) -> Optional[Summary]. A struct’s shape is "count: Int, mean: Float", an enum’s "Empty | Number(Float) | Text(String)", in declared order.
The C side #
The native-gen package makes a C header from the block: typed prototypes, argument checks and the function and type tables. The C file then holds plain C functions. This section is the API under it.
#include "agentica_native.h"
enum { SUMMARY };
static an_term summary(an_env *env, int argc, const an_term *argv) {
an_list_cursor c;
an_term item;
size_t n;
double total = 0, x;
(void)argc;
if (!an_list_cursor_begin(env, argv[0], &c, &n)) return an_badarg(env, 0);
if (n == 0) return an_none(env);
while (an_list_next(env, &c, &item)) {
if (!an_get_float(env, item, &x)) return an_badarg(env, 0);
total += x;
}
an_term fields[2] = {an_int(env, (int64_t)n), an_float(env, total / n)};
return an_some(env, an_struct(env, SUMMARY, fields));
}
static const an_type types[] = {{"Summary", AN_STRUCT, "count: Int, mean: Float"}};
static const an_function functions[] = {
{"summary", "(List[Float]) -> Optional[Summary]", summary, AN_SHORT},
};
AN_MODULE_TYPES("stats", functions, types, NULL)AN_MODULE(NAME, FUNCTIONS, LOAD)defines the entry point of a module without types,AN_MODULE_TYPES(NAME, FUNCTIONS, TYPES, LOAD)of one with them, andAN_MODULE_TABLES(NAME, FUNCTIONS, FUNCTION_COUNT, TYPES, TYPE_COUNT, RESOURCES, RESOURCE_COUNT, LOAD)of one with resources (NULL, 0for a table it does not have).LOADisNULLor a function that runs once after the entry point and returnsNULLor a message that refuses the module.- A row of the function table is the name, the canonical signature, the C function and its mode,
AN_SHORT(orAN_DIRTYfor acompute def). A row of the type table is the bare name,AN_STRUCTorAN_ENUM, and the shape; its index is the type thatan_struct,an_get_field,an_enum,an_get_variantandan_get_payloadtake, and fields, variants and payloads count from 0. Two rows of one name refuse the module (RUNTIME-NATIVE-LOAD). - Values are opaque terms. Every getter (
an_get_int,an_get_string,an_list_next,an_map_get,an_get_optional,an_get_result,an_get_field,an_get_variant,an_get_payloadand the others) returnsfalsefor a term of another type, or a field or payload out of range. The data of a String or Bytes is borrowed and read-only: copy it to keep it. - A term lasts until the function returns: keep it in C locals, never in static data,
mallocmemory or across calls. an_raise(env, message)andan_badarg(env, index)fail the call, and so does a constructor that gets bad input (text that is not UTF-8, a Map key that is not anInt,StringorBytes, a type the signature does not reach). The call then fails withRUNTIME-NATIVE-ERRORat thedefline once the function returns, whatever it returns; the call site is in the stack. A result of another type than the signature’s isRUNTIME-NATIVE-ERRORtoo.- A row of the resource table is the name, the size of its C state, and the function that frees the state (or
NULL); its index is the type ofan_resource_newandan_get_resource.an_resource_new(env, type, &term)makes a resource and gives its zeroed state;an_get_resource(env, term, type, &state)gives the state of a resource argument of the call. The state holds C data only, never a term: a per-connection cache of prepared statements lives there, and the free of the connection finalizes them. The free function gets the state alone, and the VM frees the memory after it. - A dirty function reads its arguments in place. An Int, Float, Bool or Void it makes,
noneand a variant without a payload are values at once, and the getters read them. Any other term its constructors make (a String, Bytes, List, Map,some,ok,error, struct, variant with a payload or resource) is kept apart and becomes a value once the call returns, so a getter givesfalsefor it. - A short function runs on the scheduler of the calling task, at the same time as the calls of other schedulers, so it must be thread-safe (or the program guards it with
Native.mutex()). It may allocate and build large results, but must not block, sleep, wait on I/O or run longer than about 100 microseconds. A callback of an extern block that it starts runs nothing and gives zero. an_blocking(env, fn, arg)(API 1.3) runsfn(arg)where the collector does not stop the thread: use it around a C library call that switches to a stack of its own, such as a_startor_contcall of MariaDB Connector/C’s non-blocking API. It works in short and dirty functions alike.fnmay use the module’s own memory and the state of a resource thatan_get_resourceoran_resource_newgave in the call, and read the String and Bytes data of the call’s arguments that a getter gave before; nothing else of the VM: no term, no write to GC memory, no API call. The argument data lasts until the call returns: data that a C library keeps across calls (the query of a_startthat a later_contfinishes) is copied into the resource state or othermallocmemory. In a short function it still holds up the scheduler, so a call that waits belongs in acompute def.an_error(env, message, len)makes an error of kinderror.an_error_kind(env, kind, message, len, keys, values, n)(API 1.4) makeserror(message, kind: kind, details: {keys[k]: values[k]})withndetails (NULL, NULL, 0for none);kindis a dotted name such asdb.constraint, and the kind, keys and values are NUL terminated. A kind that is not a dotted name, aNULLwhere text is needed or text that is not UTF-8 fails the call.an_get_error(env, term, &kind, &message, &details)gives the kind and message (Strings) and the details (aMap[String, String]) of aResultthat is an error,falseforok(...)or another term; passNULLfor a part it does not need.an_get_resultstill gives the message of an error as a String, so a module built for API 1.3 keeps working. A module built with the header of API 1.4 needs agentica 0.33.0 or later; an older one refuses it withRUNTIME-NATIVE-LOAD.- The module links nothing of
agentica: every call goes through a table that the VM passes in. Abuildtable in[native.NAME]ofagentica.toml(build = { sources = ["native/stats.c"] }) has a run build it with the system C compiler and the header of the running binary (see packages). Withoptional = truebeside it, a module that cannot be built is a warning and the program runs without it, so the package checksNative.load("stats")before it calls the module. To build it yourself: on Linuxcc -std=c11 -shared -fPIC -o libstats.so stats.c, on macOS-dynamiclib, on Windows-shared(a DLL), with theinclude/agentica_native.hof the release archive of the version that runs the module. - A crash in a module ends the program, as a crash in an extern library does.
- The profile (
AGENTICA_VM_TRACE=profile) has anativerow per module function that ran: its calls, the microseconds in C, and its mode.