Skip to content

Return aggregates from extern functions through an output pointer - #75

Merged
wtholliday merged 2 commits into
mainfrom
extern-by-pointer-returns
Oct 18, 2026
Merged

wtholliday merged 2 commits into
mainfrom
extern-by-pointer-returns

Conversation

@wtholliday

@wtholliday wtholliday commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

An extern that returns a tuple, struct, array, enum or f32x4 now receives a trailing output pointer and returns void:

extern fn host_pair(x: i32) -> (i32, i32)
void host_pair(void* ctx, int32_t x, int32_t* out);
  • One ABI rule, every backend. TypeID::extern_returns_via_pointer in src/types.rs is the single definition; Cranelift, LLVM, VM and stack codegen all use it. Previously each backend had its own predicate and they had already drifted (f32x4 was by-value on Cranelift/LLVM and by-pointer on VM/stack, so no one host function worked everywhere).
  • f32x4 goes through the pointer as four floats on all backends. Vector return registers are not portable and libffi cannot marshal them. Cranelift and LLVM reload the vector after the call.
  • Checker validates extern signatures. Return types no backend can marshal (function types, i8/u8/u32) and parameter types outside bool, i32, f32, f64, slices and references are rejected at check time with a message listing the supported set. Previously a struct or f32x4 parameter compiled on the JIT but panicked in VM codegen.
  • Output slot is 16-byte aligned on Cranelift and LLVM, since the host stores through it as int32_t* / float* / double*.
  • VM codegen checks the libffi argument budget at compile time instead of panicking on the first call.
  • lyte.h documents the rule, including supported parameter types, enums (int32_t* out) and f32x4 (float* out).

Tests

  • compiler.rs: one shared runner per backend and assert_extern_all_backends; tuple, struct, array, enum and f32x4 each run the identical program and host function on VM, stack, Cranelift and LLVM.
  • New golden tests tests/cases/checker/extern_fn_ret_type.lyte and extern_fn_param_type.lyte for the checker errors; tests/cases/extern_tuple_ret.lyte covers all five by-pointer kinds in check mode.
  • cargo test --workspace passes with default features, and cargo test -p lyte -p lyte-cli --features llvm passes against LLVM 19.1.7.

🤖 Generated with Claude Code

wtholliday and others added 2 commits October 18, 2026 08:35
An extern that returns a tuple, struct, array, enum or f32x4 now receives
a trailing output pointer and returns void, so a host can hand back
compound results without a platform-specific struct-return ABI:

    extern fn host_pair(x: i32) -> (i32, i32)
    void host_pair(void* ctx, int32_t x, int32_t* out)

The rule lives in one place, TypeID::extern_returns_via_pointer, and every
backend consults it, so the same host function works on the Cranelift,
LLVM, VM and stack backends. f32x4 goes through the pointer too (four
floats) rather than a vector return register, which libffi cannot marshal
and which differs across platforms; Cranelift and LLVM reload the vector
after the call.

The checker rejects extern return types no backend can marshal (function
types, i8/u8/u32) with a message listing the supported set. The output
slot is 16-byte aligned on Cranelift and LLVM, since the host stores
through it as int32_t*/float*/double*. VM codegen checks the libffi
argument budget at compile time instead of panicking on the first call.

Tests run the same program and host function on every enabled backend for
tuple, struct, array, enum and f32x4 results, plus a golden test for the
checker error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The VM's libffi path handles bool, i32, f32, f64, slices and references
and panics in codegen on anything else, while the JIT silently passed a
struct or f32x4 parameter as a pointer. Check the parameter types in the
type checker so the same program is accepted or rejected everywhere, and
document the supported set in lyte.h.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@wtholliday
wtholliday merged commit 206f7a2 into main Oct 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant