[ikrommyd]

🤖 AI text below 🤖

Makes float16, long double (float128) and long double complex (complex256) work in everything awkward does itself, and raise an error that names the type wherever another library has no such type. Before this, every reducer and sort on these types raised a KeyError from the kernel lookup (scikit-hep#4259), and a few other places crashed.

What works now

• Reducers, sorting, unique. The 22 kernels that take floating point data get a long double version (25 template instantiations, with matching entries in kernel-specification.yml). float16 data are given to the float32 kernels and the result is cast back. NumPy computes float16 the same way: it adds and multiplies in float32 and rounds once at the end.
• Type strings. ak.types.from_datashape and ak.enforce_type parse float16, float128 and complex256, so a type can read back what it prints.
• C++. Long double arrays go to and from cppyy and RDataFrame, and a LayoutBuilder::Numpy<long double> names its primitive the way NumPy does.
• ak.covar and ak.corr keep long double, as np.cov and np.corrcoef do. Narrower floats still become float64.
• Printing. formatter={"float": ...} applies to float16. The long double formatters apply on platforms where NumPy has no float128 name.
• ak.from_iter and ArrayBuilder take np.clongdouble scalars (segfault on Linux and RecursionError on macOS before).
• ak.to_json writes long double (TypeError before).
• ak.min and ak.max with a long double initial keep its digits. ctypes would otherwise round the value to a double on its way into the kernel.
• Unique values of complex numbers without an axis, through NumPy. This is what ak.validity_error uses for categorical data.

What raises

Operation	Type	Error
Reading a value in a Numba-compiled function	float16	TypeError. Passing the array through, slicing it and reading other fields still work.
Passing an array to a Numba-compiled function	long double	TypeError
cppyy, ak.to_rdataframe	float16	TypeError
ak.to_arrow, ak.to_parquet, ak.to_feather	long double	TypeError
ak.sort, ak.argsort, unique along an axis	complex	ValueError
unique along an axis	date-time	ValueError

Each message for a missing type says how to convert the data first. Numba, Arrow and C++ have no such types. Sorting complex numbers has never had kernels, and the unique kernels sort.

Decisions to check

• ArrayBuilder and JSON hold doubles. The builder has one real and one complex node type (float64, complex128), which is also why np.float32 scalars come out as float64. Python's json module writes Python floats. So a long double scalar or value is rounded to the nearest double there, on platforms where long double is wider. Keeping long double in the builder needs new node types across the C++ builder classes, which this does not add.
• The Arrow error is general. It replaces pyarrow's ArrowNotImplementedError: Unsupported numpy type 15 for complex64 and complex128 too. Where long double is the same as double (macOS arm64, Windows) it converts.
• awkward-cpp changes without a version bump. It stays at 58.
• The type parser is edited by hand. It is generated by Lark, and the grammar is not in this repository. I changed the one DTYPE pattern to exactly what Lark 0.12.0 produces from upstream's grammar with the three names added.
• A float16 total that overflows warns. The cast back to float16 emits NumPy's RuntimeWarning, as np.sum does. The float32 and float64 kernels stay silent, as before.
• Not changed: ak.to_torch, ak.to_tensorflow, ak.to_jax and ak.to_cupy. Those libraries report unsupported types themselves.
• Long double differs by platform. Data written as float128 on Linux cannot be read where NumPy has no such type. Nothing here changes that.

Tests

tests/test_4259_float16_and_longdouble.py covers each area above, and header-only/tests/test_4259-layout-builder-long-double.cpp covers the C++ naming. Run locally on three platforms:

Platform	long double	tests/	kernel suites	header-only
macOS arm64	same as double	5,596 passed, 49 skipped	22,351 passed, 628 skipped	7 of 7
Linux x86-64	80-bit	5,632 passed, 49 skipped	22,351 passed, 628 skipped	7 of 7
Linux aarch64 with ROOT 6.40 and cppyy	128-bit	5,803 passed, 27 skipped	22,351 passed, 628 skipped	7 of 7

Downstream suites against this branch (pytest tests, macOS arm64, Python 3.14):

Package	Result
dask-awkward	640 passed, 4 skipped, 22 xfailed
dask-histogram	83 passed, 5 xfailed
uproot5	1,211 passed, 81 skipped
vector	4,926 passed, 14 skipped, 1 failed
coffea	1,383 passed, 48 skipped, 2 xfailed

Byte order. There is no big-endian CI job, so the diff was checked by inspection. The kernels only do typed arithmetic and comparisons, and the Python side converts values (float16 to float32, long double to float64) or reinterprets data only between types of the same size and native order. awkward accepts arrays in native byte order only, and buffers written and read in the opposite order round-trip for all three types, with each part of a complex number swapped on its own.

The vector failure is test_issue_621, which calls ak.jax.register_and_check(). This repository has no JAX backend, so it does not depend on this change.