Local AWS in Rust, and Trainium serving without Neuron

Local AWS in Rust, and Trainium serving without Neuron

Two repos landed this week, and both are built around AWS. One lets you test AWS code without an account. The other serves LLMs on Trainium hardware with its own stack.

Developer and creative tools

aws-emulator is a local emulator of the AWS APIs, written in Rust. You can use it from the CLI or any SDK, with no AWS account. It runs one Rust crate per AWS service behind a single local endpoint. It can also be compiled into a Rust test binary, so each test gets its own isolated instance and there is no endpoint URL to set. It is self-hosted and has 3,642 stars.

Model hosting and compute

kiln is an LLM inference engine for AWS Trainium, written in Python. It replaces the managed Neuron inference layer with its own scheduler, paged KV cache, radix prefix cache and NKI kernels. It compiles every graph ahead of time on a CPU farm, so a Trainium box serves from cache. It is self-hosted, tagged Distributed Systems, and has 5 stars.