elelem-llama2

An elelem provider plugin backed by karpathy/llama2.c – vendored as a git submodule at vendor/llama2.c, same pattern as elelem-llama’s vendor/llama.cpp.

What this actually is

llama2.c is a ~1000-line single-file C implementation of Llama 2 inference, with no build system, no GGUF support, and no chat templates/tool-calling. It only understands its own .bin checkpoint format (produced by vendor/llama2.c/export.py) and the original Llama 2 architecture – typically small models trained on TinyStories, or a converted Meta Llama-2 checkpoint. This is meant as a learning-focused counterpart to elelem-llama: much smaller surface area, much simpler build (ext/elelem/llama2/extconf.rb just runs cc/c++ directly, no CMake).

Unlike elelem-ggml, this one does real inference, not just a scaffold: ext/elelem/llama2/elelem.cpp reuses run.c’s build_transformer/forward/ encode/decode/sample functions in a proper streaming generate loop. What’s still missing versus elelem-llama: chat templates, tool-calling, and any quantization beyond what upstream’s runq.c offers (not yet wired in here – see TODOs).

Setup

git submodule update --init --recursive
bundle install
bundle exec rake compile

You’ll need a llama2.c-format checkpoint + tokenizer. Easiest way to get one: follow the “feel the magic” section of https://github.com/karpathy/llama2.c#readme to download stories15M.bin (a tiny TinyStories model), and use the vendored tokenizer.bin.

Usage

export ELELEM_LLAMA2_CHECKPOINT=/path/to/stories15M.bin
bundle exec exe/elelem-llama2 ask "Once upon a time"