[{"data":1,"prerenderedAt":29},["ShallowReactive",2],{"chapter:teenygrad\u002Fcli-and-aot\u002Fteeny-cli.json":3},{"project":4,"route":5,"title":6,"titleHtml":7,"navTitle":6,"part":8,"sourcePath":9,"editUrl":10,"html":11,"toc":12,"hasMermaid":20,"prev":21,"next":25},"teenygrad","\u002Fteenygrad\u002Fcli-and-aot\u002Fteeny-cli","teeny-cli and Ahead-of-Time Compilation","\u003Ccode>teeny-cli\u003C\u002Fcode> and Ahead-of-Time Compilation","CLI & AOT Compilation","cli-and-aot\u002Fteeny-cli.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fteenygrad\u002Fsrc\u002Fcli-and-aot\u002Fteeny-cli.md","\u003Cp>\u003Ca href=\"\u002Fapi\u002Fteenygrad\u002Fteenygrad\u002Fteeny_cli\u002F\">\u003Ccode>teeny-cli\u003C\u002Fcode>\u003C\u002Fa> provides ahead-of-time (AOT) kernel\ncompilation as both a reusable library and a runnable reference binary.\u003C\u002Fp>\n\u003Ch2 id=\"as-a-library\">As a library\u003C\u002Fh2>\n\u003Cp>Flatten \u003Ccode>teeny_cli::AotArgs\u003C\u002Fcode> (a \u003Ccode>clap\u003C\u002Fcode>-derived struct) into your own binary’s CLI, and call\n\u003Ccode>teeny_cli::aot_compile\u003C\u002Fcode>:\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">#[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">derive\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">clap\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Parser\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">struct\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Cli\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    #[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">command\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">flatten\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    aot\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> teeny_cli\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">AotArgs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>This is the pattern \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fspinorml\u002Fvision-rs\" target=\"_blank\" rel=\"noopener noreferrer\">\u003Ccode>vision-rs\u003C\u002Fcode>\u003C\u002Fa> uses in its own binaries\n— see \u003Ccode>teeny-cli\u003C\u002Fcode>’s own \u003Ccode>main.rs\u003C\u002Fcode> for the full reference implementation (AOT-compiling\n\u003Ccode>teeny-vision\u003C\u002Fcode>’s LeNet-5).\u003C\u002Fp>\n\u003Cp>\u003Ccode>aot_compile\u003C\u002Fcode> traces your model with a symbolic input (see\n\u003Ca href=\"\u002Fteenygrad\u002Fcore-concepts\u002Ftensors-and-graph#tracing\">Tensors & the Computational Graph\u003C\u002Fa>), then compiles\nevery kernel the traced graph references for the backend\u002Fconfig selected by \u003Ccode>--device\u003C\u002Fcode>. Currently\nonly \u003Ccode>--device cuda\u003C\u002Fcode> is implemented; adding a new backend means adding a match arm in\n\u003Ccode>aot_compile\u003C\u002Fcode>, not changing \u003Ccode>AotArgs\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Ch2 id=\"requires-teenyc-at-runtime\">Requires \u003Ccode>teenyc\u003C\u002Fcode> at runtime\u003C\u002Fh2>\n\u003Cp>Like the rest of the compilation pipeline, this needs \u003Ccode>TEENYC_PATH\u003C\u002Fcode> set — see\n\u003Ca href=\"\u002Fteenygrad\u002Fgetting-started\u002Finstallation\">Installation & Toolchain\u003C\u002Fa>. It’s typically driven via\n\u003Ccode>cargo teeny aot\u003C\u002Fcode> (from \u003Ccode>cargo-teeny\u003C\u002Fcode>) rather than run directly.\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> aot\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --bin\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny-cli\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --device\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> cuda\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --options\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> \"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">capability=sm_90\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n",[13,17],{"id":14,"text":15,"level":16},"as-a-library","As a library",2,{"id":18,"text":19,"level":16},"requires-teenyc-at-runtime","Requires teenyc at runtime",false,{"title":22,"titleHtml":23,"route":24},"Using teeny-torch","Using \u003Ccode>teeny-torch\u003C\u002Fcode>","\u002Fteenygrad\u002Fpytorch-interop\u002Fteeny-torch",{"title":26,"titleHtml":27,"route":28},"teeny-llm","\u003Ccode>teeny-llm\u003C\u002Fcode>","\u002Fteenygrad\u002Fcli-and-aot\u002Fteeny-llm",1786271829107]