[{"data":1,"prerenderedAt":24},["ShallowReactive",2],{"chapter:teenygrad\u002Fcore-concepts\u002Fcompilation-flow.json":3},{"project":4,"route":5,"title":6,"titleHtml":6,"navTitle":6,"part":7,"sourcePath":8,"editUrl":9,"html":10,"toc":11,"hasMermaid":16,"prev":17,"next":21},"teenygrad","\u002Fteenygrad\u002Fcore-concepts\u002Fcompilation-flow","Compilation Flow","Core Concepts","core-concepts\u002Fcompilation-flow.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fteenygrad\u002Fsrc\u002Fcore-concepts\u002Fcompilation-flow.md","\u003Cp>At a high level, a model goes from traced graph to running device code like this:\u003C\u002Fp>\n\u003Cpre class=\"mermaid\" data-mermaid>graph LR\n    A["Model (teeny-core nn::Layer)"] -->|trace| B["Graph (teeny-core::graph)"]\n    B -->|lower| C["FXGraph"]\n    C -->|compile| D["teeny-compiler backends"]\n    D -->|LLVM\u002FMLIR backend, via teenyc| E["Device object code"]\n    D -->|ndarray backend| F["CPU (ndarray)"]\n    E --> G["Device drivers (teeny-cuda, ...)"]\n\u003C\u002Fpre>\n\u003Cul>\n\u003Cli>\u003Cstrong>Trace\u003C\u002Fstrong>: your model, built from \u003Ccode>teeny-core::nn\u003C\u002Fcode> layers, is called with a symbolic input,\nrecording a \u003Ccode>teeny-core::graph::Graph\u003C\u002Fcode> — see\n\u003Ca href=\"\u002Fteenygrad\u002Fcore-concepts\u002Ftensors-and-graph\">Tensors & the Computational Graph\u003C\u002Fa>.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Lower\u003C\u002Fstrong>: the graph is lowered to FXGraph, \u003Ccode>teeny-compiler\u003C\u002Fcode>’s intermediate representation.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Compile\u003C\u002Fstrong>: \u003Ccode>teeny-compiler\u003C\u002Fcode> (\u003Ca href=\"\u002Fapi\u002Fteenygrad\u002Fteenygrad\u002Fteeny_compiler\u002F\">API docs\u003C\u002Fa>)\ncompiles FXGraph to a target backend:\n\u003Cul>\n\u003Cli>The \u003Cstrong>LLVM\u002FMLIR backend\u003C\u002Fstrong> shells out to the custom \u003Ccode>teenyc\u003C\u002Fcode> compiler at runtime\n(\u003Ccode>-Zcodegen-backend=mlir\u003C\u002Fcode>) — see \u003Ca href=\"\u002Fteenygrad\u002Fcompiler-internals\u002Fllvm-backend\">The LLVM\u002FMLIR Backend\u003C\u002Fa>.\u003C\u002Fli>\n\u003Cli>The \u003Cstrong>\u003Ccode>ndarray\u003C\u002Fcode> backend\u003C\u002Fstrong> (feature-gated, default-on) runs on CPU without \u003Ccode>teenyc\u003C\u002Fcode>.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Device drivers\u003C\u002Fstrong>: compiled device code is loaded and run through a driver crate — today,\n\u003Ca href=\"\u002Fapi\u002Fteenygrad\u002Fteenygrad\u002Fteeny_cuda\u002F\">\u003Ccode>teeny-cuda\u003C\u002Fcode>\u003C\u002Fa> for NVIDIA GPUs. \u003Ccode>teeny-cpu\u003C\u002Fcode> and\n\u003Ccode>teeny-vulkan\u003C\u002Fcode> are planned but not yet implemented (only \u003Ccode>drivers\u002Fteeny-cuda\u003C\u002Fcode> exists in the\nworkspace today).\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Kernels themselves (the actual per-op device code, e.g. attention, matmul, elementwise ops) are\ndefined separately — see \u003Ca href=\"\u002Fteenygrad\u002Fkernels-and-backends\u002Fbackends\">Kernels & Backends\u003C\u002Fa>.\u003C\u002Fp>\n\u003Ch2 id=\"ahead-of-time-vs-just-in-time\">Ahead-of-time vs. just-in-time\u003C\u002Fh2>\n\u003Cp>The same compilation pipeline can run either eagerly (JIT, at model-run time) or ahead of time\n(AOT, producing cached artifacts you deploy without a compiler on the target device) — see\n\u003Ca href=\"\u002Fteenygrad\u002Fcli-and-aot\u002Fteeny-cli\">\u003Ccode>teeny-cli\u003C\u002Fcode> and Ahead-of-Time Compilation\u003C\u002Fa>.\u003C\u002Fp>\n",[12],{"id":13,"text":14,"level":15},"ahead-of-time-vs-just-in-time","Ahead-of-time vs. just-in-time",2,true,{"title":18,"titleHtml":19,"route":20},"Tensors & the Computational Graph","Tensors & the Computational Graph","\u002Fteenygrad\u002Fcore-concepts\u002Ftensors-and-graph",{"title":22,"titleHtml":22,"route":23},"The Dtype System","\u002Fteenygrad\u002Fcore-concepts\u002Fdtype-system",1786271829000]