[{"data":1,"prerenderedAt":39},["ShallowReactive",2],{"chapter:kernels\u002Forientation\u002Fsetting-up.json":3},{"project":4,"route":5,"title":6,"titleHtml":6,"navTitle":6,"part":7,"sourcePath":8,"editUrl":9,"html":10,"toc":11,"hasMermaid":32,"prev":33,"next":36},"kernels","\u002Fkernels\u002Forientation\u002Fsetting-up","Setting Up","Orientation","orientation\u002Fsetting-up.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fkernels\u002Fsrc\u002Forientation\u002Fsetting-up.md","\u003Cp>Three things to install, in increasing order of how much trouble they are:\u003C\u002Fp>\n\u003Col>\n\u003Cli>\u003Cstrong>Rust.\u003C\u002Fstrong> You have it.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>\u003Ccode>teenyc\u003C\u002Fcode>\u003C\u002Fstrong>, the compiler that turns kernel source into GPU code.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>The CUDA toolkit and a GPU\u003C\u002Fstrong>, to actually run anything.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>You can do useful work with only the first. Each step below says what it buys\nyou, so you can stop when you have enough.\u003C\u002Fp>\n\u003Ch2 id=\"step-1--check-out-the-workspace\">Step 1 — Check out the workspace\u003C\u002Fh2>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">git\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> clone\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cd\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teenygrad\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> check\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> -p\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny-triton\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>If that succeeds, you can already write kernels and have the Rust compiler check\nthem. No GPU is involved: as Chapter 3 explained, kernel bodies are type-checked\nby ordinary rustc, and only \u003Cem>compiling to PTX\u003C\u002Fem> needs anything special.\u003C\u002Fp>\n\u003Cp>This is a real checkpoint, not a formality. Most kernel mistakes are type errors\n— a mask that is not a mask, a pointer to the wrong dtype — and every one of\nthem is caught here.\u003C\u002Fp>\n\u003Ch2 id=\"step-2--install-teenyc\">Step 2 — Install \u003Ccode>teenyc\u003C\u002Fcode>\u003C\u002Fh2>\n\u003Cp>\u003Ccode>teenyc\u003C\u002Fcode> is a modified rustc. It is distributed separately, and \u003Ccode>cargo-teeny\u003C\u002Fcode>\ninstalls it:\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> install\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --git\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fcargo-teeny\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> install-toolchain\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>That downloads the toolchain package, checks its hash, unpacks it, and registers\nit with rustup under a name containing \u003Ccode>teenyc\u003C\u002Fcode>. It is not made your default\ntoolchain, so nothing else you build is affected.\u003C\u002Fp>\n\u003Cblockquote>\n\u003Cp>\u003Ccode>rustup toolchain install\u003C\u002Fcode> cannot be used here. Rustup only accepts\n\u003Ccode>stable\u003C\u002Fcode>\u002F\u003Ccode>beta\u003C\u002Fcode>\u002F\u003Ccode>nightly\u003C\u002Fcode> or a version number as a toolchain name, and rejects\nanything else before it makes a network call — hence the separate command.\u003C\u002Fp>\n\u003C\u002Fblockquote>\n\u003Cp>Check it landed:\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">rustup\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> toolchain\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> list\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> |\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> grep\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teenyc\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">rustup\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> which\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --toolchain\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> <\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">the-name-you-sa\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">w\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teenyc\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The second command prints the binary’s path. That is what \u003Ccode>compile_kernel\u003C\u002Fcode> will\nfind and run.\u003C\u002Fp>\n\u003Ch3 id=\"when-it-cannot-be-found\">When it cannot be found\u003C\u002Fh3>\n\u003Cp>\u003Ccode>compile_kernel\u003C\u002Fcode> looks in two places, in order:\u003C\u002Fp>\n\u003Col>\n\u003Cli>\u003Ccode>$TEENYC_PATH\u003C\u002Fcode>, if set — an explicit path to the binary.\u003C\u002Fli>\n\u003Cli>The one rustup toolchain whose name contains \u003Ccode>teenyc\u003C\u002Fcode>.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>It deliberately does not fall back to a bare \u003Ccode>teenyc\u003C\u002Fcode> on your \u003Ccode>$PATH\u003C\u002Fcode>, because\nthat would work only by accident and fail confusingly. So there are exactly two\nerrors you can get:\u003C\u002Fp>\n\u003Cpre class=\"code-panel\" data-lang=\"text\">\u003Ccode>no teenyc rustup toolchain found; set TEENYC_PATH to the teenyc binary, or\ninstall one with `cargo teeny install-toolchain` (see cargo-teeny)\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cpre class=\"code-panel\" data-lang=\"text\">\u003Ccode>multiple teenyc rustup toolchains found (a, b); set TEENYC_PATH to disambiguate\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Both are fixed by setting \u003Ccode>TEENYC_PATH\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>The tree’s tests and benches all call \u003Ccode>dotenv().ok()\u003C\u002Fcode> before doing anything, so\na \u003Ccode>.env\u003C\u002Fcode> file at the workspace root is the supported way to keep this set:\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\"># .env\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">TEENYC_PATH\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">\u002Fhome\u002Fyou\u002F.rustup\u002Ftoolchains\u002Fstable-teenyc-x86_64-unknown-linux-gnu\u002Fbin\u002Fteenyc\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">TEENYC_CACHE_DIR\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">\u002Fhome\u002Fyou\u002F.cache\u002Fteenyc\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Ccode>TEENYC_CACHE_DIR\u003C\u002Fcode> is where compiled PTX is kept. It defaults to\n\u003Ccode>\u002Ftmp\u002Fteenyc_cache\u003C\u002Fcode>, which most systems clear on reboot — pointing it somewhere\ndurable saves recompiling.\u003C\u002Fp>\n\u003Ch2 id=\"step-3--cuda-and-a-card\">Step 3 — CUDA and a card\u003C\u002Fh2>\n\u003Cp>To run a kernel you need an NVIDIA GPU of compute capability \u003Cstrong>sm_75 or newer\u003C\u002Fstrong>\n— Turing, from 2018, and anything since. That is the floor because Triton’s\nmatrix acceleration needs it; sm_70 and sm_72 have only a deprecated\nfused-multiply-add fallback path.\u003C\u002Fp>\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Capability\u003C\u002Fth>\n\u003Cth>Cards\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_75\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Turing: RTX 20xx, GTX 16xx, T4\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_80\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Ampere datacenter: A100, A30\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_86\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Ampere: RTX 30xx, A40, A10\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_87\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Jetson Orin (AGX \u002F NX \u002F Nano)\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_89\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Ada Lovelace: RTX 40xx, L4, L40S\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_90\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Hopper: H100, H200\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_100\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Blackwell datacenter: B100, B200, GB200\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Ccode>sm_120\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>Blackwell: RTX 50xx\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003Cp>You also need the CUDA toolkit — not just a driver. The \u003Ccode>teeny-cuda\u003C\u002Fcode> crate\ngenerates its bindings from the toolkit’s headers at build time, so without\n\u003Ccode>cuda.h\u003C\u002Fcode> on the include path it fails to build at all:\u003C\u002Fp>\n\u003Cpre class=\"code-panel\" data-lang=\"text\">\u003Ccode>wrapper.h:17:10: fatal error: 'cuda.h' file not found\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>There is no feature flag that skips this. A machine without the toolkit cannot\nbuild \u003Ccode>teeny-cuda\u003C\u002Fcode>, or anything that depends on it, which is why the workspace’s\nCI excludes those crates and why the book’s examples are behind a \u003Ccode>cuda\u003C\u002Fcode>\nfeature.\u003C\u002Fp>\n\u003Cp>Now run one:\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> run\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> -p\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny-triton\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --features\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> cuda\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --example\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> vector_add\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The program opens the first device, prints its name and capability, compiles the\nkernel for that exact card, and adds two vectors.\u003C\u002Fp>\n\u003Ch3 id=\"if-that-fails\">If that fails\u003C\u002Fh3>\n\u003Cp>\u003Cstrong>\u003Ccode>PTX .version 8.6 does not support .target sm_120a\u003C\u002Fcode>\u003C\u002Fstrong> — a Blackwell card, where\n\u003Ccode>teenyc\u003C\u002Fcode>’s default PTX version is newer than the driver accepts. Set\n\u003Ccode>TEENYC_PTX_VERSION=87\u003C\u002Fcode>. This is a \u003Ccode>teenyc\u003C\u002Fcode>-side default; the SDK cannot work\naround it.\u003C\u002Fp>\n\u003Cp>You may well not hit it. On an RTX 5070 with CUDA 13.3 and driver 610.43.02,\neverything in this book ran without the variable set. Newer drivers appear to\naccept the version; try it plain first.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>A capability you want to override\u003C\u002Fstrong> — \u003Ccode>TEENYC_CAPABILITY=sm_89\u003C\u002Fcode> forces the\ntarget, regardless of what the device reports. Useful for reproducing someone\nelse’s build.\u003C\u002Fp>\n\u003Ch2 id=\"what-you-have-now\">What you have now\u003C\u002Fh2>\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>After\u003C\u002Fth>\n\u003Cth>You can\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>Step 1\u003C\u002Ftd>\n\u003Ctd>Write kernels and have them type-checked\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Step 2\u003C\u002Ftd>\n\u003Ctd>Compile kernels to PTX and read the generated code\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Step 3\u003C\u002Ftd>\n\u003Ctd>Run kernels and measure them\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\u003Cp>Steps 1 and 2 cover Chapters 5 through 9 apart from the actual runs. Everything\nin Part 4 needs Step 3, because you cannot optimise what you cannot time.\u003C\u002Fp>\n\u003Cp>Next: the kernel itself.\u003C\u002Fp>\n",[12,16,19,23,26,29],{"id":13,"text":14,"level":15},"step-1--check-out-the-workspace","Step 1 — Check out the workspace",2,{"id":17,"text":18,"level":15},"step-2--install-teenyc","Step 2 — Install teenyc",{"id":20,"text":21,"level":22},"when-it-cannot-be-found","When it cannot be found",3,{"id":24,"text":25,"level":15},"step-3--cuda-and-a-card","Step 3 — CUDA and a card",{"id":27,"text":28,"level":22},"if-that-fails","If that fails",{"id":30,"text":31,"level":15},"what-you-have-now","What you have now",false,{"title":34,"titleHtml":34,"route":35},"From Rust to PTX","\u002Fkernels\u002Forientation\u002Frust-to-ptx",{"title":37,"titleHtml":37,"route":38},"Vector Add, End to End","\u002Fkernels\u002Ffirst-kernel\u002Fvector-add",1786271829645]