[{"data":1,"prerenderedAt":35},["ShallowReactive",2],{"chapter:kernels\u002Ffirst-kernel\u002Fvector-add.json":3},{"project":4,"route":5,"title":6,"titleHtml":6,"navTitle":6,"part":7,"sourcePath":8,"editUrl":9,"html":10,"toc":11,"hasMermaid":28,"prev":29,"next":32},"kernels","\u002Fkernels\u002Ffirst-kernel\u002Fvector-add","Vector Add, End to End","Your First Kernel","first-kernel\u002Fvector-add.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fkernels\u002Fsrc\u002Ffirst-kernel\u002Fvector-add.md","\u003Cp>This chapter runs a kernel you wrote and gets numbers out of a GPU.\u003C\u002Fp>\n\u003Cp>It is the whole thing at once: the kernel, compiling it, moving data to the\ncard, launching, and reading the answer back. Some of it will not make sense\nyet. That is deliberate — you will get more out of the explanations if you have\nalready seen the thing they explain. Every part is named below with the chapter\nthat takes it apart.\u003C\u002Fp>\n\u003Cp>The job is the simplest one that is still real: add two vectors, element by\nelement.\u003C\u002Fp>\n\u003Cpre class=\"code-panel\" data-lang=\"text\">\u003Ccode>a    = [0, 1, 2, 3, ...]\nb    = [0, 2, 4, 6, ...]\nout  = [0, 3, 6, 9, ...]\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2 id=\"the-kernel\">The kernel\u003C\u002Fh2>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">\u002F\u002F\u002F Adds two vectors: `out[i] = a[i] + b[i]`.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">#[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">kernel\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">pub\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> vector_add\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Triton\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Num\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> const\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> BLOCK_SIZE\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    a_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Pointer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    b_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Pointer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    out_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Pointer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    n_elements\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> where\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">I32Tensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> types\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Tensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">I32Tensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Comparison\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> BoolTensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">BoolTensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Pointer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> AddOffsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">I32Tensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Output\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Tensor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Pointer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>>>,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F Which slice of the vector is this program responsible for?\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> pid \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">program_id\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Axis\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">X\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> block_start \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> pid \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">*\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> BLOCK_SIZE\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F The indices it will touch: block_start, block_start + 1, and so on.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> offsets \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">arange\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> BLOCK_SIZE\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> +\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> block_start\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F The last program runs off the end of the vector. This says which of its\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F lanes are real.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> in_bounds \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">lt\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">n_elements\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">load\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        a_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">add_offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        Some\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">in_bounds\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        &[],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B79AD4\">        false\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    );\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">load\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        b_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">add_offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        Some\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">in_bounds\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        &[],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B79AD4\">        false\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    );\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    T\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">store\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        out_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">add_offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">offsets\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        a \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">+\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        Some\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">in_bounds\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        &[],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    );\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Read it as a description of what \u003Cstrong>one\u003C\u002Fstrong> program does, not what the whole GPU\ndoes. Many copies of this function run at once, and each one handles a different\nslice of the vector. \u003Ccode>program_id\u003C\u002Fcode> is how a copy finds out which slice is its\nown. Chapter 2 explains why the model works this way; Chapter 6 covers\n\u003Ccode>program_id\u003C\u002Fcode> and block sizes properly.\u003C\u002Fp>\n\u003Cp>Four things are worth naming now.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>\u003Ccode>T: Triton\u003C\u002Fcode> is the GPU.\u003C\u002Fstrong> Every operation the kernel can do is a method on it —\n\u003Ccode>T::load\u003C\u002Fcode>, \u003Ccode>T::store\u003C\u002Fcode>, \u003Ccode>T::arange\u003C\u002Fcode>. Writing them through a type parameter rather\nthan calling free functions is what lets the compiler check a kernel before it\never reaches a card.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>\u003Ccode>BLOCK_SIZE\u003C\u002Fcode> is a const generic, not an argument.\u003C\u002Fstrong> It is fixed when the kernel\nis built, not when it is launched, so the compiler can use it — to unroll loops,\nto pick register counts. Chapter 15 covers what else you can specialise this\nway.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>\u003Ccode>offsets\u003C\u002Fcode> is a tensor, not a number.\u003C\u002Fstrong> \u003Ccode>T::arange(0, BLOCK_SIZE)\u003C\u002Fcode> produces\n\u003Ccode>BLOCK_SIZE\u003C\u002Fcode> values at once, and \u003Ccode>+ block_start\u003C\u002Fcode> shifts all of them. You are\nwriting whole-block operations, which is the single biggest difference from\nCUDA. Chapter 7 covers this properly.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>The mask is not optional.\u003C\u002Fstrong> 1000 elements in blocks of 128 is seven full\nprograms and a ragged eighth, which would otherwise read and write 24 elements\npast the end of your buffers. \u003Ccode>in_bounds\u003C\u002Fcode> marks which lanes are real, and\n\u003Ccode>T::load\u003C\u002Fcode> and \u003Ccode>T::store\u003C\u002Fcode> skip the rest. Chapter 7 shows what happens without it.\u003C\u002Fp>\n\u003Cp>The three \u003Ccode>where\u003C\u002Fcode> clauses are the price of indexing memory at all. They are\nidentical in every kernel in this book, and you can copy them without\nunderstanding them today — Chapter 8 explains what they are for.\u003C\u002Fp>\n\u003Ch2 id=\"running-it\">Running it\u003C\u002Fh2>\n\u003Cp>The kernel is one half. The other half runs on the CPU: it compiles the kernel\nfor your card, allocates memory on the device, copies data across, launches, and\ncopies the answer back.\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> main\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Result\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><()>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F a[i] = i, b[i] = 2i — inputs whose correct sum is obvious by eye.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> (\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">..\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">map\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(|\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">|\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> i \u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">collect\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">();\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> (\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">..\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">map\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(|\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">|\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> (\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">*\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">collect\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">();\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F Open the first GPU and ask what it is, so the kernel is compiled for the\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F card that will run it rather than for a guess.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> env \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> teeny_cuda\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">testing\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">setup_cuda_env\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> device \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> env\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F Build the kernel for this block size, then compile it to PTX.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> kernel \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> VectorAdd\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">new\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">BLOCK_SIZE\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> ptx_path \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> compile_kernel\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">kernel\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Target\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">new\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">env\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">capability\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">),\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> false\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">compiled \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> → \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">ptx_path\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">}\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> kernel\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">name\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">());\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F Device memory: two inputs to fill, one output to read back.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> mut\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a_buf \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">buffer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> mut\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b_buf \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">buffer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_buf \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">buffer\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    a_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">a_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    b_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">b_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> ptx \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> std\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">fs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">read\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">ptx_path\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> program \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> teeny_cuda\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">testing\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">load_program_from_ptx\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">VectorAdd\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>>(&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">ptx\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F One program per BLOCK_SIZE-wide slice, rounding up so the ragged tail\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F still gets one. The threads-per-program is not ours to pick: teenyc\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F records it in the PTX, and the driver rejects any other block dimension.\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> grid \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">div_ceil\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">BLOCK_SIZE\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> cfg \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> teeny_cuda\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">testing\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">launch_config_with_grid\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">grid\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">program\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        \"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">launching \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> programs of \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> threads\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        cfg\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">grid\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> cfg\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">block\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    );\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    device\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">launch\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        &\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">program\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        &\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">cfg\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        (\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">            a_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">as_device_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> *\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">mut\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">            b_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">as_device_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> *\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">mut\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">            out_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">as_device_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> *\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">mut\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">            N\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        ),\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    )?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> mut\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_host \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> vec!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    out_buf\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">mut\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)?;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    for\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> i \u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">in\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">..\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">        let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> expected \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> +\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">        anyhow\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">ensure!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">            (\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> -\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> expected\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">abs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> <\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">e-\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">5\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">            \"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">mismatch at \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">: gpu=\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> expected=\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">expected\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">}\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">            out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">i\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        );\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">out[0]   = \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">out[1]   = \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">out[999] = \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{}\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> out_host\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> -\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    println!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">all \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">N\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> elements match the CPU result\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    Ok\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(())\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Six steps, in order:\u003C\u002Fp>\n\u003Col>\n\u003Cli>\u003Cstrong>Ask the device what it is.\u003C\u002Fstrong> \u003Ccode>setup_cuda_env\u003C\u002Fcode> opens the first GPU and reads\nits compute capability — \u003Ccode>sm_89\u003C\u002Fcode>, \u003Ccode>sm_120\u003C\u002Fcode>, whatever you have. The kernel is\nthen compiled for that card rather than for a guess.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Build the kernel.\u003C\u002Fstrong> \u003Ccode>VectorAdd::<f32>::new(BLOCK_SIZE)\u003C\u002Fcode> is a type you never\nwrote. \u003Ccode>#[kernel]\u003C\u002Fcode> generated it from your function. Chapter 8 shows exactly\nwhat it generated and why.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Compile it.\u003C\u002Fstrong> \u003Ccode>compile_kernel\u003C\u002Fcode> produces a PTX file — the assembly language\nNVIDIA cards accept. This is where your function stops being Rust. Chapter 9\nopens the PTX and reads it.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Allocate and copy.\u003C\u002Fstrong> GPU memory is separate from your program’s memory.\n\u003Ccode>device.buffer\u003C\u002Fcode> reserves space on the card; \u003Ccode>to_device\u003C\u002Fcode> copies into it.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Launch.\u003C\u002Fstrong> The grid says how many programs to start. 1000 elements in blocks\nof 128 rounds up to 8. The tuple is the kernel’s arguments, in the order the\nfunction declares them.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Copy back and check.\u003C\u002Fstrong> \u003Ccode>to_host\u003C\u002Fcode> brings the answer into ordinary Rust\nmemory, where it is just a \u003Ccode>Vec<f32>\u003C\u002Fcode>.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Ch2 id=\"run-it-yourself\">Run it yourself\u003C\u002Fh2>\n\u003Cp>You need the \u003Ccode>cuda\u003C\u002Fcode> feature and a card of compute capability sm_75 or newer —\nTuring, from 2018, or anything since. Chapter 4 covers getting the toolchain\ninstalled.\u003C\u002Fp>\n\u003Cpre data-lang=\"bash\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">cargo\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> run\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> -p\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> teeny-triton\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --features\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> cuda\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> --example\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\"> vector_add\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>On an RTX 5070:\u003C\u002Fp>\n\u003Cpre class=\"code-panel\" data-lang=\"text\">\u003Ccode>[1\u002F9] CUDA available\n[2\u002F9] found 1 device(s)\n[3\u002F9] device: NVIDIA GeForce RTX 5070 (capability: sm_120)\ncompiled vector_add → \u002Ftmp\u002Fteenyc_cache\u002Fvector_add_5f69418a643d1353dba2ce66de8ed3dc4e1644c0d9474da4517ed6e7d3f67ff9.o\n      loading PTX directly via driver JIT...\n[CUDA-JIT] info: ptxas warning : .loc directive without .file directive is found, line information in generated binary may not be complete\n\n[7\u002F9] loaded PTX: module=0x60834a26a7e0 function=0x60834a958fd0 num_warps=4 num_ctas=1\nlaunching 8 programs of 128 threads\nout[0]   = 0\nout[1]   = 3\nout[999] = 2997\nall 1000 elements match the CPU result\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Ccode>out[i]\u003C\u002Fcode> is \u003Ccode>3i\u003C\u002Fcode>, because \u003Ccode>a[i] + b[i]\u003C\u002Fcode> is \u003Ccode>i + 2i\u003C\u002Fcode>. The numbered lines and the\n\u003Ccode>ptxas\u003C\u002Fcode> warning come from the setup helpers, not from this program.\u003C\u002Fp>\n\u003Cp>Three things in that output are worth noticing now, and are explained later:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>\u003Ccode>num_warps=4\u003C\u002Fcode>.\u003C\u002Fstrong> 128 threads is four warps of 32. You did not choose that\nnumber — it was read back out of the compiled PTX. Chapter 16.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>The cache filename is a hash\u003C\u002Fstrong>, not the readable kernel id. Chapter 8.\u003C\u002Fli>\n\u003Cli>\u003Cstrong>The extension is \u003Ccode>.o\u003C\u002Fcode>, but the file is PTX text.\u003C\u002Fstrong> Chapter 9 opens it.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2 id=\"what-you-just-did\">What you just did\u003C\u002Fh2>\n\u003Cp>You wrote a function that never ran in your process. Its source text was\ncaptured by a macro, compiled by a different compiler into GPU assembly, loaded\nonto a card, and executed a thousand times over in parallel.\u003C\u002Fp>\n\u003Cp>If that sequence sounds strange, it is, and it is worth understanding before you\nwrite a second kernel. Chapter 3 explains it.\u003C\u002Fp>\n\u003Ch2 id=\"what-is-missing\">What is missing\u003C\u002Fh2>\n\u003Cp>This kernel runs on its own. It is not part of a model, it has no gradient, and\nnothing chose its block size but you.\u003C\u002Fp>\n\u003Cp>None of that is needed to run a kernel, which is why none of it is in this\nchapter. Part 5 attaches a kernel to a model’s graph and gives it a backward\npass. Chapter 6 is about choosing that block size deliberately.\u003C\u002Fp>\n\u003Cp>Next, though: what one program actually is, and how it finds its slice.\u003C\u002Fp>\n",[12,16,19,22,25],{"id":13,"text":14,"level":15},"the-kernel","The kernel",2,{"id":17,"text":18,"level":15},"running-it","Running it",{"id":20,"text":21,"level":15},"run-it-yourself","Run it yourself",{"id":23,"text":24,"level":15},"what-you-just-did","What you just did",{"id":26,"text":27,"level":15},"what-is-missing","What is missing",false,{"title":30,"titleHtml":30,"route":31},"Setting Up","\u002Fkernels\u002Forientation\u002Fsetting-up",{"title":33,"titleHtml":33,"route":34},"The Kernel Body","\u002Fkernels\u002Ffirst-kernel\u002Fkernel-body",1786271829651]