[{"data":1,"prerenderedAt":41},["ShallowReactive",2],{"chapter:kernels\u002Fin-a-model\u002Fruntime-op.json":3},{"project":4,"route":5,"title":6,"titleHtml":6,"navTitle":6,"part":7,"sourcePath":8,"editUrl":9,"html":10,"toc":11,"hasMermaid":34,"prev":35,"next":38},"kernels","\u002Fkernels\u002Fin-a-model\u002Fruntime-op","Wiring the Runtime","Kernels in a Real Model","in-a-model\u002Fruntime-op.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fkernels\u002Fsrc\u002Fin-a-model\u002Fruntime-op.md","\u003Cp>\u003Ccode>CustomOp\u003C\u002Fcode> described your operation to the graph. \u003Ccode>RuntimeOp\u003C\u002Fcode> describes it to\nthe thing that launches kernels: how many inputs, what buffers, which arguments\nin what order, and how big a grid.\u003C\u002Fp>\n\u003Cp>The trait has twenty methods. You need five.\u003C\u002Fp>\n\u003Ch2 id=\"the-five\">The five\u003C\u002Fh2>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> n_activation_inputs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> param_shapes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> input_shapes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[&[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]],\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">])\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>>;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> pack_args\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> inputs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_row_stride\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> block\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> [\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 3\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> grid\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">])\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> [\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 3\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Everything else has a default that is correct for a single-launch, inference-only\nop.\u003C\u002Fp>\n\u003Cp>Note the shapes here are \u003Ccode>&[usize]\u003C\u002Fcode>, not the \u003Ccode>Shape\u003C\u002Fcode> of the last chapter. By the\ntime \u003Ccode>RuntimeOp\u003C\u002Fcode> is consulted the batch dimension is known, so there are no\n\u003Ccode>None\u003C\u002Fcode>s left. Symbolic shapes are a graph-construction concern; runtime shapes\nare concrete.\u003C\u002Fp>\n\u003Ch2 id=\"inputs-versus-parameters\">Inputs versus parameters\u003C\u002Fh2>\n\u003Cp>The distinction matters and is easy to get backwards.\u003C\u002Fp>\n\u003Cp>An \u003Cstrong>activation input\u003C\u002Fstrong> comes from another node in the graph. It changes every\ninference. \u003Ccode>n_activation_inputs\u003C\u002Fcode> says how many your op consumes, and they arrive\nin \u003Ccode>pack_args\u003C\u002Fcode> as \u003Ccode>inputs\u003C\u002Fcode>, in recording order.\u003C\u002Fp>\n\u003Cp>A \u003Cstrong>parameter\u003C\u002Fstrong> is a buffer your op owns. Weights, biases, lookup tables,\nprecomputed geometry. It is allocated once when the model loads and persists.\n\u003Ccode>param_shapes\u003C\u002Fcode> declares them; they arrive as \u003Ccode>params\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>\u003Ccode>param_shapes\u003C\u002Fcode> receives concrete shapes, so a parameter can be sized from the\ninput:\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> param_shapes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> input_shapes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[&[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]],\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> _output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">])\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> input_shapes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">][\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">          \u002F\u002F boxes is [B, 4, A]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7FB6D9\">    vec!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">vec!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> vec!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> vec!\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]]\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">      \u002F\u002F anchor_x, anchor_y, strides\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Parameters are zero-initialised by default, because they are usually trained\nweights loaded from a checkpoint. When yours is a constant you computed on the\nhost, \u003Ccode>param_init_data\u003C\u002Fcode> uploads it:\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> param_init_data\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> param_idx\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Option\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">Vec\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">u8\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> data\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">f32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> match\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> param_idx \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">{\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B79AD4\">        0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">anchor_x\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B79AD4\">        1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">anchor_y\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#B79AD4\">        2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> =>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">strides\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        _ \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=>\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> return\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> None\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    };\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    Some\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">data\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">iter\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">().\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">flat_map\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(|&\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">f\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">|\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">from_f64\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">f \u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> f64\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_le_bytes\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">()).\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">collect\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">())\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Cem>From \u003Ccode>vision-rs\u002Fsrc\u002Fmodels\u002Fyolo\u002Fkernels\u002Fdetect_decode.rs\u003C\u002Fcode>.\u003C\u002Fem>\u003C\u002Fp>\n\u003Cp>Little-endian bytes, in the buffer’s element type — which is why that conversion\ngoes through \u003Ccode>D::from_f64\u003C\u002Fcode> rather than writing \u003Ccode>f32\u003C\u002Fcode> bytes directly. The byte\ncount must match \u003Ccode>param_shapes()[idx]\u003C\u002Fcode>'s product times the dtype size, and\nnothing checks it.\u003C\u002Fp>\n\u003Cp>If a parameter needs a name, for loading from a checkpoint under a dotted key,\n\u003Ccode>param_names\u003C\u002Fcode> supplies one per slot.\u003C\u002Fp>\n\u003Ch2 id=\"packing-arguments\">Packing arguments\u003C\u002Fh2>\n\u003Cp>This is where your op meets your kernel, and it is the part with no safety net.\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> pack_args\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    &\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    inputs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">RawPtr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">])],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">RawPtr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    output\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> RawPtr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    _output_row_stride\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">mut\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> dyn\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> ArgVisitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">inputs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">].\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\"> \u002F\u002F boxes_ptr\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">   \u002F\u002F anchor_x_ptr\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">   \u002F\u002F anchor_y_ptr\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">   \u002F\u002F strides_ptr\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_ptr\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">output\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">      \u002F\u002F out_ptr\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">b\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">           \u002F\u002F _B\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">    visitor\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">visit_i32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">           \u002F\u002F A\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Seven calls, in exactly the order the kernel declares its parameters. The\ntrailing comments are load-bearing: they are the only thing connecting this\nsequence to the function signature.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Nothing checks it.\u003C\u002Fstrong> Not the order, not the count, not the types. Swap two\n\u003Ccode>visit_ptr\u003C\u002Fcode> calls and you get wrong numbers, silently. Pass six arguments to a\nseven-parameter kernel and the seventh is whatever was in that register.\u003C\u002Fp>\n\u003Cp>The macro already knows the right answer — it generates \u003Ccode>type Args<'a> = (*mut f32, ..., i32)\u003C\u002Fcode> for the by-hand launch path — and that knowledge is simply not\nused here. It is the third item in\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fblob\u002Fmain\u002Fbooks\u002Fkernels\u002FAPI-FRICTION.md\" target=\"_blank\" rel=\"noopener noreferrer\">\u003Ccode>API-FRICTION.md\u003C\u002Fcode>\u003C\u002Fa>.\u003C\u002Fp>\n\u003Cp>Until that changes: write \u003Ccode>pack_args\u003C\u002Fcode> immediately after the kernel signature,\nkeep the comments, and make the first test one that checks numbers rather than\nthat it runs.\u003C\u002Fp>\n\u003Ch2 id=\"block-and-grid\">Block and grid\u003C\u002Fh2>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> block\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> [\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 3\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> [\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">block_a \u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> grid\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> &[\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">])\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> [\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">;\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 3\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> b \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">0\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output_shape\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">[\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\">2\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">];\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a_tiles \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">div_ceil\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">block_a \u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    [(\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">b \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">*\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> a_tiles\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> as\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> u32\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#B79AD4\"> 1\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">]\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Ccode>block\u003C\u002Fcode> is threads per program. For an elementwise kernel that is the same\nnumber as its \u003Ccode>BLOCK_SIZE\u003C\u002Fcode> const generic, and your op has to keep the two in\nstep because nothing else will.\u003C\u002Fp>\n\u003Cp>For a tiled kernel it is not. \u003Ccode>conv2d_bn_silu\u003C\u002Fcode>’s \u003Ccode>BLOCK_OW\u003C\u002Fcode> is 16 — the width of\nan output tile — while its bench launches with 128 threads. The const generic\ndescribes the \u003Cem>data\u003C\u002Fem> one program covers; the block describes the \u003Cem>threads\u003C\u002Fem> that\ncover it. Chapter 16 pulls those apart.\u003C\u002Fp>\n\u003Cp>\u003Ccode>grid\u003C\u002Fcode> is Chapter 6’s division, rounded up, now with the real output shape.\n\u003Ccode>detect_decode\u003C\u002Fcode> launches a flat grid over both the batch and the anchor tiles,\nwhich the kernel then splits apart with a divide and a remainder — the pattern\nfrom Chapter 6.\u003C\u002Fp>\n\u003Ch2 id=\"multi-launch-ops\">Multi-launch ops\u003C\u002Fh2>\n\u003Cp>Some operations need several launches. Channel-concatenation scatters N input\nchunks into one output buffer and wants one launch per chunk.\u003C\u002Fp>\n\u003Cp>Override \u003Ccode>n_launches\u003C\u002Fcode>, and the executor calls \u003Ccode>pack_args_for_launch\u003C\u002Fcode> and\n\u003Ccode>grid_for_launch\u003C\u002Fcode> with the index instead:\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> n_launches\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\"> self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">n_chunks \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> pack_args_for_launch\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> launch_idx\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">:\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> usize\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> inputs\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> params\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> output\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ...)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7F877D;font-style:italic\">    \u002F\u002F pack for chunk `launch_idx`\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The defaults delegate to \u003Ccode>pack_args\u003C\u002Fcode> and \u003Ccode>grid\u003C\u002Fcode>, so an op with one launch never\nsees these.\u003C\u002Fp>\n\u003Ch2 id=\"row-stride\">Row stride\u003C\u002Fh2>\n\u003Cp>\u003Ccode>pack_args\u003C\u002Fcode> receives \u003Ccode>output_row_stride\u003C\u002Fcode>, which is not always the last dimension\nof the shape.\u003C\u002Fp>\n\u003Cp>The default is the natural row-major stride. But a kernel using tensor\ndescriptors — the TMA path from Chapter 11 — needs rows aligned to 16 bytes,\nwhich for \u003Ccode>f32\u003C\u002Fcode> means a multiple of 4 elements. Override\n\u003Ccode>forward_output_row_stride\u003C\u002Fcode> to round up, and the executor allocates the padded\nbuffer and tells you the real stride.\u003C\u002Fp>\n\u003Cp>Use the argument, not \u003Ccode>output_shape.last()\u003C\u002Fcode>, or a TMA kernel will read the wrong\naddresses.\u003C\u002Fp>\n\u003Ch2 id=\"the-entry-point-and-an-open-question\">The entry point, and an open question\u003C\u002Fh2>\n\u003Cp>\u003Ccode>CustomOp::lower\u003C\u002Fcode> returns four things:\u003C\u002Fp>\n\u003Cpre data-lang=\"rust\" class=\"shiki teeny-datasheet\" style=\"background-color:#16181a;color:#e6e8e3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">fn\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\"> lower\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(&\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">)\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> ->\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Option\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">String\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> String\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> String\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Arc\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"><\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">dyn\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> RuntimeOp\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>)>\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\"> {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> kernel \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> DetectDecodeForward\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">new\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#FF5F9E\">self\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">block_a\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">);\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF5F9E\">    let\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\"> runtime_op \u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">=\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\"> Arc\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">new\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">DetectDecodeRuntimeOp\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">::<\u003C\u002Fspan>\u003Cspan style=\"color:#6FBF98\">D\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">>::\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">new\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(...));\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#6FBF98\">    Some\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">((\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        \"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">detect_decode_forward\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_string\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(),\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">  \u002F\u002F name\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        kernel\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#E6E8E3\">source\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">                        \u002F\u002F source\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">        \"\u003C\u002Fspan>\u003Cspan style=\"color:#D8A76B\">entry_point\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">\"\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">.\u003C\u002Fspan>\u003Cspan style=\"color:#7FB6D9\">to_string\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">(),\u003C\u002Fspan>\u003Cspan style=\"color:#7F877D;font-style:italic\">            \u002F\u002F entry-point symbol\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6E8E3\">        runtime_op\u003C\u002Fspan>\u003Cspan style=\"color:#8A9088\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">    ))\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#8A9088\">}\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The third element is meant to be the PTX symbol to resolve, and the tree is\ninconsistent about it. The trait’s documentation says “conventionally\n\u003Ccode>{name}_entry_point\u003C\u002Fcode>”, which is what every built-in op in \u003Ccode>teeny-kernels\u003C\u002Fcode>\nproduces. Every \u003Ccode>CustomOp\u003C\u002Fcode> in vision-rs passes the bare literal\n\u003Ccode>"entry_point"\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>It does not matter, because the value is never read.\u003C\u002Fstrong> \u003Ccode>LoadedModel\u003C\u002Fcode> resolves\nkernels with \u003Ccode>CudaProgram::<ErasedKernel>::try_from_ptx(&ptx)\u003C\u002Fcode>, an overload that\ntakes no entry-point argument — it parses the symbol out of the compiled PTX’s\n\u003Ccode>.visible .entry\u003C\u002Fcode> directive instead. The only remaining consumer of the field\nanywhere in the workspace is a \u003Ccode>println!\u003C\u002Fcode> in a test.\u003C\u002Fp>\n\u003Cp>So pass whatever you like and it will work. Pass\n\u003Ccode>format!("{}_entry_point", name)\u003C\u002Fcode> anyway: it matches what the macro actually\nemits, it matches \u003Ccode>Kernel::entry_point_name()\u003C\u002Fcode>, and it is what will still be\nright if the field is ever wired up.\u003C\u002Fp>\n\u003Cp>This is worth knowing rather than just worth ignoring. A parameter that looks\nload-bearing and is not is a trap for the next person, which is why it stays in\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fblob\u002Fmain\u002Fbooks\u002Fkernels\u002FKNOWN-GAPS.md\" target=\"_blank\" rel=\"noopener noreferrer\">\u003Ccode>KNOWN-GAPS.md\u003C\u002Fcode>\u003C\u002Fa>\nas item 1 with a suggested fix — delete it and derive it from the name.\u003C\u002Fp>\n\u003Cp>Next: making the op differentiable.\u003C\u002Fp>\n",[12,16,19,22,25,28,31],{"id":13,"text":14,"level":15},"the-five","The five",2,{"id":17,"text":18,"level":15},"inputs-versus-parameters","Inputs versus parameters",{"id":20,"text":21,"level":15},"packing-arguments","Packing arguments",{"id":23,"text":24,"level":15},"block-and-grid","Block and grid",{"id":26,"text":27,"level":15},"multi-launch-ops","Multi-launch ops",{"id":29,"text":30,"level":15},"row-stride","Row stride",{"id":32,"text":33,"level":15},"the-entry-point-and-an-open-question","The entry point, and an open question",false,{"title":36,"titleHtml":36,"route":37},"Your Kernel as a Graph Op","\u002Fkernels\u002Fin-a-model\u002Fgraph-op",{"title":39,"titleHtml":39,"route":40},"Training: The Backward Kernel","\u002Fkernels\u002Fin-a-model\u002Fbackward",1786271830097]