[{"data":1,"prerenderedAt":20},["ShallowReactive",2],{"chapter:teenygrad\u002Fnn-layers\u002Fbuilding-models.json":3},{"project":4,"route":5,"title":6,"titleHtml":7,"navTitle":6,"part":8,"sourcePath":9,"editUrl":10,"html":11,"toc":12,"hasMermaid":13,"prev":14,"next":17},"teenygrad","\u002Fteenygrad\u002Fnn-layers\u002Fbuilding-models","Building Models with nn","Building Models with \u003Ccode>nn\u003C\u002Fcode>","Neural Network Layers","nn-layers\u002Fbuilding-models.md","https:\u002F\u002Fgithub.com\u002Fteenygrad\u002Fteenygrad\u002Fedit\u002Fmain\u002Fbooks\u002Fteenygrad\u002Fsrc\u002Fnn-layers\u002Fbuilding-models.md","\u003Cp>\u003Ccode>teeny-core::nn\u003C\u002Fcode> (\u003Ca href=\"\u002Fapi\u002Fteenygrad\u002Fteenygrad\u002Fteeny_core\u002Fnn\u002F\">API docs\u003C\u002Fa>) provides the \u003Ccode>Layer\u003C\u002Fcode>\ntrait and a set of standard layers:\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Cstrong>Convolution\u003C\u002Fstrong>: \u003Ccode>conv1d\u003C\u002Fcode>, \u003Ccode>conv2d\u003C\u002Fcode>, \u003Ccode>conv3d\u003C\u002Fcode>\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Normalization\u003C\u002Fstrong>: \u003Ccode>batchnorm\u003C\u002Fcode>, \u003Ccode>groupnorm\u003C\u002Fcode>, \u003Ccode>instancenorm\u003C\u002Fcode>, \u003Ccode>layernorm\u003C\u002Fcode>, \u003Ccode>rmsnorm\u003C\u002Fcode>\u003C\u002Fli>\n\u003Cli>\u003Cstrong>Other\u003C\u002Fstrong>: \u003Ccode>activation\u003C\u002Fcode>, \u003Ccode>flatten\u003C\u002Fcode>, \u003Ccode>linear\u003C\u002Fcode>, \u003Ccode>pad\u003C\u002Fcode>, \u003Ccode>pool\u003C\u002Fcode>\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Every layer implements \u003Ccode>Layer<I>\u003C\u002Fcode>, with an associated \u003Ccode>Output\u003C\u002Fcode> type and a \u003Ccode>call(&self, input: I) -> Self::Output\u003C\u002Fcode> method — this is what gets invoked during \u003Ca href=\"\u002Fteenygrad\u002Fcore-concepts\u002Ftensors-and-graph#tracing\">tracing\u003C\u002Fa>\nto build up a \u003Ccode>Graph\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>Composite models (e.g. \u003Ccode>teeny-vision::mnist::mnist_lenet5\u003C\u002Fcode>) are just structs composing these\nlayers and implementing \u003Ccode>Layer\u003C\u002Fcode> themselves by chaining calls to their fields’ \u003Ccode>call\u003C\u002Fcode> methods — see\n\u003Ccode>models\u002Fteeny-vision\u002Fsrc\u002Fmnist\u002F\u003C\u002Fcode> for a complete example.\u003C\u002Fp>\n\u003Cblockquote>\n\u003Cp>\u003Cstrong>TODO\u003C\u002Fstrong>: document the parameter-management\u002Finitialization story once it’s stable enough to\ncommit to in the book (constructing layers with random vs. loaded weights, interaction with\n\u003Ca href=\"\u002Fteenygrad\u002Fcore-concepts\u002Fname-scopes\">name scopes\u003C\u002Fa>).\u003C\u002Fp>\n\u003C\u002Fblockquote>\n",[],false,{"title":15,"titleHtml":15,"route":16},"Name Scopes","\u002Fteenygrad\u002Fcore-concepts\u002Fname-scopes",{"title":18,"titleHtml":18,"route":19},"CPU, CUDA, and Vulkan Backends","\u002Fteenygrad\u002Fkernels-and-backends\u002Fbackends",1786271829002]