{"id":37983,"date":"2023-08-26T00:00:03","date_gmt":"2023-08-26T04:00:03","guid":{"rendered":""},"modified":"2023-08-26T00:00:03","modified_gmt":"2023-08-26T04:00:03","slug":"script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial","status":"publish","type":"post","link":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/","title":{"rendered":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial"},"content":{"rendered":"<div align=\"center\" style=\"padding: 2%;\"><iframe loading=\"lazy\" title=\"Create a Large Language Model from Scratch with Python \u2013 Tutorial\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/UU1WVnMk4E8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<blockquote style=\"text-align: left; background-color: #E4E4E4; border: 3px dotted gray; padding:2%;\"><p>\n<a href=\"https:\/\/www.youtube.com\/watch?v=UU1WVnMk4E8\" target=\"offsite\" rel=\"nofollow noopener noreferrer\"><\/p>\n<h3>Create a Large Language Model from Scratch with Python &ndash; Tutorial<\/h3>\n<p><\/a><\/p>\n<p>By <a href=\"https:\/\/www.youtube.com\/channel\/UC8butISFwT-Wl7EV0hUK0BQ\" target=\"offsite\" rel=\"nofollow noopener noreferrer\"><strong>freeCodeCamp.org<\/strong><\/a><br \/><strong>Published<\/strong>: Aug 25, 2023<\/p>\n<p><em style=\"font-family: 'Times New Roman',Times,serif; color: #666;\"><span style=\"font-size: 3.5em; font-weight: bold;\">&#0147;<\/span><\/em><br \/>\n<a href=\"https:\/\/www.youtube.com\/watch?v=UU1WVnMk4E8\" target=\"offsite\" rel=\"nofollow noopener noreferrer\"><img decoding=\"async\" src=\"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg\" class=\"alignright\" align=\"right\" alt=\"freeCodeCamp.org\" style=\"width: 200px; height: auto;\" \/><\/a> Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language models. You will use Python.<\/p>\n<p>\u00e2\u0153\u008f\u00ef\u00b8\u008f Course developed by @elliotarledge <\/p>\n<p>\u00f0\u0178\u2019\u00bb Code and course resources: https:\/\/github.com\/Infatoshi\/fcc-intro-to-llms<\/p>\n<p>\u00e2\u00ad\u0090\u00ef\u00b8\u008f Contents \u00e2\u00ad\u0090\u00ef\u00b8\u008f<br \/>\n(0:00:00) Intro<br \/>\n(0:03:25) Install Libraries<br \/>\n(0:06:24) Pylzma build tools<br \/>\n(0:08:58) Jupyter Notebook<br \/>\n(0:12:11) Download wizard of oz<br \/>\n(0:14:51) Experimenting with text file<br \/>\n(0:17:58) Character-level tokenizer<br \/>\n(0:19:44) Types of tokenizers<br \/>\n(0:20:58) Tensors instead of Arrays<br \/>\n(0:22:37) Linear Algebra heads up<br \/>\n(0:23:29) Train and validation splits<br \/>\n(0:25:30) Premise of Bigram Model<br \/>\n(0:26:41) Inputs and Targets<br \/>\n(0:29:29) Inputs and Targets Implementation<br \/>\n(0:30:10) Batch size hyperparameter<br \/>\n(0:32:13) Switching from CPU to CUDA<br \/>\n(0:33:28) PyTorch Overview<br \/>\n(0:42:49) CPU vs GPU performance in PyTorch<br \/>\n(0:47:49) More PyTorch Functions<br \/>\n(1:06:03) Embedding Vectors<br \/>\n(1:11:33) Embedding Implementation<br \/>\n(1:13:06) Dot Product and Matrix Multiplication<br \/>\n(1:25:42) Matmul Implementation<br \/>\n(1:26:56) Int vs Float<br \/>\n(1:29:52) Recap and get_batch<br \/>\n(1:35:07) nnModule subclass<br \/>\n(1:37:05) Gradient Descent<br \/>\n(1:50:53) Logits and Reshaping<br \/>\n(1:59:28) Generate function and giving the model some context<br \/>\n(2:03:58) Logits Dimensionality<br \/>\n(2:05:17) Training loop + Optimizer + Zerograd explanation<br \/>\n(2:13:56) Optimizers Overview<br \/>\n(2:17:04) Applications of Optimizers<br \/>\n(2:18:11) Loss reporting + Train VS Eval mode<br \/>\n(2:32:54) Normalization Overview<br \/>\n(2:35:45) ReLU, Sigmoid, Tanh Activations<br \/>\n(2:45:15) Transformer and Self-Attention<br \/>\n(2:46:55) Transformer Architecture<br \/>\n(3:17:54) Building a GPT, not Transformer model<br \/>\n(3:19:46) Self-Attention Deep Dive<br \/>\n(3:25:05) GPT architecture<br \/>\n(3:27:07) Switching to Macbook<br \/>\n(3:31:42) Implementing Positional Encoding<br \/>\n(3:36:57) GPTLanguageModel initalization<br \/>\n(3:40:52) GPTLanguageModel forward pass<br \/>\n(3:46:56) Standard Deviation for model parameters<br \/>\n(4:00:50) Transformer Blocks<br \/>\n(4:04:54) FeedForward network<br \/>\n(4:07:53) Multi-head Attention<br \/>\n(4:12:49) Dot product attention<br \/>\n(4:19:43) Why we scale by 1\/sqrt(dk)<br \/>\n(4:26:45) Sequential VS ModuleList Processing<br \/>\n(4:30:47) Overview Hyperparameters<br \/>\n(4:32:14) Fixing errors, refining<br \/>\n(4:34:01) Begin training<br \/>\n(4:35:46) OpenWebText download and Survey of LLMs paper<br \/>\n(4:37:56) How the dataloader\/batch getter will have to change<br \/>\n(4:41:20) Extract corpus with winrar<br \/>\n(4:43:44) Python data extractor<br \/>\n(4:49:23) Adjusting for train and val splits<br \/>\n(4:57:55) Adding dataloader<br \/>\n(4:59:04) Training on OpenWebText<br \/>\n(5:02:22) Training works well, model loading\/saving<br \/>\n(5:04:18) Pickling<br \/>\n(5:05:32) Fixing errors + GPU Memory in task manager<br \/>\n(5:14:05) Command line argument parsing<br \/>\n(5:18:11) Porting code to script<br \/>\n(5:22:04) Prompt: Completion feature + more errors<br \/>\n(5:24:23) nnModule inheritance + generation cropping<br \/>\n(5:27:54) Pretraining vs Finetuning<br \/>\n(5:33:07) R&amp;D pointers<br \/>\n(5:44:38) Outro<\/p>\n<p>\u00f0\u0178\u017d\u2030 Thanks to our Champion and Sponsor supporters:<br \/>\n\u00f0\u0178\u2018\u00be davthecoder<br \/>\n\u00f0\u0178\u2018\u00be jedi-or-sith<br \/>\n\u00f0\u0178\u2018\u00be \u00e5\u008d\u2014\u00e5\u00ae\u00ae\u00e5\u008d\u0192\u00e5\u00bd\u00b1<br \/>\n\u00f0\u0178\u2018\u00be Agust&iacute;n Kussrow<br \/>\n\u00f0\u0178\u2018\u00be Nattira Maneerat<br \/>\n\u00f0\u0178\u2018\u00be Heather Wcislo<br \/>\n\u00f0\u0178\u2018\u00be Serhiy Kalinets<br \/>\n\u00f0\u0178\u2018\u00be Justin Hual<br \/>\n\u00f0\u0178\u2018\u00be Otis Morgan<\/p>\n<p>&#8212;<\/p>\n<p>Learn to code for free and get a developer job: https:\/\/www.freecodecamp.org<\/p>\n<p>Read hundreds of articles on programming: https:\/\/freecodecamp.org\/news<\/p>\n<div style=\"text-align: center; font-weight: bold\">[<a href=\"https:\/\/www.youtube.com\/watch?v=UU1WVnMk4E8\" target=\"offsite\" rel=\"nofollow noopener noreferrer\" title=\"Read More on YouTube\">READ MORE<\/a>]<\/div>\n<p><em style=\"font-family: 'Times New Roman',Times,serif; color: #666;\"><span style=\"font-size: 3.5em; font-weight: bold;\">&#0148;<\/span><\/em><\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language &#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[380],"tags":[],"class_list":["post-37983","post","type-post","status-publish","format-standard","hentry","category-video-tutorials"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!\" \/>\n<meta property=\"og:description\" content=\"Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language ...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/\" \/>\n<meta property=\"og:site_name\" content=\"WHUZZUP!\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/EricBrooksDotComWeb\/\" \/>\n<meta property=\"article:author\" content=\"EricBrooksDotComWeb\" \/>\n<meta property=\"article:published_time\" content=\"2023-08-26T04:00:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg\" \/>\n<meta name=\"author\" content=\"Eric Brooks\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@EricBrooksCom\" \/>\n<meta name=\"twitter:site\" content=\"@EricBrooksCom\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Eric Brooks\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/\"},\"author\":{\"name\":\"Eric Brooks\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#\\\/schema\\\/person\\\/3b162468e1f9814f6e9f61460db00a3a\"},\"headline\":\"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial\",\"datePublished\":\"2023-08-26T04:00:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/\"},\"wordCount\":383,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i2.ytimg.com\\\/vi\\\/UU1WVnMk4E8\\\/hqdefault.jpg\",\"articleSection\":[\"Video Tutorials\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/\",\"url\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/\",\"name\":\"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i2.ytimg.com\\\/vi\\\/UU1WVnMk4E8\\\/hqdefault.jpg\",\"datePublished\":\"2023-08-26T04:00:03+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i2.ytimg.com\\\/vi\\\/UU1WVnMk4E8\\\/hqdefault.jpg\",\"contentUrl\":\"https:\\\/\\\/i2.ytimg.com\\\/vi\\\/UU1WVnMk4E8\\\/hqdefault.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#website\",\"url\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/\",\"name\":\"WHUZZUP!\",\"description\":\"What&#039;s new at EricBrooks.Com plus news and gossip all over CyberTown\",\"publisher\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#organization\",\"name\":\"WHUZZUP!\",\"url\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ericbrooks.com\\\/images\\\/posts\\\/cropped-favicon_64-1.png\",\"contentUrl\":\"https:\\\/\\\/ericbrooks.com\\\/images\\\/posts\\\/cropped-favicon_64-1.png\",\"width\":512,\"height\":512,\"caption\":\"WHUZZUP!\"},\"image\":{\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/EricBrooksDotComWeb\\\/\",\"https:\\\/\\\/x.com\\\/EricBrooksCom\",\"https:\\\/\\\/www.instagram.com\\\/ericbrooksdotcom\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/ericbrooksdotcom\",\"https:\\\/\\\/myspace.com\\\/ericbrooksdotcom\",\"https:\\\/\\\/www.pinterest.com\\\/ericbrookscom\\\/\",\"https:\\\/\\\/www.youtube.com\\\/user\\\/ericbrooksdotcom\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/#\\\/schema\\\/person\\\/3b162468e1f9814f6e9f61460db00a3a\",\"name\":\"Eric Brooks\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x\",\"caption\":\"Eric Brooks\"},\"description\":\"Web &amp; Graphic Designer, Blogger, Musician, Evil Clown. I also code websites and promote people for a living. Still kind of a jerk, though. Approach with food.\",\"sameAs\":[\"https:\\\/\\\/ericbrooks.com\",\"EricBrooksDotComWeb\",\"ericbrooksdotcom\",\"ericbrookscom\\\/\",\"https:\\\/\\\/x.com\\\/EricBrooksCom\"],\"url\":\"https:\\\/\\\/ericbrooks.com\\\/whuzzup\\\/author\\\/site-admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/","og_locale":"en_US","og_type":"article","og_title":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!","og_description":"Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language ...","og_url":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/","og_site_name":"WHUZZUP!","article_publisher":"https:\/\/www.facebook.com\/EricBrooksDotComWeb\/","article_author":"EricBrooksDotComWeb","article_published_time":"2023-08-26T04:00:03+00:00","og_image":[{"url":"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg","type":"","width":"","height":""}],"author":"Eric Brooks","twitter_card":"summary_large_image","twitter_creator":"@EricBrooksCom","twitter_site":"@EricBrooksCom","twitter_misc":{"Written by":"Eric Brooks","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#article","isPartOf":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/"},"author":{"name":"Eric Brooks","@id":"https:\/\/ericbrooks.com\/whuzzup\/#\/schema\/person\/3b162468e1f9814f6e9f61460db00a3a"},"headline":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial","datePublished":"2023-08-26T04:00:03+00:00","mainEntityOfPage":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/"},"wordCount":383,"commentCount":0,"publisher":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/#organization"},"image":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg","articleSection":["Video Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/","url":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/","name":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial - WHUZZUP!","isPartOf":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#primaryimage"},"image":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#primaryimage"},"thumbnailUrl":"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg","datePublished":"2023-08-26T04:00:03+00:00","breadcrumb":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#primaryimage","url":"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg","contentUrl":"https:\/\/i2.ytimg.com\/vi\/UU1WVnMk4E8\/hqdefault.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/ericbrooks.com\/whuzzup\/script-coding-create-a-large-language-model-from-scratch-with-python-ndash-tutorial\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ericbrooks.com\/whuzzup\/"},{"@type":"ListItem","position":2,"name":"[#Script #Coding] Create a Large Language Model from Scratch with Python &ndash; Tutorial"}]},{"@type":"WebSite","@id":"https:\/\/ericbrooks.com\/whuzzup\/#website","url":"https:\/\/ericbrooks.com\/whuzzup\/","name":"WHUZZUP!","description":"What&#039;s new at EricBrooks.Com plus news and gossip all over CyberTown","publisher":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ericbrooks.com\/whuzzup\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ericbrooks.com\/whuzzup\/#organization","name":"WHUZZUP!","url":"https:\/\/ericbrooks.com\/whuzzup\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ericbrooks.com\/whuzzup\/#\/schema\/logo\/image\/","url":"https:\/\/ericbrooks.com\/images\/posts\/cropped-favicon_64-1.png","contentUrl":"https:\/\/ericbrooks.com\/images\/posts\/cropped-favicon_64-1.png","width":512,"height":512,"caption":"WHUZZUP!"},"image":{"@id":"https:\/\/ericbrooks.com\/whuzzup\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/EricBrooksDotComWeb\/","https:\/\/x.com\/EricBrooksCom","https:\/\/www.instagram.com\/ericbrooksdotcom\/","https:\/\/www.linkedin.com\/in\/ericbrooksdotcom","https:\/\/myspace.com\/ericbrooksdotcom","https:\/\/www.pinterest.com\/ericbrookscom\/","https:\/\/www.youtube.com\/user\/ericbrooksdotcom"]},{"@type":"Person","@id":"https:\/\/ericbrooks.com\/whuzzup\/#\/schema\/person\/3b162468e1f9814f6e9f61460db00a3a","name":"Eric Brooks","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x","url":"https:\/\/secure.gravatar.com\/avatar\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/62114f8baf9a3737e7686125b69907010ad6aff5dfc07c12707e8a18bb49bc34?s=96&d=blank&r=x","caption":"Eric Brooks"},"description":"Web &amp; Graphic Designer, Blogger, Musician, Evil Clown. I also code websites and promote people for a living. Still kind of a jerk, though. Approach with food.","sameAs":["https:\/\/ericbrooks.com","EricBrooksDotComWeb","ericbrooksdotcom","ericbrookscom\/","https:\/\/x.com\/EricBrooksCom"],"url":"https:\/\/ericbrooks.com\/whuzzup\/author\/site-admin\/"}]}},"_links":{"self":[{"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/posts\/37983","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/comments?post=37983"}],"version-history":[{"count":0,"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/posts\/37983\/revisions"}],"wp:attachment":[{"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/media?parent=37983"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/categories?post=37983"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ericbrooks.com\/whuzzup\/wp-json\/wp\/v2\/tags?post=37983"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}