Create a Large Language Model from Scratch with Python – Tutorial
By freeCodeCamp.org
Published: Aug 25, 2023“
Learn how to build your own large language model, from scratch. This course goes into the data handling, math, and transformers behind large language models. You will use Python.
âœï¸ Course developed by @elliotarledge
💻 Code and course resources: https://github.com/Infatoshi/fcc-intro-to-llms
âï¸ Contents âï¸
(0:00:00) Intro
(0:03:25) Install Libraries
(0:06:24) Pylzma build tools
(0:08:58) Jupyter Notebook
(0:12:11) Download wizard of oz
(0:14:51) Experimenting with text file
(0:17:58) Character-level tokenizer
(0:19:44) Types of tokenizers
(0:20:58) Tensors instead of Arrays
(0:22:37) Linear Algebra heads up
(0:23:29) Train and validation splits
(0:25:30) Premise of Bigram Model
(0:26:41) Inputs and Targets
(0:29:29) Inputs and Targets Implementation
(0:30:10) Batch size hyperparameter
(0:32:13) Switching from CPU to CUDA
(0:33:28) PyTorch Overview
(0:42:49) CPU vs GPU performance in PyTorch
(0:47:49) More PyTorch Functions
(1:06:03) Embedding Vectors
(1:11:33) Embedding Implementation
(1:13:06) Dot Product and Matrix Multiplication
(1:25:42) Matmul Implementation
(1:26:56) Int vs Float
(1:29:52) Recap and get_batch
(1:35:07) nnModule subclass
(1:37:05) Gradient Descent
(1:50:53) Logits and Reshaping
(1:59:28) Generate function and giving the model some context
(2:03:58) Logits Dimensionality
(2:05:17) Training loop + Optimizer + Zerograd explanation
(2:13:56) Optimizers Overview
(2:17:04) Applications of Optimizers
(2:18:11) Loss reporting + Train VS Eval mode
(2:32:54) Normalization Overview
(2:35:45) ReLU, Sigmoid, Tanh Activations
(2:45:15) Transformer and Self-Attention
(2:46:55) Transformer Architecture
(3:17:54) Building a GPT, not Transformer model
(3:19:46) Self-Attention Deep Dive
(3:25:05) GPT architecture
(3:27:07) Switching to Macbook
(3:31:42) Implementing Positional Encoding
(3:36:57) GPTLanguageModel initalization
(3:40:52) GPTLanguageModel forward pass
(3:46:56) Standard Deviation for model parameters
(4:00:50) Transformer Blocks
(4:04:54) FeedForward network
(4:07:53) Multi-head Attention
(4:12:49) Dot product attention
(4:19:43) Why we scale by 1/sqrt(dk)
(4:26:45) Sequential VS ModuleList Processing
(4:30:47) Overview Hyperparameters
(4:32:14) Fixing errors, refining
(4:34:01) Begin training
(4:35:46) OpenWebText download and Survey of LLMs paper
(4:37:56) How the dataloader/batch getter will have to change
(4:41:20) Extract corpus with winrar
(4:43:44) Python data extractor
(4:49:23) Adjusting for train and val splits
(4:57:55) Adding dataloader
(4:59:04) Training on OpenWebText
(5:02:22) Training works well, model loading/saving
(5:04:18) Pickling
(5:05:32) Fixing errors + GPU Memory in task manager
(5:14:05) Command line argument parsing
(5:18:11) Porting code to script
(5:22:04) Prompt: Completion feature + more errors
(5:24:23) nnModule inheritance + generation cropping
(5:27:54) Pretraining vs Finetuning
(5:33:07) R&D pointers
(5:44:38) Outro🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
”
Category: Video Tutorials
Video Tutorials on Photoshop, after effects, coding, managing your YouTube channel, etc
[#AfterEffects #Video] This EASY VFX Shot Will BLOW Your Mind!
This EASY VFX Shot Will BLOW Your Mind!
By Cinecom.net
Published: Aug 25, 2023“
(Advertising @Storyblocks) http://storyblocks.com/Cinecom – Learn how to create a realistic gun shot wound effect in this tutorial with the help of some practical tools and Adobe After Effects.
Download Unlimited Video Assets
â–º http://storyblocks.com/CinecomMore After Effects Tutorials
â–º https://www.youtube.com/playlist?list=PLV0ZcSTi6tB7MAwC3qJUwxLdbbdZuHMS8Get started with our After Effects for Beginners Course
â–º https://www.cinecom.net/courses/adobe-after-effects-beginners/Read More
► https://www.cinecom.net/adobe-after-effects-tutorials/blow-your-mind-vfx👕 MERCH
► https://cinecom.net/merch🎬 Check our Award Winning Courses
► https://cinecom.net/courses💙 LETS CONNECT!
Instagram â–º http://cinecom.info/Instagram
Discord ► http://discord.gg/Cinecom💥 Download Unlimited Video Assets
â–º http://storyblocks.com/Cinecom#Cinecom #VisualEffects #StoryBlocks
”
[#Script #Coding] Code a Google Drive Clone using Laravel and Vue.js – Complete Course
Code a Google Drive Clone using Laravel and Vue.js – Complete Course
By freeCodeCamp.org
Published: Aug 21, 2023“
Build and deploy a fully functional file manager application with Laravel, PHP, Inertial, and Vue.js. The application is similar to Google Drive.
Learn to implement functionalities such as:
– Creating folders at unlimited depth,
– Uploading files and folders,
– Deleting and Downloading files,
– Searching for files,
– Sharing files,
– Uploading files to AWS S3,
– Deploying the project on a custom domain.âœï¸ Course developed by @TheCodeholic
💻 Code: https://github.com/thecodeholic/laravel-file-manager
âï¸ Contents âï¸
âŒ¨ï¸ (0:00:00) Intro
âŒ¨ï¸ (0:00:29) Demo
âŒ¨ï¸ (0:08:13) Project Setup
âŒ¨ï¸ (0:14:21) Install Laravel Breeze
âŒ¨ï¸ (0:17:49) Email Verification
âŒ¨ï¸ (0:22:15) DB Overview, Models & Migrations
âŒ¨ï¸ (0:35:56) Explore Intertia Folder Structure
âŒ¨ï¸ (0:37:49) Authentication Layout
âŒ¨ï¸ (1:18:08) Define Main Routes
âŒ¨ï¸ (1:24:16) Create Root Folder On User Registration
âŒ¨ï¸ (1:30:34) Creating Folder
âŒ¨ï¸ (2:06:53) Save Folder Path
âŒ¨ï¸ (2:11:46) Show Files into table
âŒ¨ï¸ (2:26:07) Navigate into Second Level of Folders
âŒ¨ï¸ (2:34:18) Create Folders Inside Subfolder
âŒ¨ï¸ (2:37:01) Implement Breadcrumbs
âŒ¨ï¸ (2:46:44) Uploading Files
âŒ¨ï¸ (4:08:30) Show Human Readable File size
âŒ¨ï¸ (4:16:00) Show File Icons
âŒ¨ï¸ (4:27:56) Show File Progress
âŒ¨ï¸ (4:34:54) Display Error When File Already Exists
âŒ¨ï¸ (4:47:49) Implement Load More
âŒ¨ï¸ (5:11:42) Increase max_file_uploads
âŒ¨ï¸ (5:21:20) Deleting Files
âŒ¨ï¸ (6:23:48) Implement Downloading Files
âŒ¨ï¸ (7:04:32) Show Success Notifications
âŒ¨ï¸ (7:15:36) Implement Trash
âŒ¨ï¸ (8:00:03) Implement Favorites
âŒ¨ï¸ (9:18:23) Sharing Files to Others
âŒ¨ï¸ (10:05:16) Implement Shared With Me Page
âŒ¨ï¸ (10:26:49) Implement Shared By Me Page
âŒ¨ï¸ (10:50:23) Searching for Files
âŒ¨ï¸ (11:15:42) Connect to AWS S3
âŒ¨ï¸ (12:15:42) Download Files from S3
âŒ¨ï¸ (12:45:19) Project Deployment & Bug Fixing
âŒ¨ï¸ (13:40:18) Outro🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
”
[#Script #Coding] SQL For Web Developers – Complete Database Course
SQL For Web Developers – Complete Database Course
By freeCodeCamp.org
Published: Aug 24, 2023“
Learn all the basics of Structured Query Language in this comprehensive SQL course. You will build out real database tables and practice querying them in flexible ways as a backend web developer would in a production setting.
💻 Follow along on Boot.dev: https://boot.dev/learn/learn-sql
💻 Raw code: https://github.com/bootdotdev/fcc-learn-sql-assetsâœï¸ Course created by: @bootdotdev
🔗 Learn back-end on Boot.dev: https://boot.dev
Follow Lane on Twitter: https://twitter.com/wagslane
Boot.dev Discord: https://boot.dev/community
FCC Forum: https://forum.freecodecamp.org/âï¸ Contents âï¸
âŒ¨ï¸ (0:00:00) Course Overview (Intro video)
âŒ¨ï¸ (0:03:00) Ch 1. Introduction
âŒ¨ï¸ (0:22:19) Ch 2. Tables
âŒ¨ï¸ (0:50:43) Ch 3. Constraints
âŒ¨ï¸ (1:15:59) Ch 4. CRUD
âŒ¨ï¸ (2:00:13) Ch 5. Basic Queries
âŒ¨ï¸ (2:24:38) Ch 6. Structuring
âŒ¨ï¸ (2:33:52) Ch 7. Aggregations
âŒ¨ï¸ (2:54:14) Ch 8. Subqueries
âŒ¨ï¸ (3:03:40) Ch 9 . Normalization
âŒ¨ï¸ (3:55:44) Ch 10. Joins
âŒ¨ï¸ (4:22:09) Ch 11. Performance🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
”
[#AfterEffects #Video] 5 Incredible NEW A.I. Tools Video Editors NEED To Start Using NOW
5 Incredible NEW A.I. Tools Video Editors NEED To Start Using NOW
By Cinecom.net
Published: Aug 23, 2023“
(Advertising @InVideo) https://ai.invideo.io/cinecom – Learn how to use InVideo, Cascadeur and other A.I. tools to enhance your Video Editing Workflow in this tutorial video.
Check out the Invideo A.I.
â–º https://ai.invideo.io/cinecomA.I. Tools mentioned in the video
â–º https://cascadeur.com/
â–º https://soundraw.io/
â–º https://color.io/match
â–º https://withpoly.com/More A.I. Tutorials
â–º https://www.youtube.com/playlist?list=PLV0ZcSTi6tB4kNtOntNXRIUIO3YW1O-7ZRead More
► https://www.cinecom.net/ai-tools/best-ai-tools-video-creators/👕 MERCH
► https://cinecom.net/merch🎬 Check our Award Winning Courses
► https://cinecom.net/courses💙 LETS CONNECT!
Instagram â–º http://cinecom.info/Instagram
Discord ► http://discord.gg/Cinecom💥 Download Unlimited Video Assets
â–º http://storyblocks.com/Cinecom#Cinecom #ai #videoediting
”
[#Script #Coding] Perl Programming Course for Beginners
Perl Programming Course for Beginners
By freeCodeCamp.org
Published: Aug 23, 2023“
Learn Perl programming in this complete course for beginners. The course covers topics ranging from basic syntax to advanced topics like object-oriented programming, regular expressions, and web development.
âœï¸ Course developed by @perltechstack
âï¸ Contents âï¸
âŒ¨ï¸ (0:00:00) Introduction
âŒ¨ï¸ (0:00:29) Running Perl via CLI
âŒ¨ï¸ (0:04:22) Perl Script File Structure
âŒ¨ï¸ (0:13:58) Data Types – Scalars
âŒ¨ï¸ (0:22:52) Debugging with Data::Dumper
âŒ¨ï¸ (0:26:10) Data Types – Arrays
âŒ¨ï¸ (0:40:52) Data Types – Hashes
âŒ¨ï¸ (0:53:26) References [Part 1]
âŒ¨ï¸ (1:05:36) References [Part 2]
âŒ¨ï¸ (1:12:09) Subroutines
âŒ¨ï¸ (1:28:48) Variable Scope
âŒ¨ï¸ (1:38:36) Error Handling
âŒ¨ï¸ (1:44:04) Logical Operators
âŒ¨ï¸ (1:57:42) Conditional Statements
âŒ¨ï¸ (2:04:19) Loops
âŒ¨ï¸ (2:15:52) Regular Expressions
âŒ¨ï¸ (2:29:32) Built-in Functions
âŒ¨ï¸ (2:40:39) Special Variables
âŒ¨ï¸ (2:47:40) OOP – Classes, instances, attributes and methods
âŒ¨ï¸ (3:03:24) “use” vs “require”
âŒ¨ï¸ (3:12:50) Exporter and method types
âŒ¨ï¸ (3:20:18) Installing modules with CPANM
âŒ¨ï¸ (3:28:33) Improved exception handling
âŒ¨ï¸ (3:37:45) Web requests with Mojo::UserAgent
âŒ¨ï¸ (3:50:46) Reading and writing files
âŒ¨ï¸ (3:59:24) Reading and writing files with Path::Tiny
âŒ¨ï¸ (4:05:30) Running system commands🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
”
[#Script #Coding] Best College Major
Best College Major
By freeCodeCamp.org
Published: Aug 19, 2023“
Build and deploy a fully functional file manager application with Laravel, PHP, Inertial, and Vue.js. The application is similar to Google Drive.
Learn to implement functionalities such as:
– Creating folders at unlimited depth,
– Uploading files and folders,
– Deleting and Downloading files,
– Searching for files,
– Sharing files,
– Uploading files to AWS S3,
– Deploying the project on a custom domain.âœï¸ Course developed by @TheCodeholic
💻 Code: https://github.com/thecodeholic/laravel-file-manager
âï¸ Contents âï¸
âŒ¨ï¸ (0:00:00) Intro
âŒ¨ï¸ (0:00:29) Demo
âŒ¨ï¸ (0:08:13) Project Setup
âŒ¨ï¸ (0:14:21) Install Laravel Breeze
âŒ¨ï¸ (0:17:49) Email Verification
âŒ¨ï¸ (0:22:15) DB Overview, Models & Migrations
âŒ¨ï¸ (0:35:56) Explore Intertia Folder Structure
âŒ¨ï¸ (0:37:49) Authentication Layout
âŒ¨ï¸ (1:18:08) Define Main Routes
âŒ¨ï¸ (1:24:16) Create Root Folder On User Registration
âŒ¨ï¸ (1:30:34) Creating Folder
âŒ¨ï¸ (2:06:53) Save Folder Path
âŒ¨ï¸ (2:11:46) Show Files into table
âŒ¨ï¸ (2:26:07) Navigate into Second Level of Folders
âŒ¨ï¸ (2:34:18) Create Folders Inside Subfolder
âŒ¨ï¸ (2:37:01) Implement Breadcrumbs
âŒ¨ï¸ (2:46:44) Uploading Files
âŒ¨ï¸ (4:08:30) Show Human Readable File size
âŒ¨ï¸ (4:16:00) Show File Icons
âŒ¨ï¸ (4:27:56) Show File Progress
âŒ¨ï¸ (4:34:54) Display Error When File Already Exists
âŒ¨ï¸ (4:47:49) Implement Load More
âŒ¨ï¸ (5:11:42) Increase max_file_uploads
âŒ¨ï¸ (5:21:20) Deleting Files
âŒ¨ï¸ (6:23:48) Implement Downloading Files
âŒ¨ï¸ (7:04:32) Show Success Notifications
âŒ¨ï¸ (7:15:36) Implement Trash
âŒ¨ï¸ (8:00:03) Implement Favorites
âŒ¨ï¸ (9:18:23) Sharing Files to Others
âŒ¨ï¸ (10:05:16) Implement Shared With Me Page
âŒ¨ï¸ (10:26:49) Implement Shared By Me Page
âŒ¨ï¸ (10:50:23) Searching for Files
âŒ¨ï¸ (11:15:42) Connect to AWS S3
âŒ¨ï¸ (12:15:42) Download Files from S3
âŒ¨ï¸ (12:45:19) Project Deployment & Bug Fixing
âŒ¨ï¸ (13:40:18) Outro🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
Learn more about CSS in this full tutorial: https://www.youtube.com/watch?v=OXGznpKZ_sA
Quincy Larson, founder of freeCodeCamp.org, tells you the best college major. Learn more about if college is worth it: https://youtu.be/N07nQ-uUsww
”
[#Script #Coding] CSS to C Major 🎹 Tutorial
CSS to C Major 🎹 Tutorial
By freeCodeCamp.org
Published: Aug 20, 2023“
Build and deploy a fully functional file manager application with Laravel, PHP, Inertial, and Vue.js. The application is similar to Google Drive.
Learn to implement functionalities such as:
– Creating folders at unlimited depth,
– Uploading files and folders,
– Deleting and Downloading files,
– Searching for files,
– Sharing files,
– Uploading files to AWS S3,
– Deploying the project on a custom domain.âœï¸ Course developed by @TheCodeholic
💻 Code: https://github.com/thecodeholic/laravel-file-manager
âï¸ Contents âï¸
âŒ¨ï¸ (0:00:00) Intro
âŒ¨ï¸ (0:00:29) Demo
âŒ¨ï¸ (0:08:13) Project Setup
âŒ¨ï¸ (0:14:21) Install Laravel Breeze
âŒ¨ï¸ (0:17:49) Email Verification
âŒ¨ï¸ (0:22:15) DB Overview, Models & Migrations
âŒ¨ï¸ (0:35:56) Explore Intertia Folder Structure
âŒ¨ï¸ (0:37:49) Authentication Layout
âŒ¨ï¸ (1:18:08) Define Main Routes
âŒ¨ï¸ (1:24:16) Create Root Folder On User Registration
âŒ¨ï¸ (1:30:34) Creating Folder
âŒ¨ï¸ (2:06:53) Save Folder Path
âŒ¨ï¸ (2:11:46) Show Files into table
âŒ¨ï¸ (2:26:07) Navigate into Second Level of Folders
âŒ¨ï¸ (2:34:18) Create Folders Inside Subfolder
âŒ¨ï¸ (2:37:01) Implement Breadcrumbs
âŒ¨ï¸ (2:46:44) Uploading Files
âŒ¨ï¸ (4:08:30) Show Human Readable File size
âŒ¨ï¸ (4:16:00) Show File Icons
âŒ¨ï¸ (4:27:56) Show File Progress
âŒ¨ï¸ (4:34:54) Display Error When File Already Exists
âŒ¨ï¸ (4:47:49) Implement Load More
âŒ¨ï¸ (5:11:42) Increase max_file_uploads
âŒ¨ï¸ (5:21:20) Deleting Files
âŒ¨ï¸ (6:23:48) Implement Downloading Files
âŒ¨ï¸ (7:04:32) Show Success Notifications
âŒ¨ï¸ (7:15:36) Implement Trash
âŒ¨ï¸ (8:00:03) Implement Favorites
âŒ¨ï¸ (9:18:23) Sharing Files to Others
âŒ¨ï¸ (10:05:16) Implement Shared With Me Page
âŒ¨ï¸ (10:26:49) Implement Shared By Me Page
âŒ¨ï¸ (10:50:23) Searching for Files
âŒ¨ï¸ (11:15:42) Connect to AWS S3
âŒ¨ï¸ (12:15:42) Download Files from S3
âŒ¨ï¸ (12:45:19) Project Deployment & Bug Fixing
âŒ¨ï¸ (13:40:18) Outro🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 å—å®®åƒå½±
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan—
Learn to code for free and get a developer job: https://www.freecodecamp.org
Read hundreds of articles on programming: https://freecodecamp.org/news
Learn more about CSS in this full tutorial: https://www.youtube.com/watch?v=OXGznpKZ_sA
”
[#AfterEffects #Video] Replicate the GRADING of your FAVORITE movies | Cinecom #Shorts
Replicate the GRADING of your FAVORITE movies | Cinecom #Shorts
By Cinecom.net
Published: Aug 21, 2023“
👕 MERCH
► https://cinecom.net/merch🎬 Check our Award Winning Courses
► https://cinecom.net/courses💙 LETS CONNECT!
Instagram â–º http://cinecom.info/Instagram
Discord ► http://discord.gg/Cinecom💥 Download Unlimited Video Assets
â–º http://storyblocks.com/Cinecom#Cinecom #CinemaGrade #colorgrading
”
[#Adobe #Design] How to Create an Icon Shimmer in Adobe Premiere Pro CC (2023)
How to Create an Icon Shimmer in Adobe Premiere Pro CC (2023)
By AdobeMasters
Published: Aug 21, 2023“
Check out my Premiere Pro Course: https://www.udemy.com/course/premiere-pro-course/?referralCode=AF659E18BEF06A7F4955
How to create an Icon Shimmer in Adobe Premiere Pro CC. We are going to go over a very simple technique to make this happen in Premiere.
Get near unlimited stock footage and premiere pro templates: http://1.envato.market/c/1413971/298927/4662
Join the Community at: https://adobemasters.net/
Request a Tutorial at: https://adobemasters.net/request-a-tutorial/If you want to learn more about the Adobe products. Here are a couple of cheap courses I learned from.
http://adobemasters.net/courses/Subscribe to see more Adobe related content, videos every other day.
”
