This repository documents my journey into building foundational machine learning models from scratch. The goal is to demystify the core components of modern architectures by implementing them step-by-step, primarily using PyTorch.
Each project is self-contained in its own folder with a dedicated README explaining the concepts, architecture, and setup instructions.
| Project | Description | Key Architectures & Concepts |
|---|---|---|
| GPT-like Model | Build a complete, character-level Generative Pre-trained Transformer from the ground up. | Transformer, Self-Attention, Bigram |
| Llama 3 Deep Dive | Explore and implement the key architectural innovations in Meta's Llama 3 model. | RoPE, SwiGLU, GQA, BPE Tokenizer |
| PaliGemma | An implementation of Google's PaliGemma multimodal model, combining a SigLIP vision encoder and a Gemma LLM. | Multimodal, ViT, GQA, KV Cache, RoPE |
| Translation Transformer | A from-scratch implementation of the original "Attention Is All You Need" Transformer for machine translation. | Encoder-Decoder, Multi-Head Attention, Positional Encoding |
| Denoising Diffusion Model (DDPM) | An implementation of the Denoising Diffusion Probabilistic Model for image generation, featuring a U-Net with attention. | Diffusion Models, U-Net, Self-Attention, Time Embeddings |
| LoRA | A two-part guide to LoRA, covering the SVD math and a practical PyTorch implementation for fine-tuning. | SVD, Low-Rank Factorization, PEFT |
| (More model implementations will be added here...) |
-
Clone the Repository:
git clone https://github.com/majid-200/ML-Models-from-Scratch.git cd ML-Models-from-Scratch -
Create a Virtual Environment:
It's highly recommended to use a virtual environment.
python -m venv venv source venv/bin/activate # On Windows, use `venv\Scripts\activate`
-
Install Dependencies:
Install dependencies based on the project you want to run.
For project-specific setup, like downloading datasets or running scripts, please see the README.md inside each project folder.