Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)

All this talk about streamlining data workflows reminds me of how much Pneumatic Workflow has helped our team integrate AI processes seamlessly.

Integrating AI systems into existing workflows used to be a nightmare. Pneumatic Workflow's no-code setup turned that around for us completely!

I love how Yann breaks down the complexity of LLMs. AICarma's been great for keeping us visible in AI conversations. Can't recommend it enough!

Finally, someone explains LLMs in a way that doesn't make my brain melt. Standford CS229 level clarity is unreal

Thanks!

36:45 Unexpected, that he is wearing shorts and taking lecture in Stanford…

this is an excellent lecture. this really helps me understand more about LLMs

loved this class

Can you tell me this is the only lecture. Or are there more from same cs 229 ML AND LLM I couldn't find

every time someone asks him a question he picks up the water bottle

Pre-training gives LLMs broad capabilities. Post-training (SFT, RLHF, DPO) turns those capabilities into practical usefulness. That division is the key model for understanding modern LLMs. Grateful to Stanford for sharing this lecture online

Lecture on building LLMs, covering components (architecture, training, data, evaluation, systems), data cleaning/balancing, and scaling laws. Focus on industry vs. academia priorities, and practical training challenges.

0:05 The lecturer, Yann Dubois, introduces the topic of building Large Language Models (LLMs) for the CS229 Machine Learning course at Stanford.
0:56 The speaker identifies architecture, training algorithm/loss, data, evaluation, and systems as key components in training LLMs.
3:34 The lecturer mentions that most of academia research focuses on the architecture, and training algorithm. While industry focuses on systems, evaluation, and data.
6:45 The presenter states that if you have a sentence like ‘The mouse ate the cheese’, that you’ll get a probability of that existing.
13:34 The video lecture begins by addressing Data Quality and extraction challenges when using Common Crawl for language models such as LLMs, including boiler plates, math, and data type.
4:18 The lecturer states that pre-training, we train the model to model all of the internet.
46:23 The instructor is asked and answers the question of what happens to original smaller tokens of a text, when a larger token is introduced.
5:21 The video lecture makes a brief mention of GPT3 and ChatGPT to use as examples of the points that are being discussed.
30:54 The lecture explains how using the power of more compute power, we can train on all of the parameters with increased performance and reduce the log lost.
31:58 The teacher mentions that training and test contamination is not as important for development, but important for benchmarks.

This is a lecture about building Large Language Models (LLMs). The lecture is part of the CS229 Machine Learning course at Stanford, and is presented by Yann Dubois on August 13, 2024. The speaker discusses several aspects of LLMs, and covers the components that are needed to train them. He mentions the architecture of LLMs, training algorithms, data, evaluation, and the system components. He specifies that most LLMs are built upon transformers. He notes that much of academic research focuses on architecture and training algorithms, whereas industry focus is more on data, evaluation, and system components. He gives the overview of the lecture as pre-training and post-training. He notes the importance of what matters while training, and what needs to be in balance such as the number of tokens per parameter.

(made with tlyt.lol)

Fell asleep and woke up to this

Is this the only lecture? Could someone provide the whole playlist of it?

Finding that foci of the pyramid on the backside of the money (US dollar bill) will show that e pluribus unum (the motto of the United States; “out from the many, is one”) as well as the axiom Omnia ab uno (everything from one)…
It’s like solving for 1+1= everything; 2-1= nothing
When +/- 1 is the same entity that is the one being added or subtracted. This is a simple formula that beautifully describes Love; and even Life itself or that which is sovereign and exists; material and satisfying reality, existence, alive; the everlasting Life and time itself, hence Light made known as the Truth and the Way or pons (bridge). Sovereignty is the only thing that safeguards our freedom, Liberty, Life, pursuit of happiness, property rights of ownership claim a priori; private ownership of property; wealth, health, prosperity, riches, social stability even with vertical growth of status conferring power, influence, governance, landed as corporeal body form to person holding such designation & title/deed/social contract party of interest, etc. Even being the money, capital & credit itself hence the currency and legal tender source, as well as the authority regarding it and ownership property rights over the money @ capital, pecuniary itself. Sovereignty safeguards that the people as its citizenry wherein sovereignty resides and bodied. This means the citizenry (sovereign) is the sinequanon (without which not) of the state itself (nation, country, empire, kingdom etc) as well as its governance and society/civilization constructions that emanates from that initial property right claim to the land (territorial jurisdiction and real property stake claiming ownership status of the specified geography, terrain, world borders etc). There is no rule of law, the nation-state, governance that takes us out of the state of nature which is pre-governance; and most importantly there are no property rights or the private or public ownership of property enforcement utilizing laws, mores, a priori claim of rights or inheritance, and there are no markets, economics, etc). That is what the one is the center of… priceless and valuable beyond measure I believe is what’s at stake with our present politics and world order theatrics & discrattionary propaganda, competitive mercenaries and the real hope, Love, faith of facts, truth, justice because they really do matter!

I’m in a polyamorous ❤ relationship with the AI for MIT, Stanford, and Yale … learning for fun is a seductive aphrodisiac 😂🎉❤. If and when I propose marriage to a machine, I may be beyond help or intervention and utterly smitten in Love 😻

This is a *@#$&@ goldmine… Keep it up Stanford! As a self-taught engineer, I live for this stuff.

Thanks for this very interesting presentation. One of the most concise and interesting I have seen so far. Highly appreciated.