Crazy Wisdom
Beyond the Parameters: Exploring the Real-World Applications of LLMs
- AI & Agents
- large language models
- machine learning infrastructure
- gpu optimization
- model fine-tuning
- startup technology
- ai deployment
About this episode
Michael Louis, the episode's guest, introduces Cerebrium as a platform that deals with abstractions on two levels, specifically focusing on GPUs and scaling for machine learning applications.
The Importance of GPUs in LLMs- Why are GPUs essential for large language models (LLMs)?
- GPUs have the capability to handle tens of gigabytes of data, which makes them superior to CPUs for LLM tasks.
- The cost challenge: Running GPUs 24/7 is not feasible due to their high costs.
- Serverless computing is becoming crucial for making GPU usage affordable.
- Generating a million API requests could cost tens of thousands of dollars, highlighting the importance of cost-efficient solutions.
- Enterprise-level solutions could cost a couple of million dollars annually.
- Instacart uses specific and fine-tuned models for its operations.
- When customer support bots answer queries, each question processed by a 100-billion parameter model can cost approximately ten cents.
- One of the challenges is latency; the time delay is often too high for practical applications.
- Companies like AWS are developing specialized chips for specific use cases to combat latency and other issues.
- GPT-4 and similar models are opening doors for generative AI and traditional machine learning applications.
- Autonomous agents and Artificial General Intelligence (AGI) differ in their approximations and semantic understanding.
- AGI is considered the terminal reference for fully self-aware AI, while autonomous agents operate based on defined workflows.
- Michael Louis aims to make machine learning more accessible for medium-sized enterprises.
- There is a significant challenge in regulating technology that we do not yet fully understand, like cryptocurrency.
- Louis discusses the high unemployment rate in South Africa and the importance of understanding the privilege of a job.
- He emphasizes the role of power in making a social impact.
- In-person relationships are essential for B2B success.
- Louis stresses the need for penetrating network circles for business advantage.
- From his early days in South Africa studying computer science to becoming an entrepreneur, Louis talks about his life journey.
- He shares the experience of selling a startup to Walmart and what led him to his current venture.
- Louis believes that context awareness for robotics is the future, citing examples from Boston Dynamics and others.
- He discusses the ethical implications of robots in everyday life and their potential in various fields like space travel and mining.
Episode transcript
Welcome to the Crazy Wisdom Podcast. This podcast is for you. If you have an insane drive to find the truth of things, it's not the good answers that we seek, but the good questions. I interview a range of different guests from many different fields, all with the intention to uncover the simple truths that are hidden in plain sight. Most people don't want to go there. I go there, my guests go there, and you benefit. Please let me know if you enjoy these episodes and as always, subscribe on itunes, Spotify or wherever you listen to the podcasts.
Welcome to the Crazy Wisdom Podcast. My guest today is Michael Louie. He is the founder at Cerebrum and they did the YC in winter 2020. I'm sorry, 2022. Welcome to the show, Michael.
So, we're a platform that makes it easy for companies to fine tune, deploy machine learning models. And so basically we create abstractions on two levels. One is we take care of all infrastructure when it comes to machine learning, whether it's GPUs, queuing, scaling, things like that, but then also the research elements. So how do we get models to run more, faster, cheaper on different types of hardware? And how do we make sure that models perform better from like a performance review? How do you fine tune them? And so, yeah, that's what we do.
What, what are the main problems with running LLMs or any other types of machine learning algorithms on CPUs? Why, why are GPUs so much better at it?
Your first full transcript is free. After that, an email opens every transcript in the index — a list of readers we can write to, not a guest book.