✦Crazy Wisdom

Crazy Wisdom

Episode #548: The Pixel Path: From Perception to Action, and the Future of Intelligent Robots with Nizar

Episode #548 · May 25, 2026 · 56 min

MP3 · Apple Podcasts · Spotify

About this episode

Stewart Alsop interviews Nizar, CEO of Pixel Robotics, on the Crazy Wisdom Podcast to explore the intersection of AI, robotics, and perception. The conversation covers a wide range of technical topics including how transformers enable multimodal representation across text, images, and voice, the role of world models in predicting physical interactions, the advantages of diffusion models over traditional LLMs for certain applications, and the challenges of achieving real-time processing for robotics applications. Nizar explains Pixel Robotics' work on creating accurate 3D meshes from smartphone cameras for companies like L'Oréal, moving away from specialized sensors to make the technology more accessible through sophisticated algorithms, and discusses the future of robotics as closing the perception-action loop to enable robots to perform real tasks beyond simple demonstrations. To find out more visit Pixel Robotics' website.
Timestamps
  • 00:00Stewart welcomes Nizar, CEO of Pixel Robotics, discussing what a pixel is as the smallest visual unit on screens composed of red green and blue colors
  • 05:00Discussion of perception systems and how logarithmic laws help compress signals in both human and artificial systems, exploring normalization layers and sigmoid functions in deep learning
  • 10:00Exploring how transformers unified different data modalities including text voice and images, creating common representations through methods like contrastive learning
  • 15:00Nizar explains transformers as brute force learning systems with room for improvement through focused attention mechanisms and knowledge graphs rather than processing everything
  • 20:00Conversation about loss functions local minima versus global minima and how mixture of experts uses specialized small models instead of one massive generalist network
  • 25:00Discussion of deterministic versus probabilistic systems and how explicitly defined task graphs often outperform orchestrator-based approaches in AI systems
  • 30:00Exploring world models as predictive physics-based systems that learn environmental flows and transformations, complementing rather than replacing language models
  • 35:00Nizar discusses real-time processing challenges for robotics requiring millisecond responses with small memory footprints using vision transformers for faster experimentation
  • 40:00Pixel's work creating three d meshes from smartphone cameras for companies like L'Oreal, moving away from specialized sensors toward accessible software-based solutions
  • 45:00Explanation of different three d representations including voxels point clouds and meshes, with meshes being optimal for manipulation and rendering in applications
  • 50:00Future direction involves closing perception-action loops in robotics, moving beyond dancing toy robots toward practical multimodal systems that perform real tasks
  • 55:00Pixel's goal is democratizing high-quality three d scanning through smartphones, making mesh creation accessible to unlock applications in gaming cinema and virtual showrooms
Key Insights
  1. Pixel Robotics derives its name from combining perception and action in robotics, where the pixel represents the digital perception component and robotics represents the physical action component. The pixel serves as a metaphor for how robots must quantize and digitize continuous analog information from the real world into discrete units that computer systems can process, similar to how pixels are the fundamental building blocks of images on a screen. This quantization process is essential because numerical systems cannot work with truly continuous data and must convert reality into tractable digital representations that algorithms can manipulate.
  2. The transformer architecture has created a fundamental unification in how different types of data can be represented and processed across multiple modalities. Before transformers, researchers working on natural language processing, computer vision, and audio analysis used completely different approaches and methodologies. The breakthrough of transformers was establishing a common representational framework that could handle text, images, voice, and other data types using similar underlying mechanisms. This unification is what enabled the development of truly multimodal AI systems and represents one of the most significant advances beyond just the language modeling capabilities that initially gained public attention.
  3. Current transformer-based systems represent a brute force approach to learning that will likely be superseded or enhanced by more efficient algorithms. Despite claims that we have exhausted internet text data for training, significant improvements continue to emerge every few months through algorithmic innovations rather than simply adding more data. Future developments will likely involve more specialized attention mechanisms that focus on relevant information rather than correlating everything with everything, mixture of experts architectures with small specialized models, and approaches inspired by biological systems such as logarithmic compression laws and event-based processing that humans use naturally.
  4. Diffusion-based language models represent a promising alternative to standard next-token prediction that could produce more accurate outputs through an iterative refinement process. Unlike traditional language models that predict one token at a time and cannot revise earlier outputs, diffusion models treat text generation like image denoising, starting with a noisy representation and progressively refining the entire output across multiple steps. This holistic approach allows the model to reconsider and improve all parts of the response simultaneously, potentially leading to higher quality results, though it may be slower than current autoregressive methods. This represents an important direction for overcoming fundamental limitations in how language models currently generate text.
  5. For robotics applications, real-time performance and small model size are critical constraints that differ significantly from the requirements of large language models deployed in data centers. Vision transformers are being used as a testbed for developing efficient real-time algorithms because they require far fewer computational resources to train and test compared to large language models, making them more practical for rapid experimentation. The goal is to achieve millisecond-level response times with minimal memory footprint so that robots can react quickly to dynamic environments and run on affordable hardware that can be embedded in actual robotic systems rather than requiring expensive server infrastructure.
  6. Practical robotics implementation requires moving beyond specialized sensors to software solutions that work with ubiquitous devices like smartphones for tasks such as three-dimensional reconstruction. Pixel Robotics evolved from building specialized scanning hardware to focusing on algorithms that can generate high-quality mesh representations of environments using only smartphone cameras, making the technology far more accessible and practical for real-world deployment. This approach enables applications ranging from industrial robotic arm control to virtual showrooms, and more importantly, it allows anyone to capture three-dimensional data without expensive equipment, which can also help generate larger training datasets for future AI development.
  7. The next frontier in AI and robotics is closing the perception-action loop to enable robots to perform real practical tasks rather than remaining as demonstration systems or toys. While significant progress has been made in cognitive capabilities through language models and in robotic mobility through mechanical engineering advances, the critical challenge is integrating perception with action through systems like Vision-Language-Action models. The fundamental starting point for learning this integration is simple perception-action exercises, such as programming a camera mounted on servo motors to track and center a colored object, which demonstrates the basic principle of using sensory input to drive physical response that underlies all more sophisticated robotic behaviors.
Episode transcript
Stewart Alsop III00:00

Welcome to the Crazy Wisdom podcast. This podcast is for you. If you have an insane drive to find the truth of things, it's not the good answers that we seek, but the good questions. I interview a range of different guests from many different fields, all with the intention to uncover the simple truths that are hidden in plain sight. Most people don't want to go there. I go there, my guests go there, and you benefit. Please let me know if you enjoy these episodes and as always, subscribe on itunes, Spotify, or wherever you listen to the podcasts.

Stewart Alsop III00:35

Welcome to the Crazy Wisdom Podcast. I've got Nizar Horty here and he is the CEO of Pixel Robotics. Welcome to the show.

Nizar Horty00:44

Nice to meet you.

Stewart Alsop III00:47

So what is a pixel?

Nizar Horty00:48

pixel. it's very interesting. In fact, it's a unit that we use on. On the screen. I mean, it's a unit of you, can say, something that could be visually visible, for a person. So you can say like, it's the atom of, of the screen. So the most, I mean the smallest part that you can see. And usually a, pixel could be composed by, red, green or blue. You know, can be combined to show you colors, etc. On your screen.

Stewart Alsop III01:28

M. And I was recently I had the same question, like, what is that for? I think we talked about this in our initial call as well, is that I had this question of like, what is that for the human eye? and like, whether it is the right analogy or metaphor. but before we go into that, why pixel robotics? What about a pixel is related to robotics?

Your first full transcript is free. After that, an email opens every transcript in the index — a list of readers we can write to, not a guest book.

Subscribe