This Repo is only possible to great work over at RoboPapers and the people who make it: Chris Paxton, Michael Cho.
The podcast is why this repo exists.
We’re Haptic Labs. We work on problems affecting physical AI research. To do so, we are always talking to researchers and engineers to follow the gradient of maximum pain. Since we have always found Robopapers to be a great way to understand the frontier of open source Physical AI space, we collected the transcripts and papers, and used AI to extract both recurring pain points and surprising/counterintuitive takeaways.
We are sharing it for three simple reasons:
-
Robopapers ALREADY did the hard work of sharing and bringing all this to the ecosystem. This is just packaging.
-
We believe strongly in open source and open AI research.
-
In the age of AI, as code gets cheaper and cheaper... the real gift you can give curious, motivated minds is "problems to chew on." We know that is what we are always looking for!
A reasonable question would be "Well, which pain point are you all working on? Are you not worried about competition?"
The short answer is: if you see this, get inspired, and for some reason we end up competing and you eat our lunch, well then that is on us. We should have done better.
The open source mentality is a brutal one, but we embrace it. We certainly do not want to be the weak link that slows down the entire ecosystem!
If you want the highest-signal summary first, these are the recurring pain points we see most often:
- Scalable robot (and human-robot) data collection (22/64) - Collecting high-quality robot data is still slow, expensive, and hard to scale.
- Generalization and zero-shot robustness (12/64) - Policies often fail when objects, tasks, or environments shift beyond training conditions.
- Dexterous and contact-rich manipulation (10/64) - Multi-finger control and force-aware contact handling remain difficult in real tasks.
- Teleoperation and whole-body data collection (10/64) - Current teleop setups are uncomfortable, limited, and hard to scale for whole-body behavior.
- Sim-to-real and simulation environment creation (10/64) - Building useful sims takes major effort, and transfer to real robots is still fragile.
- Evaluation and benchmarking at scale (9/64) - Reproducible real-world evaluation is costly and hard to standardize across labs.
- VLAs, foundation models, and world models for control (8/64) - General-purpose models still struggle with reliability, 3D reasoning, and control alignment.
- Human video / human-to-robot transfer (6/64) - Human demonstrations lack robot-ready actions, dynamics, and embodiment compatibility.
- Long-horizon and memory (6/64) - Most policies are weak on long sequences and memory-dependent decision making.
- RL scaling and offline-to-online (6/64) - Exploration, data efficiency, and pushing reliability toward deployment-grade performance remain open.
For the bigger tiered view, episode-by-episode references, and additional details, see pain_points_summary.md.
Alongside pain points, we now track surprising ideas and counterintuitive findings from each episode:
- Per-episode files:
episodes/<N>/surprises.md - Cross-episode grouped summary: surprises_global.md
If you only want the highest-signal patterns, start with surprises_global.md.
The repo is a mix of manual copy/paste, human edits, AI tweaks, and automation, so there may be errors or AI hallucinations.
If you spot an issue or mistake, please submit a PR with a fix (or open one with suggestions).
One folder per episode (1–64). Each has intro.md, transcript.md, pain_points.md, and surprises.md.
At the repo root: pain_points_global.md (all pain points in one file), pain_points_summary.md (pain-point themes and top opportunities), and surprises_global.md (grouped cross-episode surprises).
See methodology.md for details on how pain points and surprises are extracted, aggregated, and summarized.