There is a blue ocean of opportunity for AI systems that observe and act at a higher frequency than what most chatbots are capable of today. Visible light occupies a small sliver of the electromagnetic spectrum, which spans radio waves, mic…
Safety implications of geared motors 1X released a teaser video of NEO Beta to the world yesterday. You may notice that despite its agility, NEO moves quietly. If you turn up the volume on the video, you can hear the gentle whirring of moto…
ARC Prize is an intelligence benchmark intended to be hard for AI and easy for humans. With a grand prize of $1,000,000, it is currently the most popular contest on Kaggle. Most AI systems (e.g. GPT4) perform much worse than human children
All AI software will converge to looking like robotics software 1. A read/write API to physical reality 2. A source of training tokens about The Self Two years ago I left Google Robotics and wrote about the various career options that I con…
Visual Foundation Models still need robot data to work well on robots. News at 11. DeepMind Robotics published the paper “RoboCat: A Self-Improving Foundation Agent for Robotic Manipulation (PDF)” earlier this week. I read the 50-page repor…
GPT-4 can display rudimentary self-reflection on whether its generated outputs make sense. Avi Singh told me yesterday about a recent arxiv preprint, Reflexion, that proposes the following idea: use a LLM to examine whether the output of an…
Comparing the fields of generative modeling and robotic deep learning, and what we might do to speed up the pace of robotics research. I recently gave a talk (YouTube) at the RSS’22 L-DOD workshop. Here’s a lightly edited transcript and sli…
In the future, every successful tech company will use their data moats to build some variant of an Artificial General Intelligence. Plus some updates on my career. I was on the job market recently, open to suggestions on what I should do ne…
I made a neural network, YCRank, trained it on a handful of hand-labeled pairwise comparisons between YC companies, and then used it to sort the companies in the most recent W'22 batch. There are 397 companies in the recent YC 2022 Winter b…
In machine learning and optimization, it’s important to inject some noise into the optimization process to prevent local maxima. Taking that advice to heart, I’m applying some exploration noise into my life - this is my last week at Google
We all want ML models to generalize better, but defining "generalization" is hard. I suggest that the structure of language is the structure of generalization. If language models also capture the underlying structure of generalization vis-à…
Generalizing to what you want may be easier than optimizing directly for what you want. We might even ask for "consciousness". This blog post outlines a key engineering principle I’ve come to believe strongly in for building general AI syst…