Hi! I'm an ML Principal at Salesforce. Previously founding ML engineer at Convergence where I created our first web agents and developed our OS web agent proxy lite. Before that, ML engineer at Cohere where I started the Command project and led the creation of our SOTA reward models.
Pipeline parallelism wastes GPUs on idle time and ships gradients across the slowest link in the system. What if each rack just predicted the next token from its own layers instead of doing the full-stack backward pass? On its own it hurts, but a few private decoder layers per rack fix that. It moves an order of magnitude less data, never stalls, and at 13B it trains a better model than the standard method on the same budget.
70% of what a ViT token gets back from attention is information it already has. I measure this across patch sizes, show it compounds through layers, and test a projection that removes it.
Why I think Coding Agents are the future of agentic computer use.
A browser-based editor for making retro games with Pyxel, a Python retro game engine.
Finding the physics of water bending.
Learning from state changes in 1 million Python programs.