Article · Original
Our first deep dive with an AI data company. In roughly a year, Handshake built an AI training business doing nearly $1 billion in gross annualized revenue. What are frontier labs buying? Garrett Lord joins us with the people building reinforcement learning environments across biology, finance and alignment.
An AI agent filled an Excel tab with every number from one to a million. An early grader found the net-income figure it wanted and awarded a point. What would an investment banker check instead?
Who’s in the room
AI data business — Garrett Lord, Sahil Bhaiwala and Nishant Ranka open and close the episode on the market for expert data.
Life sciences — Nick Magazine and Jonas Mueller put the gel and brain-tissue tasks on screen.
Finance — Collin Schweiker and Sahil Bhaiwala take us through Project Falcon and its Excel grader.
Alignment — Steve Kelling and Nishant Ranka follow the agent’s tool calls through the safety simulations.
Chapters
0:00 Inside Handshake AI
0:39 Garrett, Sahil and Nishant on frontier training data
5:15 How to build a banking environment for an agent
13:07 How frontier labs buy data
15:51 How a lab evaluates a new dataset
20:01 Testing training data on open models
27:38 Where rubrics fall short of expert judgment
31:30 Nick and Jonas introduce the VIALS benchmark
SponsoredYour sponsor ad hereReach the AI engineers building what comes next.Latent Space is read by the people shipping agents, models and infrastructure. Enquire about sponsoring this slot.
business@latent.space →
35:52 Reading molecular structures and plasmid maps
43:48 Training vision models to interpret scientific images
47:17 Task one: reading the bands on a gel
50:59 Task two: finding necrosis in brain tissue
55:49 Can Python tools help the model see?
1:02:52 Can biology models improve on their own?
1:06:19 What a biology lab can buy from Handshake
1:12:33 Atlas Finance with Collin and Sahil
1:18:56 Inside a simulated investment bank
1:25:45 Where the agent’s financial model went wrong
1:31:18 How a banker grades the spreadsheet
1:41:12 Penalty traps, dependency gates and partial credit
1:46:41 How an agent gamed the Excel grader
1:48:52 What would make an AI banker useful?
1:54:48 Safety and alignment with Steve and Nishant
1:58:25 CAREBench and the limits of a refusal
2:04:42 How safety feedback reaches model training
2:07:21 When an agent sends the wrong files
2:11:14 A loan application tests fair lending
2:16:09 Designing private benchmarks for enterprises
2:25:37 The leadership team on Handshake’s mission
2:28:36 When enterprises need their own models
2:33:03 The market for compute and expert data
2:39:12 Why labs hold out evaluation tasks
2:43:01 Hiring at Handshake AI