Does Flapping Airplanes Mean Short Timelines and Hard Treaty Enforcement?
The massive valuation increase for 'Flapping Airplanes' highlights investor reliance on hype rather than verified breakthroughs in sample efficiency.
Decoder
- Sample Efficiency: The ability of an AI model to learn from smaller amounts of data to achieve the same or better performance.
- Chinchilla Laws: Empirical findings relating the optimal number of training parameters and the amount of training data to the compute budget.
Original article
Does Flapping Airplanes mean short timelines and hard treaty enforcement?
I wrote this on August 31 when the raise was reported as a rumor. But I got no good answers then, so I'm reposting.
Flapping Airplanes, a neolab led by twentysomethings with a pitch of "we'll invent AI algorithms that are many OOMs more sample-efficient," raised at a valuation of $1.5B in January. They are now in talks to raise at $5B, from both the $1.5B-round lead (Index Ventures, a prominent generalist firm) and a well-regarded deeptech firm (Lux Capital). (COI note: the latter invested in my company.) (EDIT Oct 22: their pitch has drifted more toward "new primitives for GPUs" that enable "more $ per MW." So possibly efficient inference more than efficient training.)
A naive interpretation is: insiders have increased their estimation of FA's success (in creating vastly more sample-efficient AI) 3.3x. My best guess is that this would be justified (explicitly) by "they've hired more smart people" and "they claim technical progress," with perhaps a dash of "I would have invested earlier if they'd let me." Any technical progress is, from what Fable and I can find out, more or less secret. (Everything here is a mix of public info and reliable anonymous sources.)
Reasons to temper that naive interpretation
- "Recruitment of elite talent" can be a reason in itself, beyond increasing probability of success–––investors to whom I've spoken have insisted there's an "acquihire floor" not too far below [whatever the most recent valuation at the time is]. That floor gets higher as you get a bigger and harder-to-replicate team.
- Prices in the VC market are far from rational. Most VC firms have sub-stock-market performance for super-stock-market risk and illiquidity. The high-performing VCs can best be modeled as brand rent extractors. Recently, Instinct, with a product still in private beta, raised a Series A at a valuation of $0.5B and then raised its Series B at $2.5B. The investors at the second price were rushing to "get in," whatever the value. The investors at the first price were, mostly, better-connected. Not much changed about the company in those 3 weeks, other than its extensive public promotion by Series A investors. In short, VCs are not pricing, except in vague vibey terms, off (P(success) * E[payoff | success]).
- Flapping Airplanes is a particularly prominent example of prices decoupling from business fundamentals. They raised 4 rounds before the first publicly documented one, starting with a valuation of $40M. At each turn, there was a deal (in at least several cases explicit) that the main value add of the current VC was pumping the bag to the next buyer. It definitely helped, before, that all rounds were secret, but they could be doing it again! The founders are extraordinarily well-connected; their babysitter was Jeff Dean. One investor told me that he invested because the CEO was so good at fundraising. The element of adverse selection here can be outweighed by confidence that the fundraising ability will carry it to at least a "greater fool."
- $5B is the 9Xth percentile privately believed valuation, not the 50th percentile private valuation (as should roughly be true for large-cap public stocks) because ≈nobody can sell stock. Though so was $1.5B. So 3.3x isn't the 9Xth percentile increase, but the increase of the 9Xth percentile. Since over time uncertainty should decrease, you might expect this to imply that the median increased by a greater factor, but I would disagree–––conditional on having a 3.3x increase in the 9Xth percentile outcome, uncertainty has probably increased. Still, if you posit some fraction of gullible people, then perhaps Flappy could be bid up this high on a fugazi. The secrecy of their innovations helps: the investors evaluating this deal might be forbidden from sharing technical proof with anyone who could audit it for the smell of BS.
- "The category" (of AI lab) has gotten massively bid up, and revenue has taken off past expectations, which could lead investors to increase their estimates of the size of the prize.
- If it'll be so darn sample-efficient, you wouldn't need so many FLOPs, would you? This one's a little snarky, but in the extremes there's something to it. Surely when doing "sample-efficient AI research" the ratio of researchers to compute should be higher than elsewhere!
Things I could learn that would untemper this
- Credible (i.e. backed by technical understanding and minimal personal interests) claims that they have made meaningful technical progress toward sample-efficient AI.
- Existing shareholders broadly declining chances to sell in a secondary offering and/or buying more.
- The same thing happening at other neolabs, particularly ones that haven't had hype-raises.
How much should it be tempered?
An Mx increase in Flappy's odds of success implies an Nx increase in the odds of our getting OOMs-more-sample-efficient AI in the next (say) 10 years, with N < M. I think strongly that we should believe 1 < M < 3.3 and 1 < N, but I'm not sure what to set as M, let alone N.
Directional implications
From a brutish, scaling-oriented perspective: many OOMs of sample efficiency should (holding compute equal) catapult us forward in effective scaling. Moreover, it makes it more likely, later on, that a single "rogue actor" could develop comprehensively superhuman machine intelligence without a large GPU footprint, making treaties harder to enforce.
I see AI research right now as much more compute-bottlenecked than data-bottlenecked. Yes, I know they trade off against each other, but my impression–––correct me if I'm wrong!–––is that current LLMs are being trained with the optimal (for LLMs) quantity of data to minimize usage of compute. Yes, they're pouring lots of money into higher-quality data, but it's pennies next to the money spent on compute.
From this perspective, then, it has the same effect as some quantum of "algorithmic improvement," depending on how the data and compute trade off against each other in this fancy new architecture. (I haven't run the numbers on how e.g. a 1,000,000x sample-efficiency improvement would play out with Chinchilla laws, because I ought to be working right now, but someone should.)
From a more qualitative perspective: Sample efficiency is a commonly cited difference between humans/AGI and current AIs! If AIs get continual learning, it will need to be sample-efficient to be truly transformative. This should not only pull forward timelines, but also shorten the gap between "the last AI humans built" and "the first comprehensively superhuman AI." Folks who are less alignment-doomer than I am may draw some encouragement from the second, depending on their theories.
So. What's it all mean? Am I just being distracted by a shiny object? Or is this a big deal?
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi