The Problem Graph Had a Long Reading List

The problem-graph post we published recently argued that the missing infrastructure for R&D is a map organized by function rather than field. The argument didn’t arrive from nowhere. Most of it is already sitting in one R&D director’s library — 605 audiobooks, 320,090 embedded chunks of text — and we decided to go look.

So we queried it. Twelve themes from the problem-graph argument — Deutsch’s “reach” of good explanations, Hayek’s local knowledge, exaptation, the adjacent possible, specialists missing solutions already found in other fields — embedded with the same model used to index the books, scored against every chunk, aggregated to the book level. What came back was not twelve unrelated books. It was one continuous argument, handed forward across roughly seven decades, that the frontier of knowledge is a coordination problem first and a discovery problem second.

Here is that argument as a reading arc.

Polanyi: the knowledge we can’t write down

The oldest ancestor in the stack is Michael Polanyi’s Personal Knowledge (1958), and it is still the most surgical. His central claim is that the working scientist operates on tacit skill. Connoisseurship, he writes, “can be communicated only by example, not by precept.” To become a medical diagnostician or a wine taster, “you must go through a long course of experience under the guidance of a master.” That sentence is why a mail-order Difference Engine would never have worked in the nineteenth century, and why, seventy years after Polanyi, an embedding model trained on patent abstracts cannot on its own tell you that catheter fouling and marine biofouling are the same problem. The decisive judgment lives in a head.

Postrel: knowledge is local

Virginia Postrel’s The Future and Its Enemies (1998) carries Polanyi’s tacit knowledge forward into Hayek. Tacit knowledge, she writes, is a special case of “local knowledge, the things people ‘on the spot’ know that are specific to time, place, and circumstance.” Her Steinway example lands the point: the skin of a small Brazilian deer is critical to covering a particular part of the piano’s hammer shank. Not a trade secret. Not classified. Just something that is never where someone outside that building would know to look. Most of what matters about any complex object is like this.

Hidalgo: the personbyte, and the shape of the network around it

César Hidalgo’s Why Information Grows (2015) turns the Polanyi–Postrel observation into a quantity. Knowledge and knowhow, he argues, “always need to be physically embodied” in humans and networks of humans — and humans are finite. He calls the per-human cap the personbyte. Complex products exist only because networks of people cooperate to hold more knowhow than any one of them can. The implication is sharp: discovery is bounded by the shape of the network that holds it, not by the cleverness of any node.

The more interesting question — and the one Hidalgo spends the back half of the book on — is what shape should those networks take? His answer begins with Coase. Firms exist wherever internal coordination is cheaper than going to the market; they stop growing the moment that calculus inverts. And then Hidalgo extracts the lever: “the cheaper the link, the larger the network.” Reduce the cost of a single link between specialists and you expand the maximum feasible size of a knowhow-holding organization. The problem graph, in this frame, is a link-cost reduction — it tries to make connecting a neurosurgeon to a marine-coatings chemist as cheap as connecting two people inside the same firm.

Link cost alone doesn’t determine output, though. Hidalgo leans on AnnaLee Saxenian’s comparison of Silicon Valley with Route 128 around Boston. Both had dense concentrations of technical talent. Only one kept producing new firms. In the Valley, “functional boundaries within firms are porous … as are the boundaries among firms and between firms and local institutions.” Route 128 was dominated by “autarkic corporations” governed by “practices of secrecy and corporate loyalty.” Same personbytes; radically different topology; radically different output. The frontier gets more explorable when networks are porous at the boundaries — between fields, between firms, between labs and industries.

Trust is the substrate that makes this possible. “Trust enables networks, but networks also enable trust,” Hidalgo writes, citing Putnam: the Rotary Clubs, the Boy Scouts, the Freemasons are all instances of civic infrastructure that catalyze the dense non-kin networks inside which specialists can later collaborate across boundaries. You cannot decree a problem-graph community into existence; it grows where repeated small interactions build social capital first.

And finally, the product-space result. Countries diversify into adjacent products, never random ones: a nation that exports mangoes next starts exporting bananas, not motorcycles. Rearranging knowhow networks at the frontier is not a free design problem. You have to move to neighbors in skill topology. The problem graph’s operational job is to show where the neighbors actually are — so that the next combination is reachable rather than fictional.

Summed up, Hidalgo’s prescription for usefully-shaped frontier networks is four-fold: lower the cost of links between specialists; keep organizational boundaries porous; seed trust through thick civic density; route diversification through genuine adjacencies rather than wishes.

Deutsch: reach comes from being hard to vary

Then David Deutsch enters with the bigger claim. Some explanations, he argues in The Beginning of Infinity (2011), have reach — they work outside the context that produced them — and some don’t. The distinction turns on whether an explanation is hard to vary.

“Demeter did it” is a simple story about where seasons come from, but every detail is arbitrary. Swap Demeter for any other god, relocate the myth from Greece to Norway, change what Demeter was angry about — the story still works. Easy to vary, zero reach.

Compare the modern account: Earth’s axial tilt means one hemisphere receives more sunlight at a time of year. Change any piece — the tilt angle, the orbital geometry, the optics — and the explanation collapses. And precisely because it is hard to vary, it already applies on Mars, which also has axial tilt, and in every solar system we will ever visit. Deutsch’s crucial point is that reach is not something we extrapolate to. It is contained in the content of the explanation itself: “the reason the explanation of seasons reaches far outside the experience of its creators is precisely that it does not have to be extrapolated.”

This is the real philosophical backbone of cross-domain discovery. If an explanation of how bacteria form a biofilm on a ship’s hull actually captures the mechanism — surface chemistry, boundary-layer flow, protein adsorption — then it is already, automatically, an explanation of how bacteria form a biofilm on a neurosurgical catheter. The reach was baked in the day it was written. The only question is whether anyone in medicine notices the paper was already written in the shipping literature.

Deutsch adds two consequences that matter for the problem graph. First, humans are “universal constructors,” able in principle to transform anything into anything the laws of nature allow — which means the ceiling on cross-domain transfer is very high and the floor very messy. Second, systems of knowledge undergo a “jump to universality” when incremental improvements cross a threshold: alphabets, the genetic code, Turing machines, the Arabic numerals. A working problem graph would itself be a candidate for that kind of jump, because it makes every hard-to-vary explanation in any field discoverable from any other field. A problem graph is infrastructure for noticing the reach that was already there.

Wootton and Dartnell: reach is built, not given

David Wootton’s The Invention of Science (2015) and Lewis Dartnell’s The Knowledge (2014) argue that this reach is not automatic. It took institutions. Wootton locates the beginning of modern science not in the telescope but in “a sociological reality — the scientific network,” enabled by the printing press and the sudden social reality of priority disputes. Dartnell arrives at the same point in reverse, through a post-apocalyptic thought experiment: the most profound problem for survivors, he writes, is that “human knowledge is collective, distributed across the population.” No individual foundry technician knows enough to keep a foundry going. The iceberg “extends unseen through both space and time.”

Johnson: exaptation and the adjacent possible

Steven Johnson’s Where Good Ideas Come From gives the engineering mechanism. Feathers evolved for thermoregulation in Cretaceous dinosaurs, then got repurposed for flight — the biologists’ term is exaptation. Apple’s 1979 visit to Xerox PARC is exaptation of the graphical user interface out of a lab that couldn’t ship it. YouTube, Johnson notes, was built by stitching together three earlier platforms: the Web, Adobe Flash, and JavaScript. The pattern generalizes. Every new platform is a stack of old platforms put to adjacent uses, and “the history of life and human culture, then, can be told as the story of a gradual but relentless probing of the adjacent possible.”

Meadows: the borders are wrong

Donella Meadows, in Thinking in Systems, names the failure mode with unusual bluntness. “The right boundary for thinking about a problem rarely coincides with the boundary of an academic discipline, or with a political boundary.” Rivers are terrible administrative borders; air is worse. So are field boundaries. When a functional problem — fouling, spurious coherence, thermal dissipation, redundancy detection — lives at the seam between two fields, both fields miss it. The universities, she notes drily, tend to become “living monuments to boundaries that got drawn in the nineteenth century.”

Gleick: the Noah’s Ark rule

James Gleick’s The Information (2011) supplies the counter-example. Warren McCulloch’s Macy conferences in post-war New York ran under “a Noah’s Ark rule, inviting two of each species so that speakers would always have someone present who could see through their jargon.” Cybernetics, information theory, and neuroscience got their common vocabulary in that room. The Macy conferences were, in miniature, a problem graph — a deliberate seating chart designed so that a neurophysiologist and a mathematician could not hide behind disciplinary vocabulary.

Plurality: the data now agrees with Polanyi

The closing book in the arc is ⿻ Plurality (E. Glen Weyl, Audrey Tang, et al., 2024), which summarizes the empirical metascience literature. The finding is blunt: decentralized scientific communities made up of mostly independent, non-overlapping teams — drawing on a broad spectrum of prior work — produce more reliable results than centralized ones with repeated collaborations and a narrow methodological palette. What Polanyi suspected in 1958, what Postrel argued in 1998, Weyl and Tang now back with the data.

What the problem-graph post adds

Pull all of that together and what the library has been arguing — for sixty-seven years, in ten voices — is that the rate of human discovery is capped by the geometry of who talks to whom. Tacit. Local. Personbyte-limited. Embodied in networks. Sometimes reach-extending, sometimes not. Always mis-bounded by the disciplines we happen to have inherited. The problem-graph post took this reading list as given and asked a new question: if the bottleneck is coordination, what does the file system look like? What’s the API between Hausmann’s finite brains and Johnson’s infinite combinations?

Deutsch explains why reach across domains is possible. Hausmann explains why it is scarce. Postrel tells you what shape knowledge actually has. Meadows tells you why the disciplines keep drawing the wrong borders around it. Johnson tells you how new things get made when those borders are ignored. Plurality supplies the measurement.

The thirteenth book in the top set, the one that surfaced against the “serendipity and discovery” query, was Laura Snyder’s The Philosophical Breakfast Club — four Victorian friends who invented the word scientist over breakfast at Cambridge and then started coordinating meteorological observations across an empire. The word was new in 1833. The problem wasn’t. It still isn’t.

If your own library scores this strongly against a problem-graph reading, it has probably been trying to tell you something for a while.

Method note. The index: 605 audiobooks, 320,090 text chunks, each embedded with Ollama’s nomic-embed-text model and stored locally. Twelve thematic queries (drawn from the problem-graph argument) were embedded with the same model, scored against every chunk by cosine similarity, and aggregated to the book level by mean-of-top-5 chunks. The arc above quotes the single highest-scoring chunk from each featured book; no passage was included that did not surface naturally from the query. Full per-theme ranking for all 605 books is in the repository report.