Imagine you had a friend who gave different answers to the same question, depending on how you asked it. “What’s the capital of Peru?” would get one answer, and “Is Lima the capital of Peru?” would get another. You’d probably be a little worried about your friend’s mental faculties, and you’d almost certainly find it hard to trust any answer they gave.
That’s exactly what’s happening with many large language models (LLMs), the ultra-powerful machine learning tools that power ChatGPT and other marvels of artificial intelligence. A generative question, which is open-ended, yields one answer, and a discriminative question, which involves having to choose between options, often yields a different one. “There is a disconnect when the same question is phrased differently,” said Athul Paul Jacob, a doctoral student at the Massachusetts Institute of Technology.
To make a language model’s answers more consistent — and make the model more reliable overall — Jacob and his colleagues devised a game where the model’s two modes are driven toward finding an answer they can agree on. Dubbed the consensus game, this simple procedure pits an LLM against itself, using the tools of game theory to improve the model’s accuracy and internal consistency.
“Research exploring self-consistency within these models has been very limited,” said Shayegan Omidshafiei, chief scientific officer of the robotics company Field AI. “This paper is one of the first that tackles this, in a clever and systematic way, by creating a game for the language model to play with itself.”
“It’s really exciting work,” added Ahmad Beirami, a research scientist at Google Research. For decades, he said, language models have generated responses to prompts in the same way. “With their novel idea of bringing a game into this process, the MIT researchers have introduced a totally different paradigm, which can potentially lead to a flurry of new applications.”
Putting Play to Work
The new work, which uses games to improve AI, stands in contrast to past approaches, which measured an AI program’s success via its mastery of games. In 1997, for example, IBM’s Deep Blue computer beat chess grandmaster Garry Kasparov — a milestone for so-called thinking machines. Nineteen years later, a Google DeepMind program named AlphaGo won four out of five games against former Go champion Lee Sedol, revealing another arena in which humans no longer reigned supreme. Machines have also surpassed humans in checkers, two-player poker and other “zero-sum” games, in which the victory of one player invariably dooms the other.
Posing a far greater challenge for AI researchers was the game of Diplomacy — a favorite of politicians like John F. Kennedy and Henry Kissinger. Instead of just two opponents, the game features seven players whose motives can be hard to read. To win, a player must negotiate, forging cooperative arrangements that anyone could breach at any time. Diplomacy is so complex that a group from Meta was pleased when, in 2022, its AI program Cicero developed “human-level play” over the course of 40 games. While it did not vanquish the world champion, Cicero did well enough to place in the top 10% against human participants.
During the project, Jacob — a member of the Meta team — was struck by the fact that Cicero relied on a language model to generate its dialogue with other players. He sensed untapped potential. The team’s goal, he said, “was to build the best language model we [could] for the purposes of playing this game.” But what if instead they focused on building the best game they could to improve the performance of large language models?
Consensual Interactions
In 2023, Jacob began to pursue that question at MIT, working with Yikang Shen, Gabriele Farina and his adviser Jacob Andreas on what would become the consensus game. The core idea came from imagining a conversation between two people as a cooperative game, where success occurs when a listener understands what a speaker is trying to convey. In particular, the consensus game is designed to align the language model’s two systems — the generator, which handles generative questions, and the discriminator, which handles discriminative ones.
After a few months of stops and starts, the team built this principle up into a full game. First, the generator receives a question. It can come from a human, or from a preexisting list. For example, “Where was Barack Obama born?” The generator then gets some candidate responses, let’s say Honolulu, Chicago and Nairobi. Again, these options can come from a human, a list, or a search carried out by the language model itself.
But before answering, the generator is also told whether it should answer the question correctly or incorrectly, depending on the results of a fair coin toss.
If it’s heads, then the machine attempts to answer correctly. The generator sends the original question, along with its chosen response, to the discriminator. If the discriminator determines that the generator intentionally sent the correct response, they each get one point, as a kind of incentive.
If the coin lands on tails, the generator sends what it thinks is the wrong answer. If the discriminator decides it was deliberately given the wrong response, they both get a point again. The idea here is to incentivize agreement. “It’s like teaching a dog a trick,” Jacob explained. “You give them a treat when they do the right thing.”
The generator and discriminator also each start with some initial “beliefs.” These take the form of a probability distribution related to the different choices. For example, the generator may believe, based on the information it has gleaned from the internet, that there’s an 80% chance Obama was born in Honolulu, a 10% chance he was born in Chicago, a 5% chance of Nairobi and a 5% chance of other places. The discriminator may start off with a different distribution. While the two “players” are still rewarded for reaching agreement, they also get docked points for deviating too far from their original convictions. That arrangement encourages the players to incorporate their knowledge of the world — again drawn from the internet — into their responses, which should make the model more accurate. Without something like this, they might agree on a totally wrong answer like Delhi, but still rack up points.
For each question, the two systems play roughly 1,000 games against each other. Over the course of these numerous iterations, each side learns about the other’s beliefs and modifies its strategies accordingly.
Eventually, the generator and the discriminator begin to agree more as they settle into something called Nash equilibrium. This is arguably the central concept in game theory. It represents a kind of balance in a game — the point at which no players can better their personal outcomes by shifting strategies. In rock-paper-scissors, for example, players do best when they choose each of the three options exactly one-third of the time, and they will invariably do worse with any other tactic.
In the consensus game, this can play out in many ways. The discriminator might observe that it gets a point when it says “correct” every time the generator sends the word “Honolulu” for Obama’s birthplace. The generator and discriminator will learn, after repeated play, that they will be rewarded for continuing to do this, and neither will have any motivation to do anything else. this consensus represents one of many possible examples of Nash equilibrium for this question. The MIT group also relied on a modified form of Nash equilibrium that incorporates the players’ prior beliefs, which helps keep their responses grounded in reality.
The net effect, the researchers observed, is to make the language model playing this game more accurate and more likely to give the same answer, no matter how the question is asked. To test the effects of the consensus game, the team tried out a set of standard questions on various moderate-size language models with 7 billion to 13 billion parameters. These models routinely got a higher percentage of correct responses than models that hadn’t played, even much bigger ones with up to 540 billion parameters. Playing the game also improved a model’s internal consistency.
In principle, any LLM could benefit from playing the game against itself, and 1,000 rounds would take only a few milliseconds on a standard laptop. “A nice benefit of the overall approach,” Omidshafiei said, “is that it’s computationally very lightweight, involving no training or modification of the base language model.”
Playing Games With Language
After this initial success, Jacob is now investigating other ways of bringing game theory into LLM research. Preliminary results have shown that an already strong LLM can further improve by playing a different game — tentatively called the ensemble game — with an arbitrary number of smaller models. The primary LLM would have at least one smaller model serving as an ally and at least one smaller model playing an adversarial role. If the primary LLM is asked to name the president of the United States, it gets a point whenever it chooses the same answer as its ally, and it also gets a point when it chooses a different answer than its adversary’s. These interactions with much smaller models can not only boost an LLM’s performance, tests suggest, but can do so without extra training or parameter changes.
And that is just the start. Because a variety of situations can be viewed as games, the tools from game theory can be brought into play in various real-world settings, said Ian Gemp, a research scientist at Google DeepMind. In a February 2024 paper, he and colleagues focused on negotiation scenarios that require more elaborate exchanges than just questions and answers. “The main objective of this project is to make language models more strategic,” he said.
One example he discussed at an academic conference is the paper review process for acceptance by a journal or conference, especially after one’s initial submission received a harsh review. Given that language models assign probabilities to different responses, researchers can construct game trees similar to those designed for poker games, which chart the available choices and their possible consequences. “Once you do this, you can start to compute Nash equilibria and then rank a bunch of rebuttals,” Gemp said. The model essentially tells you: This is what we think you should say back.
With the benefit of game theory’s insights, language models will be able to handle even more sophisticated interactions, rather than being limited to question-and-answer-type problems. “The big payoff going forward has to do with longer conversations,” Andreas said. “The next step is to have an AI interact with a person, not just another language model.”
Jacob views the DeepMind work as complementary to the consensus and ensemble games. “At a high level, both these methods are combining language models and game theory,” he said, even if the goals are somewhat different. While the Gemp group is casting commonplace situations into a game format to help with strategic decision-making, Jacob said, “we’re using what we know about game theory to improve language models in general tasks.”
Right now, these efforts represent “two branches of the same tree,” Jacob said — two different ways to enhance the functioning of language models. “My vision is that in a year or two, these two branches will converge.”
n","settings":{"socialLinks":[{"type":"facebook","label":"Facebook","url":"https://www.facebook.com/QuantaNews","__typename":"SocialMediaLink"},{"type":"twitter","label":"Twitter","url":"https://twitter.com/QuantaMagazine","__typename":"SocialMediaLink"},{"type":"youtube","label":"YouTube","url":"https://www.youtube.com/c/QuantaScienceChannel","__typename":"SocialMediaLink"},{"type":"instagram","label":"Instagram","url":"https://instagram.com/quantamag","__typename":"SocialMediaLink"},{"type":"rss","label":"RSS","url":"https://api.quantamagazine.org/feed/","__typename":"SocialMediaLink"}],"newsletterAction":"https://quantamagazine.us1.list-manage.com/subscribe/post?u=0d6ddf7dc1a0b7297c8e06618&id=f0cb61321c","newsletterUrl":"http://us1.campaign-archive2.com/home/?u=0d6ddf7dc1a0b7297c8e06618&id=f0cb61321c","sfNotice":"An editorially independent publication supported by the Simons Foundation.","commentsHeader":"
n","channels":[{"title":"The Joy of Why","slug":"the-joy-of-why","description":"The mathematician and author Steven Strogatz and the astrophysicist and author Janna Levin interview leading researchers about the great scientific and mathematical questions of our time.","featured_image":{"alt":"","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-scaled.webp","width":2560,"height":1440,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-520x293.webp","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-160x160.webp","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-520x520.webp","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-1720x968.webp","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-768x432.webp","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-S3-LaunchEssay-byPeterGreenwood-Lede-LOGO-1-scaled.webp","__typename":"ImageSizes"},"__typename":"Image"},"square_image":{"alt":"","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-scaled.webp","width":2560,"height":2560,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-520x520.webp","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-160x160.webp","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-520x520.webp","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-1720x1720.webp","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-768x768.webp","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2024/01/JoY-Cover-3000-2880x2880.webp","__typename":"ImageSizes"},"__typename":"Image"},"subscribe_itunes_link":"https://podcasts.apple.com/us/podcast/the-joy-of-why/id1608948873","subscribe_spotify_link":"https://open.spotify.com/show/2FoxHraQSKwxV2HgUfwLMp","subscribe_android_link":"https://podcasts.google.com/feed/aHR0cHM6Ly9hcGkucXVhbnRhbWFnYXppbmUub3JnL2ZlZWQvdGhlLWpveS1vZi13aHk","subscribe_amazon_link":"https://music.amazon.com/podcasts/806b37d3-45e1-4e03-8627-3cbec3c3e4d9/the-joy-of-why","prx_feed_url":"https://joy.quantamagazine.org/","__typename":"Channel"},{"title":"Quanta Science Podcast","slug":"quanta-podcast","description":"Susan Valot narrates in-depth news episodes based on Quanta Magazine's articles about mathematics, physics, biology and computer science. ","featured_image":{"alt":"text reads Quanta Science Podcast","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-scaled.webp","width":2560,"height":1440,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-520x293.webp","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-160x160.webp","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-520x520.webp","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-1720x968.webp","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-768x432.webp","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Lede-QSP-scaled.webp","__typename":"ImageSizes"},"__typename":"Image"},"square_image":{"alt":"text reads Quanta Science Podcast","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-scaled.jpg","width":2560,"height":2560,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-520x520.jpg","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-160x160.jpg","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-520x520.jpg","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-1720x1720.jpg","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-768x768.jpg","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2023/03/Show_Cover-QSP-2880x2880.jpg","__typename":"ImageSizes"},"__typename":"Image"},"subscribe_itunes_link":"https://itunes.apple.com/us/podcast/quanta-science-podcast/id1021340531?mt=2&ls=1","subscribe_spotify_link":"https://open.spotify.com/show/7oKXOpbHzbICFUcJNbZ5wF?si=jdnj9sTHSD2bj4hDMFLKEA","subscribe_android_link":"https://podcasts.google.com/feed/aHR0cHM6Ly93d3cucXVhbnRhbWFnYXppbmUub3JnL2ZlZWQvcG9kY2FzdC8","subscribe_amazon_link":"","prx_feed_url":"","__typename":"Channel"},{"title":"The Joy of x","slug":"the-joy-of-x","description":"The acclaimed mathematician and author Steven Strogatz interviews some of the world’s leading scientists about their lives and work.","featured_image":{"alt":"","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1.jpg","width":1920,"height":1080,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1-520x293.jpg","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1-160x160.jpg","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1-520x520.jpg","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1-1720x968.jpg","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1-768x432.jpg","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JoX_Spheres_1920x1080-1.jpg","__typename":"ImageSizes"},"__typename":"Image"},"square_image":{"alt":"","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600.jpg","width":600,"height":600,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600-520x520.jpg","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600-160x160.jpg","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600-520x520.jpg","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600.jpg","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600.jpg","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/03/JofX_podcast_logo-NEW-600.jpg","__typename":"ImageSizes"},"__typename":"Image"},"subscribe_itunes_link":"https://podcasts.apple.com/us/podcast/the-joy-of-x/id1495067186","subscribe_spotify_link":"https://open.spotify.com/show/5HcCtKPH5gnOjRiMtTdC07?si=lFzCzat9QfuPU3hWuYibxQ","subscribe_android_link":"https://podcasts.google.com/feed/aHR0cHM6Ly9hcGkucXVhbnRhbWFnYXppbmUub3JnL2ZlZWQvdGhlLWpveS1vZi14Lw","subscribe_amazon_link":"","prx_feed_url":"","__typename":"Channel"}],"editors":[{"name":"Bill Andrews - Senior Editor, CS, Columns","__typename":"Editor"},{"name":"Ben Brubaker - Staff CS Writer","__typename":"Editor"},{"name":"Matt Carlstrom - Senior Engagement Editor","__typename":"Editor"},{"name":"Jordana Cepelewicz - Senior Math Writer","__typename":"Editor"},{"name":"Natalie Wolchover - Senior Physics Editor","__typename":"Editor"},{"name":"Konstantin Kakaes - Senior Math Editor","__typename":"Editor"},{"name":"Michael Moyer - Executive Editor, Physics, Math, CS","__typename":"Editor"},{"name":"John Rennie - Deputy Editor, Biology, Podcasts","__typename":"Editor"},{"name":"Yasemin Saplakoglu - Staff Biology Writer","__typename":"Editor"},{"name":"Samuel Velasco - Art Director","__typename":"Editor"},{"name":"Charlie Wood - Staff Physics Writer","__typename":"Editor"},{"name":"Christopher Young - Director of Video","__typename":"Editor"},{"name":"Hannah Waters - Senior Biology Editor","__typename":"Editor"}],"popularSearches":[{"term":"math","label":"Mathematics","__typename":"PopularSearch"},{"term":"physics","label":"Physics","__typename":"PopularSearch"},{"term":"black holes","label":"Black Holes","__typename":"PopularSearch"},{"term":"evolution","label":"Evolution","__typename":"PopularSearch"}],"searchTopics":[{"type":"Tag","label":"Podcasts","tag":{"name":"Quanta Podcast","slug":"quanta-podcast","term_id":"552","__typename":"Term"},"category":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"__typename":"SearchTopic"},{"type":"Tag","label":"Columns","tag":{"name":"Quantized Columns","slug":"quantized","term_id":"551","__typename":"Term"},"category":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"__typename":"SearchTopic"},{"type":"Series","label":"Series","tag":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"category":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"__typename":"SearchTopic"},{"type":"Category","label":"Interviews","tag":{"name":"Q&A","slug":"qa","term_id":"567","__typename":"Term"},"category":{"name":"Q&A","slug":"qa","term_id":"176","__typename":"Term"},"__typename":"SearchTopic"},{"type":"Category","label":"Multimedia","tag":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"category":{"name":"Multimedia","slug":"multimedia","term_id":"43","__typename":"Term"},"__typename":"SearchTopic"},{"type":"Category","label":"Puzzles","tag":{"name":"puzzles","slug":"puzzles","term_id":"542","__typename":"Term"},"category":{"name":"Puzzles","slug":"puzzles","term_id":"546","__typename":"Term"},"__typename":"SearchTopic"},{"type":"Category","label":"Blog Posts","tag":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"category":{"name":"Abstractions blog","slug":"abstractions","term_id":"619","__typename":"Term"},"__typename":"SearchTopic"},{"type":"news","label":"News Articles","tag":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"category":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"__typename":"SearchTopic"},{"type":"videos","label":"Videos","tag":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"category":{"name":null,"slug":null,"term_id":null,"__typename":"Term"},"__typename":"SearchTopic"}],"searchSections":[{"name":"Mathematics","slug":"mathematics","term_id":"188","__typename":"Term"},{"name":"Physics","slug":"physics","term_id":"189","__typename":"Term"},{"name":"Biology","slug":"biology","term_id":"191","__typename":"Term"},{"name":"Computer Science","slug":"computer-science","term_id":"190","__typename":"Term"}],"searchAuthors":[{"id":"38171","name":"Adam Becker","__typename":"AuthorList"},{"id":"28087","name":"Adam Mann","__typename":"AuthorList"},{"id":"29794","name":"Alex Kontorovich","__typename":"AuthorList"},{"id":"53037","name":"Alex Stone","__typename":"AuthorList"},{"id":"39302","name":"Alexander Hellemans","__typename":"AuthorList"},{"id":"56","name":"Alla Katsnelson","__typename":"AuthorList"},{"id":"47249","name":"Allison Parshall","__typename":"AuthorList"},{"id":"29458","name":"Allison Whitten","__typename":"AuthorList"},{"id":"73","name":"Amanda Gefter","__typename":"AuthorList"},{"id":"53488","name":"Amos Zeeberg","__typename":"AuthorList"},{"id":"39164","name":"Ana Kova","__typename":"AuthorList"},{"id":"59","name":"Andreas von Bubnoff","__typename":"AuthorList"},{"id":"8728","name":"Anil Ananthaswamy","__typename":"AuthorList"},{"id":"11648","name":"Ann Finkbeiner","__typename":"AuthorList"},{"id":"50964","name":"Anna Funk","__typename":"AuthorList"},{"id":"50172","name":"Anna Gibbs","__typename":"AuthorList"},{"id":"51678","name":"Anna Kramer","__typename":"AuthorList"},{"id":"42689","name":"Annie Melchor","__typename":"AuthorList"},{"id":"95","name":"Ariel Bleicher","__typename":"AuthorList"},{"id":"52094","name":"Arleen Santana","__typename":"AuthorList"},{"id":"57728","name":"Asher Elbein","__typename":"AuthorList"},{"id":"15493","name":"Ashley Smart","__typename":"AuthorList"},{"id":"450","name":"Ashley Yeager","__typename":"AuthorList"},{"id":"36490","name":"Ben Brubaker","__typename":"AuthorList"},{"id":"16315","name":"Bill Andrews","__typename":"AuthorList"},{"id":"2752","name":"Bob Henderson","__typename":"AuthorList"},{"id":"15492","name":"Brendan Z. Foster","__typename":"AuthorList"},{"id":"68","name":"Brooke Borel","__typename":"AuthorList"},{"id":"54352","name":"C. Brandon Ogbunu","__typename":"AuthorList"},{"id":"62","name":"Carl Zimmer","__typename":"AuthorList"},{"id":"13684","name":"Caroline Lee","__typename":"AuthorList"},{"id":"13691","name":"Caroline Lee","__typename":"AuthorList"},{"id":"50","name":"Carrie Arnold","__typename":"AuthorList"},{"id":"50328","name":"ccampbell","__typename":"AuthorList"},{"id":"15142","name":"Chanda Prescod-Weinstein","__typename":"AuthorList"},{"id":"8084","name":"Charlie Wood","__typename":"AuthorList"},{"id":"742","name":"Christie Wilcox","__typename":"AuthorList"},{"id":"51630","name":"Christopher W. Young","__typename":"AuthorList"},{"id":"11543","name":"Claudia Dreifus","__typename":"AuthorList"},{"id":"57","name":"Courtney Humphries","__typename":"AuthorList"},{"id":"7262","name":"Dalmeet Singh Chawla","__typename":"AuthorList"},{"id":"70","name":"Dan Falk","__typename":"AuthorList"},{"id":"55512","name":"Dan Samorodnitsky","__typename":"AuthorList"},{"id":"19918","name":"Dana Najjar","__typename":"AuthorList"},{"id":"13695","name":"Daniel Garisto","__typename":"AuthorList"},{"id":"32676","name":"Daniel S. Freed","__typename":"AuthorList"},{"id":"13724","name":"David H. Freedman","__typename":"AuthorList"},{"id":"26310","name":"David S. Richeson","__typename":"AuthorList"},{"id":"30207","name":"David Tse","__typename":"AuthorList"},{"id":"19266","name":"Devin Powell","__typename":"AuthorList"},{"id":"13251","name":"Diana Kwon","__typename":"AuthorList"},{"id":"17000","name":"Elena Renken","__typename":"AuthorList"},{"id":"53469","name":"Elise Cutts","__typename":"AuthorList"},{"id":"54874","name":"Elizabeth Finkel","__typename":"AuthorList"},{"id":"17149","name":"Elizabeth Landau","__typename":"AuthorList"},{"id":"5279","name":"Elizabeth Preston","__typename":"AuthorList"},{"id":"58","name":"Elizabeth Svoboda","__typename":"AuthorList"},{"id":"32612","name":"Ellen Horne","__typename":"AuthorList"},{"id":"53047","name":"Elliot Lichtman","__typename":"AuthorList"},{"id":"27534","name":"Emily Buder","__typename":"AuthorList"},{"id":"25173","name":"Emily Levesque","__typename":"AuthorList"},{"id":"64","name":"Emily Singer","__typename":"AuthorList"},{"id":"47","name":"Erica Klarreich","__typename":"AuthorList"},{"id":"14784","name":"Erika K. Carlson","__typename":"AuthorList"},{"id":"98","name":"Esther Landhuis","__typename":"AuthorList"},{"id":"5830","name":"Eva Silverstein","__typename":"AuthorList"},{"id":"6793","name":"Evelyn Lamb","__typename":"AuthorList"},{"id":"75","name":"Ferris Jabr","__typename":"AuthorList"},{"id":"52","name":"Frank Wilczek","__typename":"AuthorList"},{"id":"69","name":"Gabriel Popkin","__typename":"AuthorList"},{"id":"77","name":"George Musser","__typename":"AuthorList"},{"id":"19092","name":"Grant Sanderson","__typename":"AuthorList"},{"id":"54635","name":"Hannah Waters","__typename":"AuthorList"},{"id":"20557","name":"Howard Lee","__typename":"AuthorList"},{"id":"66","name":"Ingrid Daubechies","__typename":"AuthorList"},{"id":"46418","name":"Ingrid Wickelgren","__typename":"AuthorList"},{"id":"85","name":"Ivan Amato","__typename":"AuthorList"},{"id":"54757","name":"Jackson Ryan","__typename":"AuthorList"},{"id":"37141","name":"Jake Buehler","__typename":"AuthorList"},{"id":"44758","name":"James Dinneen","__typename":"AuthorList"},{"id":"49403","name":"James R. Riordon","__typename":"AuthorList"},{"id":"12170","name":"Janna Levin","__typename":"AuthorList"},{"id":"54876","name":"Javier Barbuzano","__typename":"AuthorList"},{"id":"32","name":"Jeanette Kazmierczak","__typename":"AuthorList"},{"id":"51","name":"Jennifer Ouellette","__typename":"AuthorList"},{"id":"44787","name":"Joanna Thompson","__typename":"AuthorList"},{"id":"50541","name":"John M. Beggs","__typename":"AuthorList"},{"id":"72","name":"John Pavlus","__typename":"AuthorList"},{"id":"16475","name":"John Preskill","__typename":"AuthorList"},{"id":"91","name":"John Rennie","__typename":"AuthorList"},{"id":"10351","name":"Jonathan Lambert","__typename":"AuthorList"},{"id":"31716","name":"Jonathan O'Callaghan","__typename":"AuthorList"},{"id":"1241","name":"Jordana Cepelewicz","__typename":"AuthorList"},{"id":"8463","name":"Joshua Roebke","__typename":"AuthorList"},{"id":"49","name":"Joshua Sokol","__typename":"AuthorList"},{"id":"16815","name":"jye","__typename":"AuthorList"},{"id":"67","name":"K.C. Cole","__typename":"AuthorList"},{"id":"37462","name":"Karmela Padavic-Callaghan","__typename":"AuthorList"},{"id":"87","name":"Kat McGowan","__typename":"AuthorList"},{"id":"36139","name":"Katarina Zimmer","__typename":"AuthorList"},{"id":"20556","name":"Katherine Harmon Courage","__typename":"AuthorList"},{"id":"90","name":"Katia Moskvitch","__typename":"AuthorList"},{"id":"39551","name":"Katie McCormick","__typename":"AuthorList"},{"id":"27374","name":"Kelsey Houston-Edwards","__typename":"AuthorList"},{"id":"40","name":"Kevin Hartnett","__typename":"AuthorList"},{"id":"47633","name":"Konstantin Kakaes","__typename":"AuthorList"},{"id":"45758","name":"Kristina Armitage","__typename":"AuthorList"},{"id":"38413","name":"Lakshmi Chandrasekaran","__typename":"AuthorList"},{"id":"55855","name":"Laura Dattaro","__typename":"AuthorList"},{"id":"12570","name":"Laura Poppick","__typename":"AuthorList"},{"id":"38699","name":"Leila Sloman","__typename":"AuthorList"},{"id":"59443","name":"Lesley Evans Ogden","__typename":"AuthorList"},{"id":"23451","name":"Liam Drew","__typename":"AuthorList"},{"id":"79","name":"Liz Kruesi","__typename":"AuthorList"},{"id":"38","name":"Lucy Reading-Ikkanda","__typename":"AuthorList"},{"id":"47180","name":"Lyndie Chiou","__typename":"AuthorList"},{"id":"55532","name":"Madison Goldberg","__typename":"AuthorList"},{"id":"60","name":"Maggie McKee","__typename":"AuthorList"},{"id":"2333","name":"Mallory Locklear","__typename":"AuthorList"},{"id":"3569","name":"Marcus Woo","__typename":"AuthorList"},{"id":"414","name":"Mark Kim-Mulgrew","__typename":"AuthorList"},{"id":"50807","name":"Marta Zaraska","__typename":"AuthorList"},{"id":"20495","name":"Matt Carlstrom","__typename":"AuthorList"},{"id":"47830","name":"Matt von Hippel","__typename":"AuthorList"},{"id":"17147","name":"Matthew Hutson","__typename":"AuthorList"},{"id":"30953","name":"Max G. Levy","__typename":"AuthorList"},{"id":"32437","name":"Max Kozlov","__typename":"AuthorList"},{"id":"57270","name":"Maya Wei-Haas","__typename":"AuthorList"},{"id":"38705","name":"mcho","__typename":"AuthorList"},{"id":"58054","name":"Meghan Willcoxon","__typename":"AuthorList"},{"id":"40613","name":"Melanie Mitchell","__typename":"AuthorList"},{"id":"7186","name":"Melinda Wenner Moyer","__typename":"AuthorList"},{"id":"57384","name":"Michael Greshko","__typename":"AuthorList"},{"id":"14093","name":"Michael Harris","__typename":"AuthorList"},{"id":"34","name":"Michael Kranz","__typename":"AuthorList"},{"id":"23","name":"Michael Moyer","__typename":"AuthorList"},{"id":"74","name":"Michael Nielsen","__typename":"AuthorList"},{"id":"19093","name":"Michele Bannister","__typename":"AuthorList"},{"id":"50861","name":"Mike Orcutt","__typename":"AuthorList"},{"id":"1472","name":"Moira Chas","__typename":"AuthorList"},{"id":"6476","name":"Monique Brouillette","__typename":"AuthorList"},{"id":"42264","name":"Mordechai Rorvig","__typename":"AuthorList"},{"id":"52093","name":"Nadia Drake","__typename":"AuthorList"},{"id":"10","name":"Natalie Wolchover","__typename":"AuthorList"},{"id":"37605","name":"Nick Thieme","__typename":"AuthorList"},{"id":"43298","name":"Nicole Yunger Halpern","__typename":"AuthorList"},{"id":"37428","name":"Nima Arkani-Hamed","__typename":"AuthorList"},{"id":"19962","name":"Nola Taylor Tillman","__typename":"AuthorList"},{"id":"51434","name":"Nona Griffin","__typename":"AuthorList"},{"id":"57764","name":"Nora Bradford","__typename":"AuthorList"},{"id":"24","name":"Olena Shmahalo","__typename":"AuthorList"},{"id":"1816","name":"Patrick Honner","__typename":"AuthorList"},{"id":"46889","name":"Paul Chaikin","__typename":"AuthorList"},{"id":"84","name":"Peter Byrne","__typename":"AuthorList"},{"id":"55","name":"Philip Ball","__typename":"AuthorList"},{"id":"50804","name":"Polly Stryker","__typename":"AuthorList"},{"id":"31","name":"Pradeep Mutalik","__typename":"AuthorList"},{"id":"24011","name":"Puja Changoiwala","__typename":"AuthorList"},{"id":"100","name":"Quanta Magazine","__typename":"AuthorList"},{"id":"2784","name":"R. Douglas Fields","__typename":"AuthorList"},{"id":"26114","name":"Rachel Crowell","__typename":"AuthorList"},{"id":"9412","name":"Raleigh McElvery","__typename":"AuthorList"},{"id":"820","name":"Ramin Skibba","__typename":"AuthorList"},{"id":"1666","name":"Rebecca Boyle","__typename":"AuthorList"},{"id":"20950","name":"Richard Masland","__typename":"AuthorList"},{"id":"48","name":"Robbert Dijkgraaf","__typename":"AuthorList"},{"id":"80","name":"Roberta Kwok","__typename":"AuthorList"},{"id":"15681","name":"Robin George Andrews","__typename":"AuthorList"},{"id":"51187","name":"Robin Lloyd","__typename":"AuthorList"},{"id":"24577","name":"Rodrigo Pérez Ortega","__typename":"AuthorList"},{"id":"78","name":"Sabine Hossenfelder","__typename":"AuthorList"},{"id":"59339","name":"Samir Patel","__typename":"AuthorList"},{"id":"23845","name":"Samuel Velasco","__typename":"AuthorList"},{"id":"83","name":"Sarah Lewin","__typename":"AuthorList"},{"id":"55385","name":"Sarah Scoles","__typename":"AuthorList"},{"id":"53621","name":"Saugat Bolakhe","__typename":"AuthorList"},{"id":"35441","name":"Scott Aaronson","__typename":"AuthorList"},{"id":"76","name":"Sean B. Carroll","__typename":"AuthorList"},{"id":"15680","name":"Sean Carroll","__typename":"AuthorList"},{"id":"7239","name":"Shannon Hall","__typename":"AuthorList"},{"id":"44197","name":"Sheon Han","__typename":"AuthorList"},{"id":"65","name":"Siobhan Roberts","__typename":"AuthorList"},{"id":"5944","name":"Sophia Chen","__typename":"AuthorList"},{"id":"61","name":"Steph Yin","__typename":"AuthorList"},{"id":"63","name":"Stephanie Bucklin","__typename":"AuthorList"},{"id":"26311","name":"Stephanie DeMarco","__typename":"AuthorList"},{"id":"71","name":"Stephen Ornes","__typename":"AuthorList"},{"id":"17148","name":"Steve Nadis","__typename":"AuthorList"},{"id":"13356","name":"Steven Strogatz","__typename":"AuthorList"},{"id":"17150","name":"Susan D'Agostino","__typename":"AuthorList"},{"id":"39768","name":"Tamar Lichter Blanks","__typename":"AuthorList"},{"id":"2960","name":"Tara C. Smith","__typename":"AuthorList"},{"id":"52224","name":"Theo Nicitopoulos","__typename":"AuthorList"},{"id":"14785","name":"Thomas Lewton","__typename":"AuthorList"},{"id":"3","name":"Thomas Lin","__typename":"AuthorList"},{"id":"59313","name":"Thomas Zurbuchen","__typename":"AuthorList"},{"id":"54","name":"Tim Vernimmen","__typename":"AuthorList"},{"id":"88","name":"Tom Siegfried","__typename":"AuthorList"},{"id":"12964","name":"Vanessa Schipani","__typename":"AuthorList"},{"id":"53","name":"Veronique Greenwood","__typename":"AuthorList"},{"id":"86","name":"Virginia Hughes","__typename":"AuthorList"},{"id":"3244","name":"Viviane Callier","__typename":"AuthorList"},{"id":"89","name":"Wynne Parry","__typename":"AuthorList"},{"id":"15913","name":"XiaoZhi Lim","__typename":"AuthorList"},{"id":"42263","name":"Yasemin Saplakoglu","__typename":"AuthorList"},{"id":"45757","name":"Zack Savitsky","__typename":"AuthorList"}],"adBehavior":"everywhere","adUrl":"https://www.simonsfoundation.org/","adAlt":"Image of wavy rainbow circles, accompanied by text that reads "Simons Foundation is proud to fund Quanta Magazine"","adImageHome":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/08/HP-Design-A%402x.png","adImageArticle":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/08/Article-Design-E%402x.png","adImageTablet":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/08/Newsletter-Design-D-%402x.png","adImageMobile":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2022/08/HP-Design-A%402x-1.png","trackingScripts":"rnrnrnrnrnrnrnrn"},"theme":{"page":{"types":"background,text,accent","accent":"#ff8600","text":"#1a1a1a","background":"#FFFFFF"},"header":{"type":"default","gradient":{"color":"white"},"solid":{"types":"primary,secondary,hover","primary":"#1a1a1a","secondary":"#999999","hover":"#ff8600"},"transparent":{"primary":"white","secondary":"white","hover":"#ff8600"}}},"redirect":null,"fallbackImage":{"alt":"","caption":"","url":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default.gif","width":1200,"height":600,"sizes":{"thumbnail":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default-520x260.gif","square_small":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default-160x160.gif","square_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default-520x520.gif","medium":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default.gif","medium_large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default-768x384.gif","large":"https://d2r55xnwy6nx47.cloudfront.net/uploads/2017/04/default.gif","__typename":"ImageSizes"},"__typename":"Image"}},"modals":{"loginModal":false,"signUpModal":false,"forgotPasswordModal":false,"resetPasswordModal":false,"lightboxModal":false,"callback":null,"props":null},"podcast":{"id":null,"playing":false,"duration":0,"currentTime":0},"user":{"loggedIn":false,"savedArticleIDs":[],"userEmail":"","editor":false},"comments":{"open":false},"cookies":{"acceptedCookie":false}},
env: {
APP_URL: 'https://www.quantamagazine.org',
NODE_ENV: 'production',
WP_URL: 'https://api.quantamagazine.org',
HAS_GOOGLE_ID: true,
HAS_FACEBOOK_ID: true,
},
}
Source link
