Q&A: What A.I. Tells Us About Debut Authors
A data scholar reveals publishing's biases and peak TV's impact
Welcome to The Optionist! Thanks for reading, and if you're here because someone forwarded this, I invite you to sign up during our free beta period here. And please reach out at email@example.com to let me know your thoughts, ideas and suggestions. Also, if you aren’t a subscriber to The Ankler, our sibling, check it out!
Starting this week, we’re switching things up by moving to two newsletters a week. Fridays will be for the Optionist’s regular IP picks of the week. On Tuesdays (yes, I know today is Wednesday) we’ll do a variety of interesting things. Some weeks it’ll be a Q&A with an author with a book that’s newsy (like we did with Porter Bibb’s Ted Turner bio a couple of weeks ago). Other weeks it’ll be a deeper dive into one thing (like Scandinavian noir or sports books). We’ll also do some reporting on how the business works. (Some have asked for a piece about the pros and cons of shopping agreements, which we’ll tackle at some point. But send me other suggestions).
Today is a Q&A with Laura McGrath, a scholar at Temple University in Philadelphia who's doing work in the new field of computational literary analysis. Think about it as A.I. to help decision-making and identify blind spots.
In 2019, she wrote for the Los Angeles Review of Books about publishing’s unconscious bias. She tracked adult fiction comps — i.e, this new book is like this older book you liked — that publishers used in their catalogs 2013-2019 and found that among the 500 most-comped books (the top 2 percent), 478 were by white authors. Those comps are an important hook for retail sellers and audience.
Before arriving at Temple, McGrath was the Associate Director of the Literary Lab at Stanford University. She's also working on a book tentatively called Middlemen: Making Literature in the Age of Multimedia Conglomerate about literary agents. (Don’t be shocked: She has a pretty positive view of their work.)
Taking a big picture look at publishing, I asked Laura how computational analysis can make the business of optioning books better.
Q. What is computational literary analysis?
It takes the basic premise that novels, poems or any sort of written text can be analyzed as data using different sorts of computational methods from text mining that looks at things like word frequency up to large-scale statistical models. It enables us to work with not just one text at a time, but with an order of magnitude that's considerably larger — even hundreds of thousands of texts. We can ask really big questions about literary history.
Q. The data you compiled (along with Alexander Manshel of McGill and J. D. Porter of Penn) about debut book deals confirmed something I think many in publishing had long suspected.
What we can see is that debuts as a category are the fastest-growing category [in announcements about book deals]. There's the sense that publishers are willing to bet really big on a debut. That's great if you can earn out your advance. But if you can't earn out your advance, there's this real fear that publishers will drop you, or at least the story goes. This is a little less certain of a claim, but we can then look at those writers who have published debuts from 2004 to the present and see how many of them appear a second time. There's a challenge because books might still be in progress so it's not a definitive claim but we can get a sense of how few writers actually report another deal. It's something under 20 percent of writers who have a debut end up publishing a second book, or at least posting information about a second book. So that attrition is not just a complaint. That's very, very real.
Q. Reliance on debut novels must have some real implications for what gets published.
What we see is authors aren't really given the chance to mature and to find their voice over time. This preference for debuts has meant that writers have to really nail it the first time around because they might not ever be given a chance to mature as a writer and to be able to turn out these gorgeous masterpieces that come with maturity. I think topically that leads us to a different sort of novel being published. Because debut novels are most often written by younger writers these books end up being much more interested in youth, much more interested in the coming-of-age process.
Q. So Hollywood isn't the only one with a youth bias.
One of the other things that's really interesting in the data is how frequently a debut novelist's age is mentioned. Basically if you're over the age of 35, your age never gets mentioned at all, but the use of a debut novelist's age matters a ton. In particular if you're a debut novelist in your twenties, this is a key selling point. It's just so funny to watch it turn. After 35, you're just like persona non grata and definitely not a wunderkind who can be marketed that way.
Q. One thing that really surprised me was the classroom bump a book adapted for the screen got.
We expected to see a bump in Goodreads ratings — that just kind of tracked. What we didn't expect was the relationship between syllabi and adaptation. Using data from the Open Syllabus Project, it was really shocking at how much more likely a book was to be taught if it had been adapted. I think there's a number of reasons why that is. For example, I'm teaching a course this semester on the history of the bestseller and every single book we're reading has been adapted. But it was just really surprising to me to see that connection between adaptation and the university classroom.
Q. Our impulse is to see the classroom and the market as being completely separate.
A lot of scholars have talked about the relationship between academic interest and works that are published. I'm reading a book right now called Market Aesthetics by Elena Machado Sáez that's basically arguing that the academic interest in Caribbean and diasporic literature led to an increased acquisition and promotion of it. That seems to track since one consistent revenue stream is university purchasing. But I didn't expect adaptations would do that. It's one thing to have an award winner from Knopf, it's another thing to have a book that was the big literary adaptation of the year ending up on a syllabus. That's genuinely surprising to me.
Q. How do you think publishers might benefit from the data?
There were articles that came out in the New York Times after George Floyd's murder about the increased desire to be publishing books about race and racism and an increased desire to be publishing writers of color. There's this sense in the publishing industry that everyone collectively wants to do better in regards to diversity. One very easy way that publishers might reflect on whether or not they've put their money where their mouth is would be looking at the data on this — not only thinking about the writers that they're publishing, but also thinking about the content of the books they're creating. The sense from publishers I've talked to that publishing relies primarily on anecdotal evidence and sales data from Bookscan and things like that. The sort of longitudinal data that I'm suggesting is not a part. (Ed note: publishers can get a sense of some the data McGrath has compiled here, here and here.)
Q. This makes me think of the data on comps. I expected some lean to white authors but the amount was really striking to me.
It should shock us into doing something different. Comps are all based on a presumption of eventual sales so if your assumption is only white people are reading and buying books then of course you're not going to put money and energy behind advertising it and promoting [books by authors of color] because you believe that's not going to be worth the returns.
Q. Publishing has made the same mistake that Hollywood has made to imagine the market as primarily white.
Q. Can people in Hollywood learn from the data?
It's kind of a chicken-egg question: The market for adaptation has been great for literary fiction because all of a sudden there's more drive to be publishing really excellent literary fiction. The novel is alive and well. Literature is seeping into culture in ways that are really unprecedented. On the other hand, I think you could say that this might not be great for the novel because the pull is so strong, you might see a sort of homogenizing effect. Where the features that we identified that make a novel highly adaptable, like world building, ensemble casts and episodic plots, [you worry] that's creating a bunch of novels that all look the same. I could understand that sort of worry but that's not what I see. I read novels for a living and I don't see that sort of homogenizing effect occurring.
Q. You're also working on a history of agents.
It's really interesting to me how active a role that agents can play in the development of a project beyond simply just knowing who to pitch to and knowing how to drive a hard bargain. It's really surprising to me how involved they can be in the process. You can start to see the ways in which thinking about the book’s eventual sale can change or shape the way that it is composed. Sometimes that's really great. Sometimes that could be really bad for a writer. I don't think there’s anything nefarious about these relationships. I think they're often really beautiful creative collaborations.
Q. My experience is that some of the best agents are also good editors.
One very practical way that that works, especially for debut writers, is knowing when a manuscript is ready to submit and helping develop a manuscript before submission. Certainly I don't suggest that agents are secret ghost writers or something. But their guidance in knowing at what point a manuscript is ready to go out can play a real role.
Thanks for reading! See you back here on Friday with my weekly list of great material to option. Please check out The Ankler, which the New York Times calls a “hit Hollywood newsletter” if you love the business of entertainment.
On The Ankler:
Richard Rushfield delivers his State of Showbiz address.
The fight is on for the role of the decade. The Transom has the details on the battle to play Madonna in the Material Girl’s self-directed biopic.
Adult animation was supposed to be the streamers’ secret weapon during the pandemic. But things didn’t go quite as planned. Entertainment Strategy Guy breaks down the bitter numbers.
What is Brian Roberts’ secret Peacock plan? Richard Rushfield reads the tea leaves and considers the potential brilliant strategy behind the last-place Streaming Wars contender.
On the Ankler Hot Seat podcast, Epstein survivor Maria Farmer speaks out against HBO's coming Epstein project, Ghislaine Maxwell’s planned appeal, and the writer of the notorious vanishing Vanity Fair story who ended up with deals at Discovery+ and Audible: "She's culpable for the rapes of so many women."
Camera Roll has 20 great photos of Hollywood this week