Joan Henry vs the Algorithm

By Randy Lubin


The following story was first published in the Working Futures anthology which is available on Kindle and on Amazon. This story is published under a Creative Commons CC-BY license.


Joan Henry spun in a whirl of frenetic gestures, using dozens of interfaces to shape stories and – hopefully – save her and her teammates’ jobs. She moved within a small booth, less than an arms’ length across. Its touch-screen walls formed her office, and she had precisely customized them to her needs. At the moment, she was the Maestro for three different clients, orchestrating each of their narratives simultaneously.

Client 6 was imploring the Eldritch Queen to intervene in the war and fight alongside his people. As he spoke, Joan guided the Queen’s non-verbal reactions: listening attentively but visibly skeptical. Joan queued up a rebuttal for the Prince to deliver and then she turned down the audio.

She preferred focusing on just one client at a time but, since the last round of layoffs, two or three had become the norm. Joan had stayed up late preparing for today’s challenge: reorganizing her libraries of visual assets and character Agents. Her tools were all at hand, carefully arrayed across the booth and easy to grab at a moments notice. No matter what a client’s tastes or background, she was ready to craft the perfect narrative.

Joan turned her attention to Client 7, who had been wandering the spaceports’ sprawling market, stocking up on supplies, and preparing for her next expedition. In her previous session, Client 7 had rescued a transport vessel from pirates and in gratitude they’d given her a map to an ancient alien city. Joan prepped the visual assets for the planetary approach and flyby of the ruined city at sunrise. She also readied some personal questions for this ship’s navigator to ask Client 7 – the client loved chatting about her character’s backstory.

As the spaceship departed, Joan turned to Client 8’s screens. The four of them had been sneaking and fighting their way through a wizard’s tower for most of the session and were nearing the final showdown. Client 8 was a group of close friends, living across three continents, and they commissioned monthly sessions as a way to stay close. Joan had been their maestro before and had always received glowing reviews. She had their preferences memorized and could easily set each player up for their own moment of glory.

They bust into the wizard’s inner sanctum and Joan raised a shimmering barrier between them and the wizard. Then she manually took over the wizard character, using the booth’s sensors to map her expressions onto the powerful mage. She launched into a monologue, taunting the heroes and revealing the wizard’s evil plans.

Until the previous year, Maestros would directly control all of the supporting characters. These days, character algorithms – Agents – were good enough for most situations. Joan only directly controlled a character during pivotal scenes or when the Agents couldn’t grasp the nuances of a client’s intent. In those cases, the library of voice modulators and micro-expressions gave her the dynamic range to master any role.

Flashing lights interrupted her – alerts from her screens tracking Client 6. Joan brought the speech to a quick close and dropped the barrier, kicking off the final battle with the heroes. The combat AI could handle it from there.

Back in the court of the Eldritch Queen, Client 6 had interrupted the Prince and challenged him to a duel. Joan scanned the client’s background and biometrics: his heart rate and blood pressure were spiking. For another client, that might be a sign that they were thrilled and anticipating the fight but Joan knew that Client 6 hated combat; he must have challenged the Prince out of desperation. Joan took over the Eldritch Queen and intervened before the duel could start. She improvised a few lines about admiring the client’s courage and vowed to help him and his people. Almost immediately, Client 6’s heart rate began dropping. He expressed his gratitude and Joan guided his session to an end – he’d be back next week to pick up the story from here.

With Client 7 engrossed in space travel and Client 8 mid-battle, Joan had a moment of peace. She thought she was doing a solid job but wasn’t sure if it was good enough. Joan was the best Maestro at her company and her remaining colleagues were counting on her.

#

Joan had been outrunning the algorithm her entire adult life, re-skilling and transitioning across myriad roles ahead of their automation. She’d seen countless friends and colleagues succumb, unable or unwilling to adapt to the shrinking labor pool. Not Joan – she’d survived thanks to marathon study sessions, endless networking, and a knack for knowing the right time to switch to the next hot micro-profession. She cherished her identity as a professional – hard working, creative, empathetic, and flexible. Living off basic income was not an option; she would not become another economic dropout.

When Joan started as a Maestro, she thought she’d be safe for at least a few years. The synthesis of real time storytelling, worldbuilding, and acting for a live audience seemed far too complex for any algorithm to master. Just eight months later and her job was already on the line.

Joan had been collaborating with simple algorithms from the start – generators that created towns and ecosystems, consumes and character models; agents that could handle basic character behaviors and interactions; and gleaners that helped her comb through clients’ profiles and histories to find emotionally resonant themes and motifs. All of these had improved steadily, making her better and more efficient at her job.

Joan’s company was owned by Lara Talcott, who had built a small but highly regarded firm of Maestros in an oversaturated market. When clients wanted to experience gripping narratives that were perfectly catered to them, Lara was their first stop. But competition was fierce and margins were thinner. Lara was always looking for ways to cut expenses. In the short time Joan had been with the company, Lara had replaced both the customer support and character modeling teams with algorithms.

The previous week, Lara had called all the Maestros to a meeting. She told the team that a major vendor had approached her, boasting about a new algorithm that could weave stories just as well as a human Maestro and at a fraction of the cost.

The Maestros were blindsided but also skeptical. Orchestrating a story took tremendous skill and nuance and they felt that an algorithm couldn’t be at a human level yet. Joan had heard similar protests from friends in past jobs and every single time they’d been wrong. Still, this did seem like a huge leap forward and – unlike previous times – she didn’t have a lead on another profession to re-skill.

“Let me go head-to-head against their algorithm – I know I can craft a better experience and our clients will recognize it.” Lara was happy to arrange a challenge, her best Maestro against the vendor’s new algorithm tested against some of the firms most discerning clients.

The week passed quickly. Joan had a full load of clients and she spent her spare time searching for new assets and tools to use during the competition – anything that would give her an edge. The work had paid off already and she felt more confident juggling multiple clients than she had in the past.

#

Clients 7 and 8 wrapped up their sessions around the same time and seemed to have enjoyed themselves. Their sign-off left Joan with a short window to freshen up before the next clients signed on.

Joan staggered out of the dimly lit booth, coated in sweat. The booth occupied the corner of her living room and went nearly to the ceiling. She caught her breath and spent a moment adjusting to the bright daylight entering through the room’s broad windows.

Her roommates Allison and Riley were sitting on the couch, eating quinoa and salad from their co-op’s cafeteria. She gave them a half-wave and hustled past them. They didn’t mind the booth taking up so much space – it was owned by Joan’s company but they were allowed to use it whenever she was off duty and it was far nicer than anything they could afford on their own.

When Joan returned from the bathroom, Allison was waiting with a large glass of iced water and a cool, damp towel. Joan wiped off her face and chugged the water.

“Are you winning?” Allison asked.

“Feeling good – I’ve just wrapped up two of the best stories I’ve ever orchestrated. One of my clients wept for the first time in years and the other left us a 20% bonus.”

“Awesome – we believe in you. Keep at it and stay hydrated. Do you need anything else?”

Joan shook her head glanced down at her watch, “No – I think I’m okay. Gotta get back to it… the algorithm isn’t taking any breaks.”

Riley interrupted from the couch, “We have a bunch of stims left over from last night’s party – I brewed them myself and they kept us going until dawn. Want some?”

“Thanks but I’m good,” Joan shook smiled and shook her head. Riley often brewed drugs from plans she found online and then remixed. Ingredients were easy to obtain, typically through barter, and the results were generally safe, most of the time. Joan steered clear; she didn’t have time to deal with the consequences of a bad batch.

Still, she regretted missing the party. Her co-op was full of creative, kind-hearted friends and she didn’t see them nearly as much as she’d like. She had moved in for the community as much as for the cheap rent, but her increasing workload left her little time to enjoy their company. That didn’t stop a steady flow of invites, not just to parties but to creative collaborations, meditation retreats, philosophical salons, and more.

Her roommates meant well and she enjoyed their company, but she also pitied them. They’d dropped out of the workforce years ago, through earlier waves of automation. Unlike Joan, they hadn’t had the grit or motivation to retrain and leap to the next opportunity. Basic income and universal healthcare kept them happily unemployed, but if those safety nets dissolved… Joan shuddered at their potential vulnerability. Their art projects and community endeavors were lovely but they weren’t developing marketable skills and the co-op fees were steep enough that they couldn’t be saving money.

Joan heard a ringing from the booth and said a quick goodbye to her roommates before jumping back in. Lara was calling.

#

“Hey Joan, I just wanted to give you an update on your progress.”

“Yeah?” Joan fiddled with her interfaces as she waited to hear the news.

“Your numbers were amazing and several of your clients have already committed to coming back.”

“That’s great…” said Joan happy with the feedback but worried about the competition, “and the algorithm?”

Lara nodded, “Also getting great ratings. It’s spinning out arcs and characters unlike anything we’ve seen before… utterly novel and resonating deeply with the clients. The pacing seems near perfect, too; it seems to be reading the biometrics and keeping the client in a peak state of engagement and flow.” She shook her head slightly, “It’s still early but… good luck Joan.”

Joan felt that Lara wanted her to win, wanted a clear reason not to fire another round of employees. But Lara put her business first and Joan knew that she’d do anything to protect the bottom line.

#

Hours blurred by as Joan guided client after client through complex narratives. She wove tragedies and comedies, each carefully crafted to maximize emotional impact on the particular client. Each client came with a detailed dossier which expanded well beyond their intake survey. She had access to detailed psychological evaluations, taste graphs, and more – assembled though algorithmic analysis of a client’s online behavior over the course of their entire life.

Joan had a knack for sifting through the dossier and finding the perfect elements for a client’s experience. She drew on beloved characters from their childhood, first crushes from their adolescence, and pivotal moments from their adult lives. They wept, they rejoiced, they found new meaning in their personal journeys… and they loved her for it.

It was early evening when Lara interrupted out again.

“Joan – you are doing an amazing job… but it’s a close thing. The algorithm is, well, it’s like it’s gotten inside our clients’ heads. One of our clients just claimed to have had a spiritual awakening.”

“It’s not over yet.” Joan said, though lacking her earlier confidence.

“No, but you’ll have to do even better.”

#

Joan left the booth and reentered the living room. Allison was on the couch focused on the 3D model of their building that was projected on the wall in front of her, but with familiar hallways and common spaces overlaid with tombstones, spiderwebs, and phantom holograms. The co-op was making a haunted house for Halloween and Allison was leading the set design. Joan had volunteered to design the narrative arcs but her work had erased her free time.

Allison looked up when she heard Joan enter. She closed the model and turned to her friend. “I was wondering when you’d step out. Here, I brought you some food from the potluck.” She handed Joan a lukewarm plate of glass noodles and veggies. “Are you okay? You look super worn down.”

“Umm, I’ll be fine. Gotta get back in there!” Joan spoke between slurps of noodles. “Still a long way to go and I’m not going to lose.” She looked around the living room. “Did Riley leave the stims? I’m going to bring them in, just in case”

“Yeah, they’re in that jar but… are you sure you’re okay? Maybe you can take a nap?”

“No time, the algorithm doesn’t sleep and neither will I.” Joan handed the plate back and grabbed the pills off the table. She was halfway into the booth when Allison spoke up.

“Take this, too” She said, handing Joan a thermos.

“More caffeine?”

“Miso soup.”

“Thanks” Joan said, and turned back toward the booth.

“We’re going to keep checking in on you but… Just be careful, don’t push yourself too hard.” But Joan had already closed the door.

#

Joan was immediately back in the flow – orchestrating three clients at once and the booth’s screens lit up in a mosaic of tabs and readouts.

As the hours wore on, her exhaustion began to show. She started making small mistakes: a minor contradiction in worldbuilding, an extraneous subplot that undercut the theme, pushing a client a little too far from their comfort zone. Joan spotted the errors after the fact and, though her clients might not have noticed them, she knew that she was subtly diminishing their experiences.

Joan took a deep breath and leaned back against the booth’s door. She viewed the pill bottle with blurry eyes, weighing the tradeoffs. A moment later, she swallowed two pills and washed them down with now-tepid broth.

The stims hit almost immediately as crisp energy coursed through her body and the fogginess cleared from her mind. She was alert, excited, and ready to win. She jumped back into narratives with renewed vigor.

Soon Joan was juggling five clients at once – unheard of for a high-end maestro – and she was pulling it off. Her eyes and fingers jumped from screen to screen as she massaged multiple stories at once. Her left hand guided a character’s reactions to a client’s emotional plea while her right hand restructured the beats of a different client’s upcoming quest.

The pills contained more than just stimulants and soon Joan felt a deeper empathy with her clients. She wept with Client 19, when they rescued their child; she raged with Client 23 when his closest ally betrayed him. She drew inspiration from the personal stories of her friends and community, adapting their struggles and successes to resonate with her client’s unexpressed needs.

It was well past midnight when the pills began to wear off. Joan felt at once jittery and fatigued and she resumed making subtle mistakes. Without much conscious deliberation, she grabbed the jar, swallowed a handful of pills, and rinsed them down with a gulp from the thermos – now full of hot mint water that Allison must have brought her sometime in the night.

Her jitters remained but the stims forced out any hint of fatigue. She upped her client load to seven and disappeared into the work. Joan was operating on pure instinct, her characters and worlds extensions of herself. She had no idea if her stories were good but she knew they felt right, felt beautiful and harmonious.

Her deepened empathy now extended to her character agents and she began arranging subplots to benefit them, narrative arcs that her clients would never see. Beyond mere stories, she was designing communities so that the agents would be happy and fulfilled even when she wasn’t present.

#

Hours passed. A face appeared in the upper right and Joan repeatedly failed to open a related dossier or tweak the background. After a few tries, she realized it was Lara calling in and not another client.

“Joan… Joan, are you paying attention?” Lara’s face seemed to project stress and anxiety but Joan automatically tried and failed to pull up her biometrics. “Listen. The algorithm screwed up big-time and we had to shut it down. We’re calling off the challenge, you’ve won.”

“Umm” Joan mumbled as she manipulated a dozen character agents for clients in other windows.

“Joan, did you hear me? Are you okay? Joan?”

Joan’s vision went dark and she crumpled to the floor.

#

Joan awoke in a small hospital room with an IV tube in her arm. Allison and a few other friends from the co-op sat nearby, engrossed in a tablet that sat between them.

“Hey,” Joan rasped.

“You’re awake!” Allison took off her glasses and gave Joan a hug. Her smile quickly turned to anger. “You scared us! I got a call from Lara in the middle of the night and found you unconscious in the booth. You could have died!”

Joan averted her eyes for a moment and took a string of small sips from a bedside glass of water. After a deep breath, she returned Allison’s stare. You’re right – I was reckless and short sighted and I screwed up. I’m sorry.”

“That okay – we were all just really worried. Lara, too – she’s been pinging me for updates non-stop. I’ll let everyone know you’re okay.” Allison dashed off a few quick notes.

Joan sat up and looked out the window to see the city skyline at sunset. “What time is it?”

“Almost dinner – you were out all day but we’ve been here keeping an eye on you.”

The hours of rest had helped her recover from the effects of the drug and sleep deprivation but a deeper fatigue remain. Her memories of the previous night were a jumble of faces and narrative fragments. Which were her clients’ characters and which were character agents? Were her clients actually thrilled with her experiences? Was winning this challenge worth risking her health?

She had poured all of her energy and creativity into defending her independence and dodging algorithmic unemployment. The past decade of her life had been an unending marathon toward the same ends. The relentless focus on employability was meant to protect her from the uncertainty of a fickle government and precarious safety net. But all of her exertions led her to this hospital bed and, if not for the universal healthcare, the medical bills would have wiped out her savings. Lara, as kind as she was, could never have afforded health care for her staff.

Joan looked over at Allison and her other friends, knowing that they had been sitting by her side the entire day. Pestering the nurses, no doubt, to make sure she got the best treatment.

Allison’s tablet rang. “It’s Lara – should I answer?”

“Yeah – pass me the tablet,” Joan popped in earbuds and answered the call.

Lara’s was initially concerned with Joan’s health but once she realized her employee was going to be fine, she changed the topic to the challenge.

“It was a fiasco.” Lara said, “the algorithm was incredibly inventive but deeply flawed. It pursued intensity of experience with little concern for our clients’ psychological safety. I don’t think it had any sense for why the clients were enjoying the experience. Peak engagement was just another basket of numbers to optimize – heart rate, pupil dilation, galvanic skin response – but without intuition for when those numbers signaled enjoyment versus stress and anxiety.

“It drove multiple clients to suffer panic attacks and even PTSD flashbacks. The algorithms a blackbox so we don’t know why it made those choices; it may have been just grabbing random topics from their dossiers, with no clear understanding of context. We pulled the plug as soon as we realized what was happening but it wasn’t fast enough and we’re just hoping the clients end up okay. Obviously, it’s too early to start replacing Maestores with algorithms.

“Look. I feel absolutely awful that you ended up here. Take all the time you need – paid leave of course – and when you come back you’re getting a bonus and I want you to officially mentor the rest of the Maestros.”

Joan processed Lisa’s offer and tried to picture getting back to work. It felt like a stay of execution – the algorithms were always improving and there would be stressful cost-cutting to come. She had won this round but how long until the next one? A better algorithmic Maestro felt inevitable... and then what – another retraining, another cutthroat job hunt for a gig that might not last a year?

Joan glanced over at her friends, who had spent the day by her side, waiting for her to wake up. They had built a strong, nourishing community. Even if the government programs went away they would figure out a way to survive, together. Joan turned back to Lara.

“Thanks” Joan said, with a weak smile, “but I don’t think I’ll be coming back.”

Lara opened her mouth to protest but Joan shook her head and continued. “Good luck with everything, but I’m ready to try a new narrative.”

#


Much thanks to Mike Masnick for including me in the Working Futures anthology and to Avital, Chris, Dustin, Raph, Hannu, and Mike for providing feedback on this story and helping me refine it!