Tribeca Immersive Diary 2021 (Updated 6/18/21)

No Proscenium’s staff including Noah Nelson, Kathryn Yu, Asya Gorovits, and Edward Mylechreest are covering the 2021 Tribeca Film Festival’s Immersive Programming for us.

This festival diary will update through the event with notes, thoughts, and capsule reviews through the end of the fest.

This year’s Tribeca Immersive is a beast. There are several site-specific installations at Tribeca’s Virtual Arcade venue in NYC, a number of projects available anywhere via their own iOS and Android apps, a live theatrical performance happening on VR Chat, an outdoor concert, soundwalks in lower Manhattan, as well as a dozen projects available via PC VR in the Museum of Other Realities. Did you get all that?

That’s thirty projects in total. Thirty. And we’re going to do our best to discuss as many as we can.

New to the Museum of Other Realities? Here’s a pro-tip on getting around: Immediately after entering the white arches, make a pit stop at the “How to Use This Museum” area on your left. There’s a real time museum map that shows all the explorable areas and which users are currently inside the venue.

Use the teleport mechanism (left thumbstick) directly onto the map to warp-tube yourself to the “XR Virtual 2021”section which houses content from Tribeca, Cannes XR, and NewImages. The Tribeca wing will be on your right.

(You’ll know you’re in the right place if you see the Unity sculpture in front of you.)

Other tips:

• Press Escape to exit out of an experience and return to the Museum.
• While you’re in the Museum, you can also change your settings by hitting Shift + H.
• After pressing Shift + H, use the “power” button on the left to quickly exit the Museum altogether.

June 18, 2021

CURRENT (Outdoor & Interactive Experiences, NYC): The Financial District of Manhattan is an ever busy part of town, crowded with tourists and workers; it’s also surrounded by water on three sides. Annie Saunders and Andrew Schneider, the two narrators and the co-creators of CURRENT, invite the audience to join them for an hour-long soundwalk through its streets: all you need is a charged phone and a pair of headphones. The tour starts in Zuccotti Park every half hour. The one-and-a-half-mile-long route takes the listeners down Broadway, past the New York Stock Exchange, and through a labyrinth of winding streets and alleys, some of which are so narrow you could easily miss them otherwise.

Obscure historical facts are intertwined with the authors’ peculiar and poetic observations about the surroundings. Just like the city blocks we are strolling through, the narrative reveals multiple layers of local and personal histories. Did you know that multiple burial grounds of free and enslaved Africans are found here, where skyscrapers are built? Or that Pearl Street once was the natural edge of Manhattan Island? CURRENT starts out as an unconventional city-centered audio tour and gradually builds up to be a philosophical reflection on urban development and the precariousness of human biology.

There is no one single map that someone can follow during CURRENT as all the directions are given verbally, which presents some challenges at a couple of pivotal points. Occasionally, I exchange questioning glances with a stranger who was clearly also listening to CURRENT in tandem, and we are able to navigate as a collective. This act doesn’t feel burdensome or annoying, however; if anything, it felt a bit like a treasure hunt. “Find a ‘no parking sign’ and stand next to it,” says Annie’s friendly voice. And soon I’m scanning the environment with intensity and attention to the minute details of the cityscape. It’s a focus I can rarely afford when running errands in this part of town.

CURRENT’s listeners are encouraged to follow the pace of Annie’s footsteps in order to not to fall behind during the experience. Normally, I consider myself a fast walker but this time I wasn’t able to always keep up to the narration. Luckily, there are a few rest stops along the way, during which we get to hear fragments of interactions and interviews with the inhabitants and visitors of Manhattan. But I wished the pauses that followed the dialogue fragments weren’t completely dead. Hearing nothing but silence for a few seconds after a conversational moment made me compulsively check my phone to make sure that I was still connected to the experience.

I have a similar thought around the ending of the experience. The audio track unfortunately stops rather abruptly, right after we complete our loop and arrive back at Zuccotti Park; it’s a choice that leaves the listeners of CURRENT feeling somewhat adrift. — Asya Gorovits

Paper Birds Pts 1 & 2 (Museum of Other Realities): In Paper Birds, the story of a little boy and his sister who has gone missing unfolds as a series of highly detailed, beautiful dioramas in front of your eyes. As in, literally in front of your eyes… by which I mean way too close to your face. I had trouble figuring out where I was supposed to “be” and where I was supposed to be standing for much of this experience. Sometimes I would discover I was unable to see the main action because the set or camera had swiveled itself into an obstructed view behind a pillar or a tree. Other times, I would get a face full of foliage or a building spire in my eye because the camera in Paper Birds moves around of its own accord to an almost dizzying amount. (I was especially taken out of the moment on more than a few occasions since I was certain I was going to collide with a character.) And the dioramas containing the various scenes also twist and turn on their own, meaning the experience is often at odds with the kind of agency of focus or agency of perspective that viewers have come to expect from many other 6 DOF VR pieces.

The viewer in this piece also does not have a clearly defined role or part to play in the story. You’re an invisible giant who isn’t present in the plot… for most of the time but not always, for some unexplained reason. On occasion, your hands become visible in VR for a moment and you’re prompted to move around your controllers in aid of the narrative. Perhaps the interaction leads to trails of particles floating behind you in the night sky. Other times you’re shining a light down on the ground or opening a portal to another world in relation to the action. Why you have these powers in Paper Birds is never explained and the audience participation aspects feel tacked on, as opposed to core to the experience.

Don’t get me wrong, this experience is gorgeous. The characters and dollhouse-sized sets are exquisite. The sound design and music are exemplary, particularly for a VR piece. And the animations and special effects are breathtaking. However, I’m not certain virtual reality is actually the best format for this piece given the amount of camera movement out of the viewer’s control and lack of a specific audience role.

Honestly, I’d rather see a lot of this beautiful imagery from Paper Birds on the big screen, leaning back in a legit movie theater, not set a few inches from my face. — Kathryn Yu

The Passengers: The Kid (Museum of Other Realities): In The Passengers, viewers have the choice of experiencing the same train ride from one of four perspectives: four people who happen to be sitting together on a passenger train. There’s a man, a woman, an elderly woman, and a child. They’re not traveling “together” and yet they are somehow connected? At least that’s my assumption. This is the third chapter in a piece that showed at SXSW earlier this year.

Previously, I’d viewed “the man” and “the woman” chapters and found much of the same kind of content that I saw here with “the kid.” The man was struggling with his ability to hit on women; the woman was struggling with whether or not to have a child with her partner; and the kid is blaming himself for his parents separating. You as the viewer see the events on the train from the perspective of your chosen “passenger” and as you observe the other travelers around you and the changing scenery, you hear the inner monologue of that character. Sometimes, your character flashes back to a memory (shown as a 360 degree video) which helps give more context to the scene.

The interaction design of the experience has evolved since SXSW but I found the new interactions added to the story to be problematic. At one point in the story, you as the kid make eye contact with one of the adult characters in an attempt to make them smile. Five large dots appear on the screen in the shape of a semicircle; you’re meant to trace a curve using your (invisible) hands. Not only is this difficult because you can’t see your controllers in virtual reality while the headset is on (meaning there’s no feedback to the user), I felt quite uncomfortable literally tracing a smile on someone else’s face. It felt like I was treating the other characters like puppets and removing their agency and consent by forcing them to smile.

The Passengers is also “lightly” branched in that on occasion, you can hear slightly different content depending on which of the other three characters you happen to be gazing at at the time. Unfortunately, like the previous two chapters, the third chapter suffers from the same flawed base assumption: that the inner monologue of this particular person is actually interesting. — Kathryn Yu

WarpSound (Live performance, NYC): Promising a first-of-its-kind, audience-interactive live music experience, my hopes were high for the sunny concert on the Brookfield Place waterfront in lower Manhattan. With procedurally generated AI music being performed by onscreen AI characters (including “Dragoon,” an energy drink-addled, half-iguana DJ), the audience would be able to “collaborate” to alter the music live with our smartphones. Certainly a promising set up for a live concert experience, I thought.

However, the character animation quality seemed to be equivalent to an early version of The Sims; the avatar design was imaginative but it became clear to me that the animations were on a loop, and one not set in time with the music or responsive to the beat. This fact made any link between the character and the music “they” were creating in real time tenuous at best. The procedurally-generated AI beats were fine but there were awkward moments of silence between “songs” which killed the vibe of the show. (I assume this would not have happened with a human DJ.) That said, there were actually three live performers who each did a short set between AI’s set, but the crowd was tough given the concert’s timing of early evening on a Wednesday.

The main audience interaction component involved preset choices to alter the music, voted on Jackbox-style. The “AI” would then take the vote and alter the music accordingly. However, after the fifth time when a winner, such as choosing to “drop the beat,” scraped through the competition at the last second, a bit of cynicism started to seep in my mind. It didn’t help that for some options, the vote was already 100% in favor of a particular option, so in order for it to swing a different way, a significant number of votes would have to go against, and to compound these contests, no hard figures or percentages were displayed on screen, making the agency of the voting portion fee rather pointless.

Overall, I was underwhelmed with WarpSound. None of the key components (animation, “live” music, gameplay, oraudience interaction) were done to a high enough standard to salvage the others, despite the impressive (and presumably expensive) stage set up.

In the words of one of the live human performers, “I see a lot of confused faces, who have no idea what’s going on. I’m just getting paid a lot of money to be here, so let’s get going, New York!” Sadly, for me, not this time. — Edward Mylechreest

June 17, 2021

Un(re)solved (Outdoor & Interactive Experiences, custom app, NYC installation): Tucked behind the Netherland Monument at one of the entrances to Battery Park in Manhattan, you will find Un(re)solved, a temporary monument to a different page in American history, but a more humane and ephemeral one.

Un(re)solved is an interactive installation that can be activated with a custom app. A brief audio introduction, prompted by the “Start” QR code on the pavement, invites you to step into the delicate structure on a slightly raised (but accessible) platform. Thirteen plexiglass panels of various heights form two concentric circles. Each panel contains a quilt with a drawing of a tree facing the circle and lists of names on the outside: 151 in total. Each of those people was murdered during the Civil Rights Movement-era, often simply because of their race. The murderers were not found or not punished. These cold cases were reopened by the Department of Justice in 2008 under the Emmet Till Unresolved Civil Rights Crime Act, but, for a lack of effort, resources, or evidence, most of them remain unsolved.

This touring installation provides a physical space to engage with the stories of political activists and humble workers, children, and adults — both named and anonymous — all lost to racial violence. The process feels like modern spiritual science. First, one needs to scan a QR code next to a name and, when prompted, say that name three times. Upon each call, augmented reality leaves appear on the phone screen and intensify their swirling until one of them comes forth, bearing a photo of the person summoned. More often than not, there is just a silhouette representing the deceased. The Un(re)solved app shows a short dossier consisting of their name and age, the circumstances in which the victim died, and the date and the place of their killing. The “read more” button leads to a longer description of the fatal events, the details of the initial investigation, and the current status of the case.

Some of the entries contain fragments of audio interviews with members of the family and the community of the deceased; some of the open cases are also featured in an interactive documentary. (I came across a couple of the audio interviews but didn’t watch any video fragments.) Simply reading these dossiers already felt like a visceral experience. The physical discomfort of squinting at my phone under a blazing sun, trying to shield myself in the shade of one of the taller panels seemed appropriate. For those who wish to take a break, however, there are two rows of park benches running on either side of the installation.

The capacity of this multimedia project to grow as investigations progress in the future is remarkable. The combination of an expanding digital archive and a beautiful physical portal at the entrance of a park illustrates the ambivalent nature of time and history. The Civil Rights Movement-era belongs to the past, yet anti-Black violence continues to be a real threat in the United States. One can stay in the Un(re)solved installation as long as they need or want to. I couldn’t help but want to hear more people’s stories, yet I was also feeling completely overwhelmed at the same time. Eventually, I made it to the outro of the piece (an audio message activated by another QR code in the center of the circles). On my screen, overlaid with scenes from a summer day in the park, ghostly presences were summoned; they flooded the air in the form of the names of the 151 people unjustly killed. And I am horrified to think of how many more there might be. — Asya Gorovits

June 15, 2021

Critical Distance (Museum of Other Realities): This project takes audiences into the world of the endangered Southern Resident orcas by plunging viewers deep into their ocean habitat. It’s magical to feel like you’re deep in the water as several orcas swim around you; in fact, I was so mesmerized by the sight of Kiki and her family, I momentarily dismissed their somewhat low resolution appearance. But as they circle you, you also start to notice signs of mankind: smaller boats and then enormous ships with huge engines and rudders in the ocean. And the orcas start to flicker and glitch out, as if part of a computer simulation gone off the rails. The message is powerful: only we humans can prevent their eventual disappearance. — Kathryn Yu

Polae (iOS): Pronounced “polly,” this slick interactive AR app brings to life a science fiction future where the tech juggernaut of the day is the aforementioned Polae, a “materials science” company founded in 2022 and run by the genius Adelaide Engelsen. The experience starts out like a slickly wrapped presentation of Polae’s corporate web site and product catalog, with the crown jewel in their portfolio being a creepy flying AI drone that also has more than a passing resemblance to a futuristic collar. Then—surprise!—disaster strikes and it’s up to you and the app to figure out what comes next. I won’t spoil the augmented reality trick that they play on you either (but trust me, there is one). Just enough to say that I half-chuckled, half-scoffed when it appeared. —Kathryn Yu

A science fiction story told through that most science fiction-y of devices: the smartphone, Polae jumps us forward into a near future where nanotech and drones are about to be household items. That is, if a major disruption to all life as we know it wasn’t around the corner.

For a piece that just a small slice of what is intended to be a larger story, Polae is incredibly slick. Just as master class in techniques for telling a story in a form that feels native to a mobile device while incorporating AR elements. No gimmicks here, or at least none that feel like a gimmick, just some solid storytelling that treats you like you live in the world they are building. Frankly, that’s all we ask for around here.

The looming disaster is also a good metaphor for climate change, even though the actual form the destructor takes is different. This shift into the pitch of metaphor made me think that we need more science fiction that’s about the looming climate crisis that isn’t weighted down by the actual details of the climate crisis. Stories are how a culture gains perspective on its issues, and sometimes what’s needed is to step outside the actual discourse and get a handle on the values and stakes without losing the forest for the trees.

Anyway: try Polae… it’s neat! — Noah Nelson

Republique (iOS): Republique is a taut, 40-minute white-knuckle ride into the lives of several strangers who are thrown together by happenstance during a terrorist attack on the Parisian subway. The project — an “interactive film” — comes to life through three Internet video streams showing three parallel but interconnected storylines.

A simple interface in the app lets viewers swipe from one camera view to another camera, so in effect you’re curating your experience the entire time. Flip. Now I’m watching a journalist who happened to be in the wrong place at the wrong time and gets caught up in the drama of a couple who’s been separated during the attacks. Flip. Now I’m in the feed of two popular urban explorers who inadvertently walked in on the terrorist attack while livestreaming in the Metro tunnels. Flip. Two friends who were on the platform waiting for the train are now trying to take cover in a storage closet.

And in all of these, one of the characters was live streaming at the time the attacks began and continues broadcasting as a way to both send out information and receive information from the outside world. All three streams occur simultaneously in “real time.” You can pause or go back 10 seconds at any time, in case you miss something; the film also shows what segments you may have missed once you reach the end. The omnipresent chat is also there, with trolls, gawkers, and naysayers but also well-wishers and people genuinely worried for the folks they’re seeing on screen.

Created by Simon Bouisson and Olivier Demangel, Republique cleverly takes advantage of the way live streaming can facilitate parasocial relationships and how we can perceive intimacy with a total stranger over the Internet. The overall effect is like Facebook Live… but on steroids. — Kathryn Yu

June 14, 2021

The Changing Same: Episode 1 (Museum of Other Realities): There’s an understated elegance to the title of this “magical realism time-travel” series that begins by exploring America’s original sin: the 400 year history of oppression of Black Americans by a system of social control driven by greed, arrogance, and an unnatural willingness to dehumanize Black people. “Changing Same”: as in the more things change, the more they stay the same.

Creators Michèle Stephenson, Joe Brewster, and Yasmin Elayat use volumetric capture technology to place performers into virtually realized sets that alternate between the mundane and the surreal, cross cutting across centuries in a single moment to illustrate the ways in which the criminal justice system of the modern day United States replicates the patterns of the era of enslavement. Literally in your face, the work here manages to not be figuratively in your face. We are shown, not told, the evils that have lingered in our law books.

This is just the first part of the planned series, and our conversation with the creative team the arc of the series has set course towards themes of Black joy and liberation. Which if tackled with as much visceral potency as episode one does the themes of injustice and inequality will be something to behold. — Noah Nelson

Discriminator (Outdoor & Interactive Experiences): Can a piece be both chilling and effervescent?

If so that’s exactly what filmmaker Brett Gaylor’s interactive documentary about how large corporations and governments used millions of images released under Creative Commons licenses to train the first generation of facial recognition algorithms which now, uh, threaten the privacy of every human on the planet. Oops.

One is tempted to say abused the pictures that millions uploaded and then licensed out under a legal framework that was intended to create a safe haven for remixing culture but legally those corporations and governments were in the clear. Score one for unintended consequences and add another good intention to the cobblestone path to Hades.

While the message is a dire one the messenger is a delight. This work puts the interactive, data-driven journalism projects of big media companies that win huge awards to shame. You’ll learn oodles. You’ll be delighted and horrified by the tech in equal measures. You’ll start contemplating how long it would take you to learn how to code a virus that would shut down the internet forever and ever.

It’s that good. — Noah Nelson

Jupiter Invincible (Outdoor & Interactive Experiences): I’m a giant comics nerd, at the level where it is both a problem and often the only thing that keeps me sane. Graphic novels. Monthly issues. Digital libraries. I’m into all of it. Yes, I have a soap opera like addiction to superheroes, but that’s not all I read.

I am, however, very skeptical about the use of augmented reality to truly enhance the form.

There are some ways that the creative team of Jupiter Invincible leverages the technology that are laudable: I quite enjoyed the bits where the AR app reveled a photo model for an image or a bit of dramaturgy on the scene. This feels like a solid use which lets the reader go deeper on a second pass.

However they also deploy some motion comics and soundtrack techniques that feel gimmicky. Comics is a brilliant form all by itself, and when sequential art is executed on by a dynamic team you don’t need anything else. Too much of the AR used here just gets in the way of the story of an enslaved man in the American South, Jupiter, suddenly discovers his immortality. Too much emphasis is put on the motion graphics, leaving the actual sequential storytelling uninspired. — Noah Nelson

Knot: A Trilogy (Outdoor & Interactive Experiences, custom app):
Before we get into it: I love Darkfield Radio. I’ve done everything short of banging pots and pans and screaming at my neighbors to go check out their work. So it would be an understatement to say that I was eager to jump on the latest from them.

Which is why I have to caveat this one: I had technical problems a go go with two of the three chapters. Half of which was my fault for not scouting out a proper park bench in LA with good signal, and the other half just the cruel whims of Spectrum and my AirPods.

Knot is a twisty narrative that bends time and space into a dark Gordian binding of horrific fate. Unlike other Darkfield works (THE INVISIBLE, ETERNAL) the masters of spatial audio don’t use the tool to create jump scares, but some of the (verbal) imagery is enough to make you want to turn away. Only you can’t really, because it’s all happening in your head(phones).

Get No Proscenium’s stories in your inbox

Join Medium for free to get updates from this writer.

SubscribeSubscribe

For those who haven’t done a Darkfield Radio show before: everything hinges on you being in a particular type of place at a particular time. You go there. You put your headphones on and you close your eyes. Then the world you were just looking at it replaced by an alternate world. The sounds all in the right place, but you know that they aren’t really there. You’re tempted to break the spell but if you hold out you can almost fell the breath of the characters on your ear as they whisper dark truths to you.

Unfortunately the spot I had picked out for the first section didn’t have good signal, so I spent the twenty minutes before the first part scrambling to find an alternate spot and that meant I couldn’t quite get into the right mood. It was still good, but not as good as it would have been if I could have actually been sitting on a park bench. I know this for a fact because the second part — which asks you to sit in the passenger seat of a car — was FLAWLESS. It’s so good. Provided you don’t turn your head too much and break the spell, I long for the day that all of this is keyed to work with positional tracking on headphones so that audio sources stay persistent the way they do in VR. Even so: compelling in the extreme.

The final segment — well the Internet failed me. Nothing would synch up right. I’m pretty sure that was on MY end because at 6PM every night the Internet goes haywire. I think some bot takes over the lines. It’s a shame. I was really into the story which revolves around a group of mysteriously amnesiacs who are trying to work together to prevent a tragedy from happening. To say more would be to spoil the fun.

All in all: you should sign up for Knot. Only make sure to scout your locations ahead of time. It’s a little more work than you might expect to do for a show, but it is worth it. — Noah Nelson

Whenever someone asks me about spatial audio, I immediately point them towards Darkfield; current masters of the form, and fairly prolific, to boot with three experiences available in multiple countries on their custom Darkfield Radio mobile app.

Premiering at Tribeca, Knot: A Trilogy takes a Darkfield Radio narrative in a new direction for the team. There’s more locations and objects and voices than ever before to keep track of: a little boy, the smoker, the woman in the wig, someone named Natasha, Scott Temple. There’s a mysterious box. And a lost jacket. And a cookie that appears out of nowhere. And the list goes on. Darkfield’s signature technical mastery is on full display in Knot: A Trilogy. Yes, I jumped when it felt like a cyclist was about to clip me on that park bench. I honestly did think it was raining more than a few times, particularly when the fat heavy drops hit the roof of the car “above” me. The audio is stunningly accurate and deceptive.

But the participant’s role in the Knot experience also felt diminished over time, or, at the very least, uneven. Whereas the characters in the park bench and car chapters directly address the listener from time to time, during the final segment, as the collective of characters began their “meeting” it was almost like I, the audience member, didn’t really need to be there at all. This was in stark contrast to the way the characters in previous Darkfield projects such as VISITORS or ETERNAL come across, where it seems like the voices you’re hearing almost rely upon you for their very existence.

I also have to question the artistic choices of asking participants to take in the first 20 minutes on a park bench, the second 20 minutes in a car, and the final 30 minutes of content somewhere in their homes. While some buffer time between chapters makes sense for logistical and travel purposes, I have to ask if the breaking up of the story was completely necessary; it felt like, to this writer, a lot of moving around.

The plot of this experience is meant to be cyclical with an element of what seems like a science fiction time loop, one dependent upon the actions of multiple people in three different places. It’s quite complicated and, dare I say, overly convoluted, and after a while I simply didn’t feel emotionally invested in the outcomes or why any of these particular folks were stuck in the scenario presented by Knot: A Trilogy.

Maybe it’s just me, but I’d rather leave this particular knot tied. — Kathryn Yu

Kusunda (Storyscapes): This deeply moving documentary from filmmakers Gayatri Parameswaran and Felix Gaedtke blends 2D film, volumetrically captured footage, and CGI to create a portal into the world of the Kusunda people of Nepal and their language, as told through the stories of three members of that indigenous culture.

The central crux of the piece is the simple fact that Kusunda, the language, is what’s known as a “dormant language” with a dwindling number of speakers. There’s a sense of urgency to the piece in the desire to preserve the language and place it in situ: with interviews with the shaman Lil Bahadur, who has largely forgotten the language of his youth and his granddaughter Hema who was a student of one of the last speakers of their mother tongue taking place in and around their homes.

Volumetric technology means that the viewers are more or less “teleported” into those spaces, but that’s not the only point of connection. The piece uses some light branching paths to unlock sequences, which is accomplished by having the viewer recite the word in Kusunda that corresponds to the narrative path ahead. It makes for a delicate dance, and a reminder that to participate in a language is to participate in a culture, even if only for a brief moment and thousands of miles away from that culture’s homeland. Potent, potent stuff. — Noah Nelson

Rebels (Oculus): Imagine a dystopian future where instead of cities populated by humans, they’re brimming with sentient giraffes, rabbits, and more, and all forms of personal expression are outlawed: music, art, dance. But, in one decrepit city, there’s a resistance fomenting. And the soundtrack to this rebellion? It’s not punk or rock’n’roll, it’s jazz.

From the imagination of Federico Moreno Breser comes the world of Rebels, a stylish VR short created entirely using Quill which looks very much like a comic book come to life. And like many good comics, it plays tricks with perspective and scale as well as space and time, but turning it up to 11 in the ways that only VR can do. The 10-minute ride where we follow our hero as he evades flying drone sentinels and giant two-legged walking robots across the rooftops of an unnamed city is short but thrilling. — Kathryn Yu

June 13, 2021

Inside Goliath (Storyscapes): In this short piece, an anonymous man “Goliath” retells his true lived experience as a person with schizophrenia and psychosis. As he describes his symptoms and his eventual time in a residential psychiatric hospital, the environment around the viewer changes to fit the narrative. We see a cube-based visualization of his inner life or find ourselves placed atop a giant interactive record player as he describes having a breakdown while DJing at a club. But the most memorable is an arcade game cabinet complete with joystick and buttons that shows an 8-bit version of Goliath’s life; you, the participant, play as Goliath in the game while he describes what happens next in his story. As an avid gamer, it makes sense that he might frame some of his actions as a video game: the constant ongoing struggles he faces and the need to overcome obstacle after obstacle.

Inside Goliath is a snippet of a much larger documentary work, Goliath: Playing with Reality: by Barry Gene Murphy and May Abdalla which will launch later this Fall. And I’m very curious to see how much deeper the relationship between games, schizophrenia, and healing can go. — Kathryn Yu

Madrid Noir (Museum of Other Realities): This roughly 45 minute animated VR experience by No Ghost and Atlas V takes us into the story of Lola, a woman who is returning to Madrid to empty out the apartment of her estranged uncle Manolo after he’s been declared dead. The viewer begins in the role of a good friend who has offered to help her pack up his things; you see things from a first person perspective and can interact with some of the objects in his home while Lola reminisces about the past. But, of course, as the two of you rummage through his belongings, the past comes rushing back and Lola can’t help but recount the tale of the last time she saw her beloved uncle as a small child. And so, the adventure really begins there.

Madrid Noir cleverly switches between these three perspectives: you as the friend in the present, you in the shoes of as young Lola in the past, and you as an observer of young Lola and Manolo as Lola begins to suspect that her uncle isn’t quite on the up-and-up and starts investigating his life. I greatly appreciated the moments of interactivity embedded in the piece: being able to press the shutter button on Lola’s camera as she spied on her uncle from a window or spin a flashlight around while Manolo scurried through the darkened alleyways of Madrid.

The sense of tension is heightened by the lighting and scenery choices in Madrid Noir. Each sequence appears as if on a “stage” in virtual reality — a bedroom with only two walls and no ceiling or a street corner with a single streetlight raised up on a platform — surrounded by darkness and a stylized painted cityscape in the background. The viewer sometimes sees the scene from above, as if on a balcony, but other times it’s as if you had a front row seat to the action. Typically, I’m not a huge fan of tabletop or dollhouse-scaled staging in VR but the openness of the environment here felt entirely in tune with the theme.

Madrid Noir cleverly mashes up elements of first person video games, theatrical staging, and a stylish crime drama into something I’ve never quite encountered before in VR; somehow it all just works seamlessly.

If I had a seat, I certainly believe I’d be sitting on the edge of it. — Kathryn Yu

June 11, 2021

Bystanding: The Feingold Syndrome (Museum of Other Realities): Bystanding is a bit of a hard one to categorize; it’s fiction based upon a real life situation. Israeli rowing champion Jasmine Finegold lost consciousness when her boat capsized in Tel Aviv’s Ha’Yarkon River and a crowd of spectators on the banks of the river hesitated for over four and a half minutes to rescue her.

Bystanding recreates this scenario and places participants on the river bank, surrounded by gawkers and passersby. The viewer sees a freeze-frame fictionalized version of the incident and is able to hear the internal monologues and thoughts of those who did not rescue Jasmine even though they potentially could, although that act in and of itself might have been dangerous given the river’s polluted state.

As a silent, invisible observer, you can navigate around the scene and choose to listen to any of the various rationalizations provided by the characters present as their descriptions are animated in front of you. The characters represented in the scene are not specific individuals but rather archetypes: a cautious environmentalist, a wary mother, an introvert suffering from childhood trauma around swimming, a nervous rabbi, or one of many others. One character even mentions the murder of Kitty Genovese in her bit (a very on-the-nose reference, sure, but I’ll also note here that later investigations revealed the truth to be far less cut and dried than the scandalous newspaper headlines — a fact not mentioned in Bystanding).

It’s an interesting way to explore the “bystander effect” but my fear is that viewers will extrapolate outwards a rather simplistic takeaway from this one incident when in fact human psychology is much more nuanced and complex, especially when it comes to the idea of an individual making a sacrifice or taking action for the greater good. — Kathryn Yu

Jailbirds — Chapter 1: Bwa Kayiman (Museum of Other Realities): This short animated 6 DOF piece is the prologue to what appears to be a much larger work. You spy upon two cellmates at a terrible looking prison set in a castle on an island as they interact with each other and then the nasty Chief-Warden of the jail. The art style is distinctive and memorable, with textures brought to life using lines of ink as shading. Even as the Chief-Warden abuses the two prisoners, one of them — the gentle giant Felix — seems completely unfazed no matter what he says or does. And we only start to get a small hint of Felix’s secret to happiness as the piece finishes.

An intriguing start to what may become an epic story in the future. — Kathryn Yu

POV: Points of View (Museum of Other Realities): It’s the year 2025. We’re in Los Angeles, California, in a dystopian future where the LAPD has begun using AI-enabled sentient drones (of course they have). And as an “Insighter” (which I took to be some sort of internal investigator), you’re looking through a case file of some sort of drone-related incident. You quickly meet the young Cassius Moore, a gifted Black hacker and college student who is getting out on parole under a certain set of chilling conditions: being followed around by a drone as he goes about his day. However, beyond that initial framing, there’s not much you can do here as an investigator as there’s no decisions to make about Cassius’ fate or an ability to choose what aspect of the incident you examine next.

The 180 degree film plays out in front of you with many quick transitions and cuts between scenes. The camera often jumps around of its own volition — quite quickly I might add — which I found to be disorienting and dizzying (something to note if you get motion sickness in VR). Sometimes the characters are speaking directly at the camera, as if to address you (the investigator). But then it becomes clear that the viewer is often looking at things from the perspective of the drone itself which is why the scenes all look slightly distorted at the edges; it’s like you’re looking out at this sci-fi world behind a wide angle lens.

That said, the story the creators are trying to tell takes some surprising turns and hints at a future where (spoiler) artificial intelligence agents might actually be better than humans at making decisions, as the ones in this future don’t have the same prejudices as humans (in stark contrast to the evidence much of our 2021-era technology suffers from a not insignificant mount of algorithmic bias).

I’m struck by the fascinating (yet unlikely) hypothetical they’re posing and looking forward to seeing more from this team. Note: this piece has two parts inside the Museum of Other Realities so make sure you view both segments.

— Kathryn Yu

(NoPro correspondent Michaela Ternasky-Holland is a producer on POV; she has not had editorial input, nor has she been privy to the contents of this review before publication.)

June 10, 2021

Breonna’s Garden (iOS): She who plants a garden, plants hope. Those are the words that start off the meditative, beautiful Breonna’s Garden experience, a new augmented reality app. Created in honor of Breonna Taylor, the Black medical worker who was shot and killed by police officers while asleep in her home in Louisville, Kentucky in 2020, the app allows you to place a statue of Breonna in AR wherever you are standing by scanning the ground or floor. The project was created by artist Lady PhOenix and Sutu in collaboration with Ju’Niyah Palmer, Breonna Taylor’s little sister.

Here, Breonna appears slightly larger than life, wearing a gorgeous, shimmering gown and holding red tulips in her hands. Breonna looks peaceful and content as her little sister appears next to her, captured via volumetric video. Ju’Niyah starts speaking and shares some memories of her older sister: stories ranging from the mundane and to the significant. Butterflies flit around the pair as large glowing iridescent flowers stand nearby with their long, elegant stalks. These glowing plants contain even more memories. The audience is encouraged to record and share their own stories via the app: remembrances of Breonna or anyone who they miss.

And the dulcet tones of “You Are Everything” by The Stylistics playing in the background remind us that yes, this is a place of grieving and sadness, but it can also be a place for comfort and healing. Perhaps this garden — Breonna’s garden — will someday yield that most precious bud of all: hope. — Kathryn Yu

Marco & Polo Go Round (Museum of Other Realities): Sometimes, in a relationship, you can feel like down is up and up is down, especially when you’re arguing with your partner and the two of you can’t quite seem to see eye to eye. That’s exactly the type of emotion that Marco & Polo Go Round capitalizes upon and brings to life in virtual reality.

The piece allows the viewer to follow along as a silent, invisible observer during a fight that the couple Marco (played by Emmanuel Schwartz) and Polo (played by Léane Labrèche-Dor) are having on Marco’s birthday. Marco wants to laze about in his underwear for this special occasion, while Polo has scraped together a cake from whatever she could find in the kitchen and has arranged all sorts of plans for her boyfriend’s big day. And that’s right about when things start to fall apart. And as always, the big fight between romantic partners seems like it’s about one thing but it quickly becomes evident that it’s actually about something deeper and broken between them.

Their living space acts as a metaphor as their life starts to fall apart, literally. There’s duct tape and rope seemingly holding together cabinets and shelves in the kitchen; it does the job until gravity reverses itself and cups, plates, knives, and forks start falling upwards to the ceiling. The refrigerator vomits up a cornucopia of groceries. Coffee gets spilled on the ceiling. The result is strangely poetic and beautiful, all infused with sadness and dread for the impending inevitable break up. After all, we all know how this story ends.

But, somehow, Marco & Polo Go Round find a new and compelling way to tell it. — Kathryn Yu

Mine (Museum of Other Realities): This 2D animated short takes viewers into the world of a tightly knit futuristic society whose dependence on specially treated water — bolstered by some sort of mysterious technological and scientific improvements — allows them to live more ecologically and not like “the old ways.” Blaze, the story’s protagonist, senses something wrong with the community’s life-sustaining liquid; but, of course, none of the Elders believe young Blaze. Soon, disaster strikes and we’re left with more questions than answers. The creators do a fantastic job of hinting at the customs and rituals of this community and I only wish I knew more about the intriguing world they have built.

While I thoroughly enjoyed what I assume to be the first chapter of the Mine story, the staging within the Museum of Other Realities left something to be desired. The film was projected in 3D space on a huge wall, but this huge wall was also placed in a very shallow viewing area such that the only way to take it all in was to crane your neck upwards for the duration of the 13-minute movie. It was like sitting in the front row of a movie theater (remember those?), but with a heavy headset on your head. — Kathryn Yu

June 9, 2021

Kinfolk (iOS): Kinfolk is an educational mobile app which allows users to place augmented reality monuments to real life people in real life environments. Created by Movers & Shakers, the project hopes to illuminate the histories of Black historical figures such as Shirley Chisholm, Toussaint L’Ouverture, or Frederick Douglass. The biography of each chosen person is available as text or narrated audio and each one accompanied by additional resources such as scans of relevant documents, music, podcast episodes, or video content.

Each of the six monuments is lovingly rendered and the amount of material is robust but I can’t help feel like both the scaling of the figures and design of the app doesn’t lend itself to comfortable usage. Ida B. Wells felt far too small to me, standing about than three feet high high my kitchen. The app also obscured the bottom half of the camera view with text information by default on my iPhone; I struggled to scroll through half-screenfuls of text while holding up the device such that the monument was still in view. The audio component can also feel like you’re compelled to keep your phone in the same position for several minutes just to hear it all.

The history itself is quite compelling but I’m not sure this is the best format to share this information. — Kathryn Yu

A Life in Pieces: The Diary and Letters of Stanley Hayami (Museum of Other Realities): Decades after WWII had ended, Stanley Hayami’s family found his diary: the one he kept as a young Japanese-American man imprisoned at the Heart Mountain camp in Wyoming. Moments from that journal have been brought to life in VR as part of A Life in Pieces.

A smart, caring artist and writer, we hear and see reenactments of Stanley struggling with the unfairness of leaving his home of San Gabriel, California; the difficulty of attending school while in an internment camp; and his life during wartime: the giant question mark that is next year or the year after that. Within the early parts of the experience, the juxtaposition of Stanley’s idle observations as a teenager mixed in with the realities of living in the camps or the world outside can be quite jarring at times, particularly as he doesn’t seem to be overtly angry about his family’s situation; this changes quickly as he nears the age of 18 and the sceptre of military enlistment begins to overshadow his future.

Stanley’s words are brought to life in a mixture of visual styles during the narration of his journal entries (his diary, art, and letters were donated to the Japanese American National Museum by his family). Sometimes we see people playing his family members captured on video embedded into photo realistic looking environments; other times his surroundings turn into detailed illustrations or colorful 3D painted environments or simple black and white doodles lifted directly from his journal itself. Interspersed with the environments are scans of photographs or documents from the era as well. It’s all beautifully rendered but a lot of it moves by far too quickly; slowing down the pacing and allowing some additional breathing room might do some good here.

And while Stanley’s story sadly ends in tragedy, this VR documentary provides a vivid, caring look at his rich inner life as well as his dashed hopes and dreams. — Kathryn Yu

Procession (iOS): This augmented reality art by Dustin Yellin comes to life like a massive pop-up book in your living room; it’s a bit surreal to realize you can move through the layers of the digital art by stepping forwards and backwards in space.

Your task is to pixel-scan your way through the Noah’s Ark-like scenery and find the endangered animals hidden in the tableau. Each creature comes to life once you’ve properly scanned it. And once you’ve found them all, disaster strikes.

The effect is very cool and the conservation message comes through loud and clear, but if you play a lot of video games, you may find yourself speed-running through all the content (there was only one “room” and ten different animals to find when I tried) at a breakneck pace. — Kathryn Yu

The Severance Theory: Welcome to Respite (VR Chat): This project is a scripted interactive theatre performance taking place on VR Chat for a small audience. It is adapted from an IRL show in LA created in 2019; the original live version was produced by Lyndsie Scoggin, Danielle Levesque, Payden Ackerman, and Kelley Pierre. Luckily for us, Welcome to Respite has now been adapted for VR by Lyndsie Scoggin and Braden Roy and is being presented as a collaboration between CoAct Productions and Ferryman Collective at the Tribeca Film Festival.

The main participant in this immersive theatre experience is cast as Alex, a person who has dissociative identity disorder (previously known as multiple personality disorder); people with DID develop one or more alternate and distinct identities, known as “alters.” During the show, Alex is reliving a lost childhood memory while interacting with their mother and father, both played by live actors within VR Chat; for the purposes of this writeup, I participated in the experience as Alex. And so I found firsthand that this scenario feels fairly true to life: Mom chuckles as I answer “I don’t know” (to a question that I didn’t actually know the answer to) and tells me to wait for dinner when I ask for a cookie. And Dad responds with a sarcastic “Are you sure about that?” when I ask him to sing me a song. So when it comes to casting the audience in an immersive work, it’s pretty easy to pretend to be a clueless kid when you’re asked to do so.

Polished and lovingly crafted, Welcome to Respite is inspired by the real stories of people suffering from DID. And the experience certainly feels of a certain time and place. The hand-crafted details in the 1980s-era world — from the cassette deck on the front porch to the blanket fort in the attic — really shine. It’s evident the team has carried over a lot of expertise from the physical immersive theatre world into a virtual space, particularly in the thoughtful treatment of their custom onboarding and offboarding spaces (as they relate to a piece with themes around mental health). Dynamic use of animation, evocative music, and other theatrical effects also do a good job bringing us through this singular memory in Alex’s mind.

Welcome to Respite also takes full advantage of the affordances of VR: we move magically through space and time when needed, the adult characters appear enormous in size compared to child-sized Alex, some characters take on non-humanoid forms or turn invisible as appropriate, and at times the environment disappears around us or warps in surreal, non-Euclidean ways to support the story. Additionally, other participants are present here as Alex’s alters. They are set up as invisible, silent observers during the experience, but are not entirely powerless when the story takes a turn. Yes, even the alters still have a part to play (note: I’m being deliberately vague here). It’s a nice touch for folks who might not want to be the star of the show but still interact with the world.

While at times the quirks of VR can break immersion (limbs clipping into torsos, lack of facial expressions, objects dragging behind someone as they teleport-walk towards you, needing to respawn on occasion), this scrappy and resourceful team of indie creators is able to accomplish a lot within both the VR Chat platform and the medium’s overall limitations. Plus, it all works seamlessly on an Oculus Quest (a technological feat in and of itself).

So: bravo. I can’t wait to see what future chapters of Alex’s story have in store for us. — Kathryn Yu

See more of our Tribeca Immersive 2021 coverage here.

Discover the latest immersive events, festivals, workshops, and more at our new site EVERYTHING IMMERSIVE, new home of NoPro’s show listings.

NoPro is a labor of love made possible by our generous Patreon backers. Join them today!

In addition to the No Proscenium website, our podcast, and our newsletters, you can find NoPro on Twitter, Facebook, YouTube, Instagram, in the Facebook community Everything Immersive, and on our Discord.