CommentaryIn April 2020, Amazon released a comedy series called “Upload.” The show extrapolates a future in which technological advances have led to the successful simulation of human consciousness in silico. This technology is used by companies to “upload” dying individuals into digital worlds where they can “live” in perpetuity. When uploaded, human consciousness is converted into data and executable code, which can be edited, throttled, or even deleted depending on each upload’s membership plan and payment status. By portraying this world, the show breaks the boundaries between reality and virtual reality, consciousness and artificial intelligence, and even life and afterlife, entangling various legal questions in novel ways.
Given that such upload technologies are not expected anytime soon, it may seem strange to give “Upload” any sort of legal analysis. The show’s world-building, however, provides an opportunity to explore—through a possible distant future—the legal, social, and business decisions being made today. It serves as an exaggerated mirror through which we explore legal norms that are currently taken for granted.
Three interrelated topics serve as the foundation of our analysis: (i) the distinction between human consciousness and artificial intelligence in the context of copyright and contract law; (ii) how legal personhood may be doctrinally denied to “uploaded” minds, even if they are—for the sake of argument—as capable, conscious, and creative as living human minds; and (iii) how the power imbalance between individuals and technology companies will continue to exacerbate the problem of contracts of adhesion and surveillance capitalism (even after physical death!). Ultimately, we conclude that an unyielding, unquestioning, myopic desire for digital rights (at all costs) may lead to the unintentional loss of crucial non-digital rights.
“Upload” depicts a future in which the current legal regimes of copyright and contract law unintentionally reinstate a virtual form of servitude, robbing individuals of agency, autonomy, and ultimately legal personhood. To understand how this is so, consider first the existing law governing ownership of AI creations. In the United States, the Copyright Office has adopted a stringent human authorship requirement according to which the Office will only “register an original work of authorship, provided that the work was created by a human being.” Even in jurisdictions that seem more welcoming to the idea that AI-generated content may be copyrightable, the ownership rights ultimately belong to the entity using the AI. In 2020, for example, a court in Shenzhen, China, recognized that stories produced by the Dreamwriter AI possessed sufficient originality to qualify for copyright protection. The court, however, granted the rights over the stories not to the Dreamwriter AI but to the Chinese tech titan Tencent that created it.
This restriction raises a theoretical question: is there a qualitative difference between a living human brain and a computer? Nathan, the protagonist of the show, argues that there is none: “[i]n a sense both of our consciousnesses are simulations, mine on a silicon computer and yours on a computer made of meat, your brain” (S01E07 at 12:07). Unfortunately, the law does not see it this way. It simply ignores the existence of uploads. They are not allowed to work, contract, or directly earn money in any way. More importantly, the scanned uploads are themselves “owned.” Nathan complains that he is “just this plaything” for his girlfriend (S01E05 at 1:30). She controls all his purchases, what he does, what he wears, and what he eats.
One ramification of this is in copyright law. Throughout the show, various uploads continuously produce content: cinematic golf shots, innovative pranks, and artistic prose. All these creations, however, are owned by Horizen, the corporation behind Nathan’s virtual world. The uploads are treated much like the Dreamwriter AI.
Uploaded human minds are not entitled to copyright protections nor are they allowed to contract around this limitation. They are stuck. Under fear of permanent deletion, they may be forced to engage in endless intellectual labor without legal recourse as they lack legal personhood.
But why does this lack of personhood matter? Given the historical and theoretical connections between personhood and human rights, one can imagine the myriad problems this situation may create. We will highlight just two. First, is the growing problem of contracts of adhesion. Second, is surveillance capitalism’s role in transforming human minds into products available for purchase.
Regarding contract law, it is true that each upload ‘agreed’ to this loss of personhood in Horizen’s terms of service before being uploaded. Therefore, under current California law, for example, such terms would be upheld. Not even “oppression or surprise due to unequal bargaining power” are enough to render a contract of adhesion unenforceable.
Contracts of adhesion must contain “substantive unconscionability,” which focuses on “‘overly harsh’ or ‘one-sided’ results,” to be unenforceable in California. In “Upload,” it seems clear that courts do not treat uploaded individuals as legal persons and therefore will not recognize their mistreatment as substantive unconscionability.
Furthermore, uploads are not only forever bound by Horizen’s terms of service, they also lack basic legal protections if or when Horizen breaches its own terms. This is even more alarming because Horizen advertises such loss of legal personhood as a heavenly afterlife. It targets individuals at an extremely vulnerable time: when they are dying. Nathan, for example, is forced to accept Horizen’s terms of service while dying on a hospital gurney.
In the process, Nathan forfeits legal personhood, the ability to contract, the right to own property, and any other future legal recourse. After scanning the minds of dying individuals, Horizen has the uploads locked in a golden cage where it not only collects monthly fees for hosting these unfortunate souls but also has the legal power to make further profit off the fruits of their labor. The uploads have only their Horizen account holders to turn to, who may themselves also decide to delete, freeze, or alter the uploads without recourse.
In fact, the relationship between Horizen, the uploads, and the living account holders raises the second problem inherent in the power dynamics of the show: surveillance capitalism. In the modern digital marketplace, “if you are not paying for the product, you are the product.” This phrase has become a modern proverb. Some have argued, however, that this axiom is already outdated. Nowadays, “[n]o matter how much you pay, under current US law, you’re the product if the company’s privacy policy says so.”
This is taken to the extreme in “Upload.” Horizen presents itself as offering a service for the individual uploaded, but uploads are not the paying customers. Rather, Horizen serves the living by selling the simulation of their loved ones’ minds. If you are that simulated mind, you are the service being sold. Your very existence can be extinguished whenever the consumers who pay—the living individuals—cancel their subscription.
In this way, “Upload” demonstrates what can happen when individuals are reduced to only two roles: consumers or products. If you are not a consumer (because you are dead) and you are not a product (because there is no purchaser) then you are quite literally deleted.
In spite of these terrifying possibilities, the allure of an eternal digital afterlife was too much for society to bear. Human rights defenders on the show were so angered by the wealthy individuals who got to experience digital immortality, that their immediate goal was to democratize Horizen’s technology.
According to the activists, everyone should have the right to digital life extension. Instead of fighting for the rights of uploads, activists are fighting for the rights of humans to be uploaded (S01E04 at 11:44). In other words, they are fighting for the purported human right to enter what amounts to a labor camp and retail shop for consciousnesses.
In this age of cyber instability, new digital rights are cropping left and right: from a “right to be forgotten”; to a “right to cybersecurity”; to a “right to internet access.” But perhaps the biggest lesson to be learned from “Upload,” is that we must resist our tendencies to immediately embrace and apply human rights law. Instead, we must reassess “the ethical foundations underlying” human rights, exploring the “outer limits” of the discourse and its potential pitfalls. Especially when the subject matter to be regulated is a paradigm-shifting technology, we should consider leaving some of our existing legal paradigms at the door.
In conclusion, what sets “Upload” apart from other shows about technological servitude (and inevitable robotic uprisings) is that in “Upload” human consciousness is the AI. Unfortunately, in “Upload,” we are so doomed by technological determinism that we voluntarily put our very consciousness at stake. Through this short analysis, we hope to reignite the legal community’s engagement with science fiction, leading to a more nuanced conversation about what is artificial in artificial intelligence, what is virtual in virtual reality, and what is digital in digital rights. We argue that becoming early adopters of a new reconceptualized language around “us” and “them,” the “self” and the “other” can perhaps future proof our society from the cyborg perils that await. Let us trade a looming Horizen for at least a slightly better horizon. This short analysis is an abridged version of our article found here.
João Marinotti is a Visiting Fellow at the Information Society Project at Yale Law School, a Visiting Fellow at the Center for Quantum Networks, a Fellow at the Indiana University Center for Intellectual Property Research, and the Jerome Hall Postdoctoral Fellow at Indiana University’s Center for Law Society and Culture.
Asaf Lubin is an Associate Professor of Law at Indiana University Maurer School of Law, a Fellow at the Center for Applied Cybersecurity Research at Indiana University, an Affiliated Fellow at the Information Society Project at Yale Law School, and a Faculty Associate at the Berkman Klein Center for Internet and Society at Harvard University.
Suggested Citation: João Marinotti and Asaf Lubin, Law, Artificial Intelligence, and Science Fiction: Reflections on Implications of “Upload” on Copyright, Contract, and Human Rights Law, JURIST – Academic Commentary, May 24th, 2021, https://www.jurist.org/commentary/2021/05/marinotti-lubin-upload-legal-implication/.
This article was prepared for publication by Vishwajeet Deshmukh, a JURIST staff editor. Please direct any questions or comments to him at commentary@jurist.org.
