The metaverse dissonance

Disclaimer: While the views in this article are mine, please be aware that I work on metaverse things for Microsoft.

There has been a lot of talk about the metaverse lately. Not as a pop-culture vision, but as a real thing. Or rather: The pop-culture version of the metaverse (as depicted in books, movies, ..) was used interchangeably with the metaverse as it is currently being developed. And whenever a strong vision is at odds with the real-world implementation, it leads to problems.

For example: Snow Crash is maybe a good book about a metaverse, but it is “bullshit in a professional sense”. No one I work with on metaverse things would cite the concepts in the book as something to achieve or to look at for professional guidance (not to mention it describes a dystopia). When we look at Ready Player One we see a science fiction book / movie that is about as relevant for our work as Independence Day is for a NASA space engineer. And that’s fine — they are pieces of entertainment, not technical literature.

Right now the term “metaverse” then means different things to different people and the most powerful visions (reaching and “educating” the most people) are not reflecting what we (the professionals in the metaverse field) currently create.

But this is not a simple black and white. Those visions will inevitably inspire and influence public expectations and thus investors and thus our end product. So while these visions do not inspire professionally, they inspire spiritually and that’s hard to separate long-term. They also distract from what’s actually happening and prevent public discussions we should be having.

And then there was an interview

When I listened to Casey Newton interviewing Mark Zuckerberg about Facebooks vision of the metaverse I was first stunned, then angry and by the end I was screaming at my walls.

So many things were wrong with it, but I want to talk about two things in particular.

One: Mark talks about the metaverse mostly from a consumer endpoint perspective. Understandably so, as he wants to push Oculus from FACEBOOK as prime devices to experience their version of the metaverse. He talks about presence, dancing and different types of fitness. And he talks about workstations, meetings and opportunities for content creators and other experiences that Facebook wants to own. This is performativity towards their own goals.

Two: Mark still evades (or doesn’t understand) questions around responsibility and ethics in social virtual worlds. Granted, he doesn’t answer them with “What ethical implications?” anymore, but the answer: “It’s a great business opportunity for businesses and content creators!” is somehow even worse.

If you have an hour and want to know more what was so wrong with Mark’s vision I highly recommend Raph Koster’s talk at GDC 2017:

So, Dirk, what do you think the metaverse actually is and the potential it enables?

Matthew Ball wrote a good article and a series identifying key characteristics of a metaverse. But if you ask me personally, I have written about this and explained in detail why some version of a metaverse is currently happening. I used the term “Connected Worlds” instead of metaverse which I personally like better, since it invokes less pop-culture visions.

Here is the quick summary: My assumption is that Social Virtual Worlds (aka. Social Media), Internet of Things, Computational Graphs, Cognitive Services and Mixed Reality, will converge and lead to a connected digital-physical world, where the infrastructure as well as the inhabitants (people, environments and things) are connected and digitally augmented. As processing power, storage and bandwidth increases, there are fewer limits at how many dimensions can exist at the same time. This means that there will be not only one connected world, but many parallel ones.

These digital overlays interact with the real world, but can change as easily as switching TV channels. Some of those channels might be educational, some inspirational, some might feature entertainment, others would be for work and so on — providing specific capabilities and adding (additional) context to the physical world.

Depending on the currently selected channel, the context, usage, behavior and capabilities of physical locations might change as well. A physical chair is a chair and not a chair as it might be augmented or adapted to be something else entirely in some digital dimension, sometimes also changing physical attributes. Access to physical capabilities might be controlled via interfaces in digital dimensions that only authorized users can experience.

If we consider the physical reality changing completely for each individual, we can only differentiate between “Shared versions of reality” (multi-user, public) and “Personal versions of reality” (single user, private) that can both be manipulated digitally at will. It’s all about enhancing us through merging digital attributes with the physical world, providing capabilities when we need them, where we need them.

Note that I didn’t talk about headsets.

Where dissonance happens

The visions described in pop culture (and since then influencing technology visions) are mostly focused on headsets or some form of individual, separated experience: “I need to wear a headset to enter the metaverse — and this metaverse is something separate from the physical un-augmented reality”. That makes narrative sense since it breaks down the messy bits into a clear story of “metaverse vs. reality”. In Ready Player One the reality is bleak and dreary while the metaverse is exciting and shiny. It’s the same problem as with Mark’s interview and hides a very important point:

The currently emerging metaverse will encompass the real world and digital dimensions in a way that is inseparable.

The metaverse as it is build right now will not be a separate thing from reality. It won’t be “I need to put on a headset to be in the metaverse”, instead interfaces to the metaverse will be incorporated into almost every thing and environment, making reality itself a part of the metaverse — and vice versa the metaverse part of reality. I will transform any physical space into my work, my workout, my entertainment, without hard boundaries. I will be able to reach out and manipulate digital dimensions, which will have real-world implications, and I can manipulate the physical world to change the digital dimensions. Any action and value will be the same in digital dimensions as in physical reality. The metaverse IS the real world and vice versa.

How I know this? Because we have been doing this for years. The implementation of the metaverse began when we created virtual worlds containing digital assets that are worth real-world money, only we call them “massive multiplayer games” and later “social media”. We accelerated it when we connected things to the internet and created a two-way channel, only we call it “Internet of Things”. And we cemented it when we put devices in our pockets that turned ourselves into connected things, only we call them “smartphones”. The metaverse is an MMO (or a spatial social platform if you prefer that term) on many maps with different intentions and goals that overlay the real world at scale 1:1:n. We already did that.

Look at Pokémon Go. Do you think that’s not a metaverse case just because it’s on a phone? Do you think there would be an actual difference if we had the same thing on some goggles? Do you think there is a difference between Pokémon Go and Swarm (Foursquare)? Google Maps? What about Yelp? Your work portal? Philips Hue? They all just drag real world data into digital dimensions, augmenting reality with digital data at specific points, simulating and connecting, supporting desired goals. And they all have real-world implications whenever you act within them in the digital dimensions.

The goggles do not matter as much as you might think.

This of course makes the reality of the metaverse way more messy and.. well… even more scary than the neatly separated future visions. Even the dystopic ones.

Never trust the client

As these digital objects within virtual dimensions interact with the physical world, everything (people and things) needs to be tracked in all dimensions (physical and digital). This requires systems that keep a record of every person and object in existence (real or virtual), including its behaviors, capabilities and states and locate that on a map of the world as well assigning it to a specific channel. These systems would provide consistency and concurrency of the digital channels as well as ensuring constant synchronicity between the physical world and the digital channels.

Yes, this is about as surveillance-heavy as you think.

There is also a hard rule in multi-user systems design: Never trust the client. The system represents the only source of truth and the whole purpose of the client (the UI or user experience) is to present a nice visualization of the system’s data. But never ever is the client to be allowed to do any direct manipulation of the data itself, otherwise one person could (intentionally or unintentionally) manipulate every one else’s reality.

That means: In the currently emerging metaverse, the clients — individual people and things — are not to be trusted by design. Instead the inherit world-view is that systems are keeping the source of truth. That should make you pause.

The goggles do not matter as much as you might think.

Let’s talk about systems and governance

As a result these technical systems will emerge as the infrastructure of the world. And they will shape society. Intentionally or unintentionally, the systems will influence how people behave. They will do so regardless of the user touchpoint.

Example: What do you think is responsible for the rage-problem at Facebook: The pixels, images and buttons on the smartphone or tablet or monitor or the way the algorithms and systems are designed?

The actual hard part of the metaverse is a systems and governance one. In other words: Social problems are currently much harder to state and solve than the technical problems.

It raises questions around accessibility when digital channels that can manipulate the physical world are only usable by specific groups. If you think technology access and literacy are a problem now, wait until you need to have access to a specific technologies and platform to be able to do anything in the real world (press a button, turn a dial, turn a key). Or to be physically enabled to use them. This is not a UI or technical problem, it’s a social and systems problem.

Let me rephrase:

If you don’t design these systems with intention (values, governance) in mind, you will design them unintentionally.

Because there is no scenario in which systems that influence the physical world are not also influencing society. You either do it on purpose by being deliberate (and ideally transparent) about your intentions or you will do it by accident — in a random, unconscious way (also leaving room for malicious actors to manipulate you).

I’m sure Pokémon Go didn’t mean to be racist, but it was.

The systems make their providers and operators very, very powerful. And this makes Mark’s ignorance on ethical implications so dangerous. In the interview he talked about governance only in technical terms of interoperability and portability between system providers. And he talked about experiences and client things. Literally the least relevant problems.

I hated it.

Let’s TALK about systems and governance

Look, as I mentioned — the metaverse is not a future, it is already here. You can argue about the “canonical definition” and if it’s “maybe just a little bit here yet”, but digital dimensions exist right now with the ability to change the physical world: Turn on lights, change the room temperature, change the behavior of your car, set the time on public clocks, regulate traffic, control the output of a nuclear power plant and so on. That’s reality now. It’s just not codified into unified systems… yet.

Fine, maybe we need the goggles after all so that the public finally understands the extent and impact of those digital dimensions when everybody can see and experience them in a tangible and relatable way. But by then it will be too late. The systems will already be there.

We need to have this discussion right now while they are still being designed and built. We need to make sure that:

1. The systems are created with transparent intentions (you can read them)
2. The intentions map to what we desire as society (you can discuss them with the creators)
3. Systems are accessible and equal to all members of society (you can use them fairly, regardless of your race, gender, sexuality, ability, .. and so all of you can verify that intentions match the outcome)
4. Systems are flexible and have a defined process to change them if and when social desires change (giving you back trust & empowerment)

And finally: It’s the governments job to govern, not the job of hyper powerful companies that just happen to write systems of governance.

So let’s talk. Let’s align our visions and understanding of the metaverse. Maybe come up with a couple more terms to differentiate different visions. Maybe be more vocal around our design principles. Make sure they include different views, perspectives, values and interests. That’s the true form of interoperability we should worry about: A social one, not a technical one.

💙