Tech To Feel the Metaverse

The bulk of the conversation about the metaverse usually revolves around sight, and sometimes as an afterthought, sound. We want to be able to experience a false reality visually because we are, by-and-large, visual beings. However, what makes the real world feel so real is all of the supporting things we experience thanks to our other senses. A windy day is not just windy because we see things flapping about in the wind, or hear the sound of wind rustling – we mimic that bit decently well in software right now. What we cannot feel in a virtual world is the rush of wind on our skin, the force of the wind on our clothes, and if it’s an especially strong wind, the laboured breathing and watery eyes. This is what ruins a good visual experience, even if the accompanying audio engineering is top-notch. 

And the audio engineering will need some serious work as well. Think about how you do a video meeting now. Everyone’s voice comes through their microphone with the background noise of whatever is happening around them. And it’s not spatial audio, because there’s no need for it to be, yet. Inside a metaverse, in, say, a large conference room where you’re seated around a table and each person has a position, it's important. Everyone has people close to them on their left and right, and people further away from them at the ends of the table or across from them. How useless is it to have the voice of your boss, who is seated on your right, come into your headset at equal left and right balance? Just audio processing for each member of the team, to spatially place them and then give them all true spatial sound, in real-time, is a challenge to overcome, but thankfully a lot of this work has already started. Some estimates suggest that the spatial audio market will be worth $280 billion in just three years! And that’s a place we’re not starting from scratch. Can you even imagine how valuable a company that solves the problems of smell, feelings on skin and other things like inner ear balance and the like will be worth? 

Past experiences

In 2003, Second Life allowed us to create an avatar with desired physical and psychological traits and live the life of our dreams. Quite literally. You could choose to teleport, fly, drive expensive cars (if you could design them), or just plant a field of carrots and walk amongst carrots all day… Your imagination and creativity was the only thing holding you back. The problem was that the graphics weren’t good, and the delays and slow response were terrible. 

Games like GTA Online allowed you to do things such as heists, raid airbases, be chased by cops, and generally be the exact opposite of the real you.  But it was scripted fun. Yes there was a lot to do, and the attempt was to make it feel like total freedom, but it wasn’t. Minecraft is a similar title, which might be a gateway to the metaverse, but with far fewer illegal things happening on screen. Targeted at kids, it is a fun world-building experience that comes in handy for a future metaverse. 

The senses

Even though we have a long, long way to go before we can even be happy with the level of sight and sound we might experience in the metaverse, servers are going to take a massive beating if they have to render and send out visual information about hundreds of players in a relatively small area of the metaverse, those are still problems that are fixable, or can be worked around. In terms of sight and sound, we might be 40 or 50 per cent of the way towards mimicking real life. For the other senses, however, we’re essentially at 0 per cent, and that’s where a lot of research is being focussed. 

Smell

This might be the sense that we’re most capable of replicating. Work was ongoing for a long time before the concept of metaverse was ever dreamed up. This is because people wanted to digitally create smell for a lot of marketing reasons. Imagine seeing a video ad on TV / online about a bakery, and then smelling freshly baked goodies. Or imagine an ad for a beach holiday in the Maldives, and you get a whiff of sea salt and freshly cut coconuts… you’d be far more likely to buy the holiday package if you sensed it in more ways than visual, right? 

And so people have been researching smell for some time now. In fact, a company called OVR technology is already selling scent devices to corporate and governments that attach on to VR headsets to then allow the programmers of content to script smells into the mix as well. In demos, they’ve shown people some roses, which you can pick up and smell, and reports are that the smell is very natural and real feeling. While it’s mostly earthy and natural scents that are commonly experienced such as flowers, citrus, pine forests and the like, it’s a start. And this is a product, there’s plenty more being worked on in laboratories. 

Another product already on sale comes from Feel Real, which is a VR headset that not only gives you smells, but also blows air at you if it senses fast movement or explosions, amongst other things. It uses a combination of nine individual aroma capsules to mix and create the smell of your VR world. They currently have 255 scents available.

Touch

Meta (Facebook) itself is working on, or rather, providing in house funding to researchers that are working on haptic gloves, which we have shown in the picture above. Reality Labs is like an incubator for research in the field of better simulating VR and AR, and their haptic glove is an attempt to solve touch inside a VR world. The goal here is to provide feedback to fingertips so that, let’s say, a sculptor could work in the metaverse in a clay simulator, and feel his creation. It would also allow you to type on a virtual keyboard, but give your fingertips the sensation of hitting keys, instead of just flapping them about in the air. 

The technology works by using many little microbubbles to inflate and deflate and provide the sensation of feeling to your fingertips. And they’ve developed a chip as well which they call a high-speed microfluidic processor to control and quickly inflate and deflate the tiny microbubbles that give your fingertips the sensation. It’s a super complex job tying the haptics into the visuals, so there are tonnes of physics computations happening. For example, imagine a leaking faucet, which you see in VR. You take your finger and place it under the faucet and the first drop misses, you adjust your finger and the second drop hits the right side of your finger, you adjust slightly again, and the third drop hits the middle of the finger. The first one is simple, do nothing. However, the second and third drops will need a very different computation to transfer the right sensation to your finger. 

Taste

There’s a concept device called Taste The TV (TTTV) that was developed by Meiji University professor Homei Miyashita. What this device does is spray a set of complex flavouring agents onto a screen that can be licked off to mimic the flavour of something displayed on it. The idea is essentially the same as what people are working on for smell, which is to narrow down smell and taste to a fixed set of fundamental ingredients, and then mix and match to change tastes or smells. There will be increased spending for more research and development now thanks to Meta pushing the metaverse concept to everyone, and we’re bound to hear of new breakthroughs in these fields.

Movement

One of the hardest things to mimic is the sensation of movement. We often forget about this and think of us humans as having only five senses, when in fact we have six. Proprioception (or kinaesthesia) is our ability to sense the body’s position and movement without looking. It’s why you can close your eyes and touch your nose. Alcohol (or drugs) messes with this ability of the body, and this is why traffic police will ask you to touch your nose or walk in a line while looking up or walk a few steps backwards to check if you are impaired. 

This also comes into play when we are inside a metaverse because we need to be able to sense movement. In fact, it’s when senses disagree about what we’re experiencing, a feeling of nausea arises. Most commonly, a visual and auditory sensation of movement while the body is sitting still causes us to feel sick. This is why so many suffer from car sickness on long journeys. It’s because there is not too much acceleration to remind us we’re moving, but the sound and sight of us moving at high speed causes mental dissonance and makes us dizzy. This is a problem we have solved with complex gyroscopes, usually used in military or astronaut training, but it’s not something that’s ready for your living room or gaming room just yet. 
 
Will any of these technologies develop fast enough to make the metaverse more real, or will the metaverse remain just a VR gimmick that a lot of us use from time to time, but don’t really “live” inside of. That is the ten trillion dollar question, isn’t it? 

Read more articles about the Metaverse.

This article was first featured in the January 2022 issue of the Digit magazine. To subscribe to the magazine, head to this link. To get the digital subscription, head over to Geek.Digit.in

Follow Us

Satvik Pandey

Satvik Pandey, is a self-professed Steve Jobs (not Apple) fanboy, a science & tech writer, and a sports addict. At Digit, he works as a Deputy Features Editor, and manages the daily functioning of the magazine. He also reviews audio-products (speakers, headphones, soundbars, etc.), smartwatches, projectors, and everything else that he can get his hands on. A media and communications graduate, Satvik is also an avid shutterbug, and when he's not working or gaming, he can be found fiddling with any camera he can get his hands on and helping produce videos – which means he spends an awful amount of time in our studio. His game of choice is Counter-Strike, and he's still attempting to turn pro. He can talk your ear off about the game, and we'd strongly advise you to steer clear of the topic unless you too are a CS junkie. View Full Profile