The future of
augment our bodies.
The future of mobility
will augment our
The day when technology will expand the capabilities of our bodies is almost upon us. Perhaps one augmentation of the body could be the i-ROAD, a vehicle that offers new ways to enjoy the city and fresh physical sensations to drivers thanks to a feel like becoming one with the wind while behind the wheel. Join us as we speak with Junichi Rekimoto, a human-computer interaction researcher who use augmented reality technology (AR) to study the expansion of human capabilities and the future of sports, about what changes mobility and our environment will bring to our bodies and cities.
Professor at the Tokyo University Graduate School of Interdisciplinary Information Studies, vice-president of Sony Computer Science Laboratories, human-computer interaction researcher. He is known as a consistent pioneer in his field, producing results such as “NaviCam”, the world’s first mobile augmented reality system, “CyberCode”, the world’s first tagging-based AR, and groundbreaking core studies in multi-touch technology. Rekimoto is also one of the co-founders of Koozyt, Inc., a venture company exploring new business opportunities in positional information and AR technology. In 2007 Rekimoto was awarded the CHI Academy※ from the ACM international academic society.https://lab.rekimoto.org
※ The CHI Academy is a group of researchers honored by SIGCHI, the Special Interest Group in Computer–Human Interaction of the Association for Computing Machinery. Wikipedia
We can augment sports
The future of
augment our bodies.
Professor Rekimoto, as someone active internationally in the field of human-computer interaction (HCI), what themes are you working on currently?
My research into the relationship between computers, humans, and technology continues, but most recently I am studying human augmentation. By this I don’t mean finding ways to improve our interactions with devices, like designing smartphones that are easier to use, so much as I do puzzling out the best ways to actually make technology a part of us as humans and improve our innate capabilities.
Along those same lines, you have recently been promoting the concept of an “Internet of Ability” (IoA). Could you explain a bit more about what this term means?
To put simply, I guess I would say it refers to technology that will allow us to share our human abilities over networks. This might sound a bit crazy, but I think that many of the abilities we described as “super powers” in the 20th century have actually been made reality by technology today. You could say that SNS and similar things are real-world versions of powers like telepathy that let you know what people far away are thinking.
What’s more, using virtual reality technology (VR) with cameras and sensors, we can also realize the ability of tele-presence, or seeming to be present in a different location.
We can also use technology to augment sports. One example is my research project “Jack-in”, where we are using video filmed with wearable all-around cameras that is then displayed on a head-mounted display (HMD) to create an experience like that of actually being inside someone else’s body. If you watch from the perspective of a pro athlete, for instance, you could assume their viewpoint and feel as if you were playing the sport yourself. Coaches could also jack-in to athletes and offer them advice.
Technology can be effective for physical training, too. Drones with cameras can be used to capture your movements from all angles in real-time while playing a sport, which could prove useful in improving your form. We’re using engineering to realize out-of-body experiences, or seeing ourselves from the outside, which is perhaps also like a super power.
It seems like this technology could also prove useful in therapy for the elderly or those who have trouble moving.
Yes. I’m not very athletic myself, so I was interested in the idea of what I could do to enjoy sports through technology. The IoA involves adopting technology to activate human behavior and enhance our capabilities. And this applies to me, too. I have found that as I wear devices I have made that take capture various readings of my body and record data while running that my jogs last a bit longer each time.
The “AquaCAVE” we have here in the lab is part of our research into combining sports, in this case swimming in a pool, with VR video. The walls of our aquarium-type pool feature three-dimensional projections. The videos give the experience of swimming through a coral reef in the ocean, but other simulations like spacewalks or soaring above the streets of Manhattan like a superhero are also possible. If we use a projection of a model swimmer, it can even help others improve their technique while training.
It’s also possible that technology could lead to the birth of entirely new sports. For example, we are studying something called “HoverBall”, in which a drone is actually built into a ball. If someone made a game where the ball can change direction freely while floating in midair, you could have all sorts of crazy techniques. The speed of the ball could also be adjusted in accordance to players’ strength and abilities. That would allow both adults and children to play and push their physical limits.
Technology that blends
The future of
augment our bodies.
What are your thoughts on the relationship between cities and sports? For example, one of your projects at the Rekimoto Lab was “Running Gate”, which involved placing sensors within urban spaces to record the forms of people running.
For Running Gate, we placed gates with sensors attached along jogging trails. When runners passed through the gates, we could record their running form in 3D without the need for them to wear any special markers or tags. The fundamental idea here was to seamlessly blend technology into the environment of the city.
What are some other ways to implement technology in cities aside from sports?
We’re also doing research on smart windows called “Squama” for future architecture. The project arose from the question of how we can use the environment to make things more pleasant for people living in the city. Squama means “scale”, as in the scale of a fish, but the actual technology involves windows that can adjust their transparency portion by portion. Say there is a day that is very sunny, but you want to keep a certain part of a room dark. If you use Squama for the surface of your building, you can keep make just the portions of the windows over that part provide shade.
When you start talking about putting liquid crystal technology into windows it’s easy to imagine that it means inputting information into them. But, this also raises concerns about whether a world flooded with information is actually pleasant. What we’re more interested in are ways to resolve the more basic desires in life like “I wish it was less bright” or “I wish I had more privacy” that are integral to creating comfortable spaces for individuals within cities full of all sorts of people.
What sort of technology fulfills these personal and basic desires?
I feel there is potential in a hybrid of real physical sensations and the sort of experiences only technology can provide. That’s why I think we need to know about those experiences that can’t be replaced digitally in addition to researching VR, which is vision-centric. You can view realistic videos of going to a sushi restaurant with an HMD, but you can’t actually eat anything. VR can’t even recreate a single cup of water, after all. We use VR for the visual portions of the AquaCAVE as I mentioned earlier, but we have people actually swim in the pool so that they can experience the real resistance of the water.
There is a valley between the reality we are capable of recreating with VR and true reality, so I don’t think there is any fear of people becoming completely absorbed in some fantasy world as is often said. On the contrary, I think perhaps there will always be plenty of things in the realm of human happiness that cannot be replicated with technology.
Unifying the body and
The future of
augment our bodies.
If a person’s physical sensations are augmented by technology, will that also change their relationship with mobility?
So let’s take skiing for example. When you lean and pull off a really good turn while gauging the friction of the snow, it really feels like you are one with the skis.
It’s the same with any sort of mobility. While we are coming closer and closer to seeing true automatic driving with AI, there is also a growing call for the sort of comfort that bringing people and machines closer together creates.
Before I rode the i-ROAD for the first time it looked like it would tip over if it got a flat tire, but it turned out there was no fear of that. The vehicle is designed in such a way that it absolutely will not fall over.
This amazing technology allows the drivers to have a thorough command of their vehicle and enjoy such a sense of undiluted unity that they even forget how it all works. When the vehicle’s chassis tilts, the driver’s body naturally leans, too, so the physical experience is quite direct and unfiltered.
What do you think mobility will be like in the future?
I think it could end up positioned as something that brings an even greater level of premium experiences. This will be particularly true in cities with solid transportation infrastructure, where the value of the act of transportation itself will take on importance. I think we might see divergent paths of technology, with some exploring further comfort for passengers, and some aimed at making driving more fun.
There is one future depicted in a work of science fiction where traveling by car is the most extravagant experience in a world where teleportation has become the norm. Having the time to spare on traveling slowly had become a sort of status symbol.
The progress of automatic driving and services like Uber is only natural. But, much like with the question of why people play sports, I don’t think greater convenience is all that we’re after. I think there will always be a movement towards using technology to improve the fun of driving, too.
If you were to create a future form of mobility, what would you make?
When driving I’ve often thought I would like to make a form of augmented reality that would make it appear as if there were zebra or rhinoceroses around me. All that dull highway scenery would be replaced with a simulation of dashing through the wild savannah. That would make driving through urban spaces pretty exciting, wouldn’t it! But of course this is only if the technology of the time could guarantee safe driving while doing so (laughs).
Edit by Arina Tsukada / Text by Hirokuni Kanki / Photo by Takeshi Shinto / Translation by Luke Baker