TECH Meets BUSINESS
Virtual reality (VR) technology, in which the senses of sight and hearing are used, has been researched all over the world, and the technology reaches as far as it can go. In this situation, a factor that can bring a further real sensation that attracts attention is the “3D gesture feedback technology,” which was invented by AIST and on which Miraisens is working to commercialize it. This technology, which can reproduce “touch, weight, and resistance, as if things were really there” using a small device, is attracting people’s attention in Japan and overseas countries. How far can the future expand, if this “gesture feedback technology” is fully utilized?
Co-founder, CEO of Miraisens, Inc. After joining Sony Kihara Research Center in 1993, he was engaged in research and development of 3D computer graphics technology and VR technology. After leaving Sony in 2007, he participated in starting up a variety of technology companies such as 3DCG, robotics, and medical imaging. He founded Miraisens in April, 2014.
Founder, Chief Technical Officer of Miraisens, Inc., Doctor of Engineering. In 1995, he joined the Institute of Bioscience and Human-Technology at the Agency of Industrial Science and Technology in the Ministry of International Trade and Industry, and he was engaged in research and development in the fields of VR, information technology, neuroscience, ergonomics, and medical welfare. He moved to the Human Life Technology Research Institute at the National Institute of Advanced Industrial Science and Technology, and he is a senior research scientist at the Human Environment Interaction Research Group at the Human Informatics Research Institute. He founded Miraisens in April, 2014.
“Having a brain to misunderstand, and expressing sensation”
-- At CEATEC JAPAN 2015, which was held in October, the original technology “3D gesture feedback technology” won the Jury’s Special Award. What kind of technology is it?
When we touch something, we receive sensation through our skins and muscles such as “its surface is rough,” “it is dented when we hold it tight,” “it is a bit heavy,” and so on. The “3D gesture feedback technology” of Miraisens is what gives us a sensation that is similar to the sense of touch mentioned above by just using the vibration of a small device. When we touch the device, the stimulation from the vibration is sent through the hand to ultimately reach the brain, and the brain misunderstands and decides that “the hand is pulled or pushed,” “it is soft or hard,” and so on. That is why, even when there is nothing, we think that the hand is “pulled” or the finger is “touching something soft.” Everyone is surprised because such kind of sensations are expressed even when the eyes are closed.
The secret is in these specific vibration patterns. What vibration do we need to give so that the brain will misunderstand that the hand is pulled? What vibration can express the difference between “smooth” and “rough”? We delved into human sensory characteristics, created stimulation patterns to produce the sense of touch and force, digitized bodily sensations such as the sense of touch and a response, and digitally realized physicality that creates operability and the operational feeling. Those are the research results obtained by Dr. Nakamura.Dr. Norio Nakamura (hereafter Nakamura):
In colors, there are “the three primary colors,” which can produce various colors by combining them. Similarly, in skins, there are “three primary tactile senses.” Those are “the sense of force,” which feels as though it is being pulled and being pushed, “the sense of pressure,” which feels a touch, hardness, and softness, and “the sense of touch,” which feels the roughness of a surface. Combining these, we create one sensation. This means that if we control these three elements as with “the three primary colors,” we can create any type of sensation. Conventionally, we needed large equipment because sensation was created by physical forces, but we hit on an idea and recognized that “we only need to have the brain to misunderstand that there is a thing,” which led us to create the three primary tactile senses and to miniaturize a device.
“Tactile sense feedback device,” which transmits the tactile sense of CG to the fingertips, and which enables fine expressions such as hardness and roughness.
A flat type “gesture feedback device,” by which we can experience a feeling of pushing a button, rotating a dial, and clicking.
“Creating every bodily sensation world”
-- To what fields do you plan to apply that technology?
I have been asked this question in many places, but I always answer that “there are numerous application fields.” We can apply the technology not only to one field, but also to all the fields. It can be applied whatever a human sense of touch arises.
If a product has a touch panel, the use of “3D gesture feedback technology” enables intuitive operation. For example, even when a panel is a flat plane, this technology can give feedback so that we can see that “we surely pressed it” by providing sensations of pushing and clicking a button, and it can induce proper motion and action by providing a force to pull a finger. In this way, it is certain that this technology can add a large value to existing technologies such as games, mobile devices, automobiles, and medical treatments, but what we are aiming at is a much larger possibility.
The basic business of Miraisens is “to provide total solutions by utilizing a sense of touch and a bodily sensation.” It is not our goal to incorporate our technology in a product, but we have a broad and large vision in that we implement “bodily sensations” in every product, we construct a vast market of the bodily sensation business, and people all over the world can share space using bodily sensations. For example, when a new product is developed, instead of having a meeting online using a display from a remote place, if material is placed in the center and everyone can share it by touching it, we can spread physical and sensitive information such as interactive operability and the operational feeling, as though we share the space, can’t we? If we can feel the sensation of shaking hands with a person seen in the display, a feeling of closeness and a sense of unity can arise, which may change the relationship drastically. We think that sharing such bodily sensations is the fundamental communication of human beings; thus, our goal is to provide a new sensation world based on “3D gesture feedback technology.”Nakamura:
We are controlled by a view of the world from digital terminals, mainly by sounds and images, and we tend to expand the view of the world with a premise based on them. In contrast, this technology can create a new view of the world with a feeling of depth, adding a completely different dimension. This technology added a new axis, the sense of touch, to a place where there is only a passive two-dimensional world such as a plane formed by the conventional sense of sight and hearing, and a radically change the shape and functions of devices and service, removing the bias and the limit of development philosophy that we have been unconsciously obsessed with so far.
Those who experienced can understand, but if a technology that we thought “impossible” appears in front of us, our idea will be changed and we will consider that “then we may be able to do this, too,” right? In “3D gesture feedback technology” there is the innovative power to cause a chain reaction of such a release, and we have already started to have a partnership after we received ideas from several tens of companies. Making use of our creative original technology and manufacturing technology with the ability of planning and product appeal of Miraisens, we wish to lead haptics technologies in the world.
“In 2020, the use of TV would enable us to understand the sensations of athletes”
-- Will the day soon come when we can use products to which “gesture feedback technology” is applied?
After the technology was announced, product development with various spans started all at once. The application to game consoles will be realized within several years. Development of industrial robots and medical tools has already started. Within five years, this technology will be introduced in every product including appliances. At that time, we will not be able to imagine a world without “gesture feedback technology,” which is similar to the present time where we cannot imagine a life without cell phones.Nakamura:
At this time what we actively show is output devices that provide sensation, but at the same time we are developing input systems that sample sensation, and we are also trying to miniaturize those devices. Technically, reproduction of sensation can be done by having a brain to misunderstand, so it is possible to have a small device with its size close to that of a ring, also a noncontact device that uses a certain illusory stimulation. Speaking of five years later, the Tokyo Olympic Games will be held, right? I have played soccer, so I think that we can develop a device or a sensation TV that can provide the feeling and bodily sensations of athletes.
-- Listening to your stories, I feel that your teamwork is very good. Is there a division of roles to disseminate “3D gesture feedback technology”?
Mr. Koda has a broad view and experience that are useful for considering “what is the best way to appeal to users,” such as the commercialization of business and the direction of exhibitions, and he spares no effort to improve them. In contrast, I am good at finding out what technologies are needed to realize a creative future, and refining those technologies.Koda:
I am so to speak a type of a person who widely expands business horizontally. I proceed with my work dynamically and speedily, but sometimes I overlook a golden vein buried right below us. In such a case, Dr. Nakamura, using his experience as a researcher, says to me that “wait a second, dig one centimeter more, there is definitely something here.” I say “no way,” but if I dig, I can reach the gold vein exactly. Since the startup of the company, maybe we have been repeating this way of working. Neither Dr. Nakamura nor I can be lacking.
“We would like to be Doraemon”
-- Since January 6th, your products have been exhibited at the International CES (international appliances exhibition), held in Las Vegas, haven’t they? How are the responses?
I feel that CES is on a global scale, as I thought. CEOs and CTOs in the world top-level companies come to the booth of our company, experience a demonstration, and everyone is very surprised. It is very exciting, because discussions are heated there such that this can be incorporated to that product, we can apply this to such a thing, and so on.
“International CES 2016,” held in Las Vegas, United States.
Booth of MIRAISENS, Inc.
-- Can we experience “3D gesture feedback technology” in Japan?
Yes, at our office we have a place for demonstration, so anybody can come to experience the state-of-the-art devices. If you bring an idea about “I would like to do this kind of thing,” even at the level of imagination, the conversation will become concrete and the discussion will be heated. We would like to be “Doraemon”(Japanese popular animation character) in this field. If we are asked “Don’t you have such a tool?” we take a tool and say “Here it is,” and we would like to surprise the world and change it. This is a realm nobody has ever been to, so there are plenty of possibilities for a small idea leading to a new future. Partners who try to create a new future together are welcome, so please contact us.
* The contents of this article are based on information as of January 8th, 2016.
神奈川県横浜市緑区白山１丁目１８-１ 村田製作所 横浜事業所内
18-1, Hakusan 1-chome, Midori-ku, Yokohama-shi, Kanagawa 226-0006 Japan
*MIRAISENS,Inc. has become a wholly-owned subsidiary of Murata Manufacturing Co., Ltd.[22-Jan-20]
Research Institute for Earth Science Visualization Technology Co.,Ltd.
Visualizing the History of the Earth, and Creating a “Museum of the Future”!
- Precision 3D Modeling and Projection Mapping –
Mottainai Energy Co., Ltd.
Brightening the Future with Electricity Produced from Heat!
LEAG Solutions Corporation
New image measurement for the smart society!
- 3D position and attitude measurement using high-accuracy markers -
HikariPath Communications Co., Ltd.
Realtime 4K video eliminates distance using new optical communication technology!
Manipulating light to open up the future! Technology for measuring the unknown underwater world
Nanolux co. ltd.
Deliver Color Images in Darkness! Unique Technology Enabling Color Image only by Infrared Illumination
Penguin Instruments Inc.
Implementing Research Seeds in Society through Application of Light and Smell to Instruments
Realizing an Eye that can See like a Human!
– Development of Optical Steering Device and New Standard Lidar -
Reproduce human on the palm !
-Development of the world's first human protein array-
Site Sensing Inc.
Expanding a New Business with Excellent Measurement Technologies for Tracking, 3D Modeling , and Face Recognition
Controlling Voice to Open Up Next Generation Business!
- Commercializing AI and Voice Recognition Technology -
Automatic Generation of iPS Cells!
- Contributing to Gene Therapy Worldwide -
HSP Technologies Inc.
Producing New Materials by Mixing Materials that do can not Mix!
A feeling as if things were really there!
~ Digitization of physicality and bodily sensations ~
Supporting information society by technology to measure thermophysical properties
Tsukuba Technology Co., Ltd.
Thought for nondestructive inspections
Dedication to the ultimate material, diamond