TECH Meets BUSINESS

About 2015 I was involved in the development and testing of high-accuracy markers as a member of the technical staff under Hideyuki Tanaka (Technical Consultant), a Senior Researcher at AIST. I had never encountered 3D marker systems during several tens of years involvement in semiconductor manufacturing equipment and 2D and 3D image processing. Having spoken to Mr. Tanaka about their usefulness I strongly felt that they had great prospects for the future, and this led to the founding of the company. After establishing the prospects for commercialization, the company was founded in May 2018.

The time is approaching for practical application of technologies using real-time image processing and 3D spatial information, such as the start of 5G, autonomous vehicles, etc. For this, technologies for bringing into 3D correspondence the actual physical world and the virtual world of cyberspace and for exchanging information will be important. Using our company’s high-accuracy markers it is possible to construct a 3D coordinate space with the marker as origin, with one image taken by a camera. In other words, when the high-accuracy markers are used, 3D spatial information can be incorporated into a 2D image, and using this cyberspace can be precisely overlapped with physical space. This is possible with all kinds of camera, such as a digital camera, smartphone, etc.

Image processing and spatial measurement technologies are becoming more and more important. In these times when images are becoming generally easier to use, in association with the timing in which the value of images themselves is increasing, I felt that the range of application of this marker business is widening. I think that this will be recognized by many people and that we will contribute to modern society.

High-accuracy markers are produced from a combination of a lenticular lens and a fine stripe pattern. A lenticular lens is a fine semi-cylindrical lens arranged horizontally, having the special effect that it shows a different picture depending on the angle that it is viewed from. In high-accuracy markers the picture is not switched, but it is designed so that a black line can be seen to move continuously.

Our name, LEAG (LEnticular Angle Gauge), has been derived from this lenticular lens part, so the company name of LEAG Solutions incorporates the concept of using this technology in various forms.

The black lines move depending on the angle viewed

Structure of LEAG using a lenticular lens

Their function is that they can enable the measurement of the six degrees of freedom of position and attitude from X, Y, Z, roll, pitch, and yaw, the same as conventional AR markers. The disadvantage of conventional markers is that their angular measurement accuracy is extremely low, with large errors of up to 10 degrees. With high-accuracy markers the movement of the black lines is photographed with a camera and image analysis is performed, so that the angle can be measured with an accuracy of less than one degree.

In addition, positional information can be stably measured because the manufacturers produce the patterns to extremely high accuracy. In numbers, data can be obtained with errors of 1 mm or less or 1 degree or less, which is more than 10 times better than the accuracy with conventional markers. As long as the markers are visible measurement can be carried out with this guaranteed level of accuracy. That is why we refer to these markers as “high-accuracy markers”.

We consider that the four major fields are measurement, control, AR, and positioning. First, in the measurement field 3D measurement and tracking the movement of objects is carried out as a measuring instrument. It can also be used in the same way as motion capture technology. In the control field, in robotic control such as autonomous driving and drones, the acquisition of data on angle and position enables application to constructing systems that control moving bodies.

In the AR field markers are mainly used as positions for the entry of characters. At present uses as guidance for obtaining positional information with smart glass, etc., are increasing. In this field, displays can be smoothly produced if positions are stably known. It is considered that one of the advantages of our markers is that they enable extremely important reality to be constructed.

At present positioning is a field where navigation using GPS is widely used. With our markers not only 2D positions on a flat plane, but also 3D position and attitude data can be obtained, so it is possible to know what direction an object is oriented and at what height. One example of application is targets in indoor spaces. There are other methods such as beacons, etc., but there are no methods that enable data to be obtained with millimeter order accuracy, so systems that enable position to be known indoors with high accuracy can be proposed. Even when positioning markers are separated by 10 m, large size markers that can be seen on the camera are produced, and in the black line pattern the line moves sensitively unlike with normal size markers. With the mechanism in which the line moves when there is a small angular movement, the accuracy achieved maintains errors to within 10 cm and less than 0.1 degree.

Different size high-accuracy markers depending on the application

Precise positioning is possible by just taking a photograph of the marker with a camera

We have prepared a library of software for incorporation into systems with the concept that the software can be extended into various fields and used in various ways. Therefore, the first step is to incorporate the software. From my experience as a software engineer, what we want to do is to provide this to the many people that think they do not want to spend any time on measuring position and really want to be doing something else.

Our thinking is that we want the systems and cameras being used by each customer to continue to be used as they are, that we provide the software and the markers, and that they try using them together. We think that co-creation by many people in a variety of fields using the product little by little is the style of operation for this day and age.

Also, there are strong points and weak points regarding difficult matters, such as 3D spatial images, methods of calculation, cameras, etc. We provide support for all our customers through my own knowledge of image processing and Tanaka’s know-how and knowledge of applied fields as the technical consultant.

In contrast to other measurement technologies, such as LIDAR used in autonomous driving and construction of 3D shapes from camera images, etc., that measure the 3 degrees of freedom of a point, our high-accuracy markers enable measurement of the 6 degrees of freedom of the position of a point and the attitude of a plane. They can be measured with a single camera, which has the advantage that the system is very compact and can be constructed at low cost.

The market scale of autonomous driving, robot control, and positional information exceeds 30 trillion yen, and we would like to capture several percent of this market. Technologies have advantages and disadvantages, so they should complement each other. Therefore, we do not want to say “completely replace your system”. These markers do not solve everything, so our stance is to try it as one advanced system. For example, LIDAR and beacons have their advantages. In environments where these are difficult to use, for example where the electrical power infrastructure is not provided, these markers can be used without electrical power, so they can support the operations.

In the IoT and smart society, it is important that the position, attitude, and movement of people and objects be incorporated into cyberspace. It is said that a major task is to enable use both outside and inside buildings, and to enable use as social infrastructure. At this time when 5G is about to start, we must start by getting the recognition of those core persons responsible. Gradually coming to the fore, we will bring forward these technologies and products at the right timing to catch people’s attention. As the use of drones increases, special control will become necessary, and what we have to do is precisely link positioning and imaging technologies as what is necessary for the future. I want to see this technology play a role in realizing the smart society in which various technologies and systems are seamlessly connected.

* The contents of this article are based on information as of January 8, 2020.