Reading Technology of Data Matrix under High Stressful Conditions

In recent years, the importance of traceability is increasing to respond to quality problems such as recall of automobile industry. In order to realize traceability and fulfill its social responsibility, companies are trying to expand the object of individual management by printing two-dimensional symbols on everything and every place. For the two-dimensional symbols to be printed, Data Matrix is often used because of space saving of printing and flexibility of printing shape. However, with the expansion of the range of objects to be printed, there is a problem that Data Matrix cannot be stably read in the production lines. This is because stressful conditions such as dirt and damage are newly added to Data Matrix due to the change in the reading environment accompanying the expansion of the reading process and the diversification of the part shape.
In this paper, we propose Data Matrix reading technology considering the reading environment of the production lines, and confirmed the performance improvement compared with the conventional technology. We believe that Data Matrix can be read stably under this high stressful condition.

1. Introduction

In recent years, the importance of traceability often goes up to the topic when recalls of automobiles or food contamination occurs. What is the reason for this? To respond to this question, the importance and trends of traceability will be described in this section.
The term “traceability” combines “trace” and “ability.” In the manufacturing businesses, the term is often used as “to enable the tracing of production history data such as manufacturers, suppliers and distributors in the respective processes of the procurement, processing, assembly, distribution and marketing of raw materials or parts.”
The realization of traceability enables businesses to enjoy the benefits of: (1) fulfilling corporate social responsibility; (2) defending the safety and security of consumers; and (3) enhancing corporate competitiveness. Regarding (1), when a quality-related issue occurs such as a defect or failure of a product, the company can take adequate actions rapidly by tracing when and where the causing raw materials or parts are used. In this manner, the company can fulfill its corporate responsibility without rousing distrust among consumers and client companies^1）. Regarding (2), for a product that is in the hands of consumers, retroactive tracing of data along the time sequence may verify whether it is genuine or fake, or whether it contains allergens, thereby enabling consumers to use the product safely1). Regarding (3), because all the production data within the manufacturing processes are recorded, the company can analyze such data to understand of the causal correlation of production quality. Thereby, the competitiveness of the company can be improved by improvement of production quality^2）.
Since realizing traceability requires the capability of recording and tracing information of when, where and how raw materials or parts are used, individual article management wherein unique numbers are assigned to raw materials or parts is required. One method for realizing individual article management uses one-dimensional or two-dimensional symbols on labels on which a unique number is printed^3）. In recent years, two-dimensional symbols are often used because the amount of information to be managed is increasing^4）. Furthermore, under an environment where the intended article is small or the label tends to peel off, Direct Part Mark (DPM), which prints a twodimensional symbol directly on the intended article, is used. A Data Matrix is frequently used because it only requires a small amount of printing space and the print shape (square/rectangle) can be chosen.
In recent years, in the automobile industry, the number of parts intended for individual article management is increasing. Behind this situation are the following two points: (1) For critical safety parts, the introduction of individual article management is being accelerated^2） after the issue of airbag recalls, because even a single recall affects business operations; and (2) The number of on-board electrical components is increasing in association with the shift to EVs. As a result, Data Matrix are increasingly being printed on all sorts of objects and places, covering large parts such as engine blocks and small parts such as inductors. From the viewpoint of traceability, the objects that have unreadable Data Matrix would be discarded even if that have no problem in terms of product quality, because it could lead to the disruption of records of individual objects. For this reason, there is an increased need for technology to read Data Matrices in a stable manner even under high-load condition.

2. Conventional Technology and Problems

2.1　Features of Data Matrix

Data Matrix is defined in ISO/IEC 16022^5）. Fig. 1 shows the features of the code structure.

The minimum structural unit of Data Matrix is called “module” and the centers (grids) of the modules are arranged in a checkerboard pattern [Fig. 1(a)]. A Data Matrix consists of finder pattern and data [Fig. 1(b)]. The finder pattern consists of a width of one module and is divided into an “L” shape arranged at the left and bottom sides, and a clock track arranged at the right and the top sides. The “L” shape is arranged only for the dark module and is used to establish the position and orientation of the Data Matrix. For the clock track, light/dark modules are arranged alternately, and it is used for establishing grids. As just described, the finder pattern assumes a very important role.

2.2　Conventional Technology

Omron Corporation has had Data Matrix reading technology for some time^6）. Fig. 2 shows the flow chart of reading in the conventional technology.

The “detecting L shape” process finds the “L” shape of the finder pattern among the images captured. The “estimating grid” process calculates the clock track from the “L” shape detected and calculates the grid. The “binarizing module” process judges, for all grids, whether the grid is a dark module (1) or light module (0). The “converting to characters” process converts the binarized 0/1 data into a string.
The action to vary conditions and execute the converting to characters again to enhance the possibility of readability is called “retry or retrying.” Conventionally, the retries were implemented by varying the threshold value of the module binarization.
Furthermore, the retry includes the method like the image pre-processing such as morphological operation^7）.

2.3　Problems

Because of the fact that the number of intended parts for individual article management is increasing and Data matrices are printed on all sorts of objects and places, a problem whereby the codes cannot be read stably on production lines has occurred. Fig. 3 shows some examples.

The situation of Fig. 3(a) occurs when the imaging environment cannot be optimized, including cases where a reader is retrofitted to an existing process. In this case, the Data Matrix becomes unreadable because the contrast thereof becomes lower or the S/N ratio decreases. The situation of Fig. 3(b) occurs when the imaging environment cannot always be optimized because the printing positions of symbols are slightly displaced or for other reasons. In this case, halation occurs and the luminance of part of the finder patterns or the data domain varies, and the code becomes unreadable. The situation shown in Fig. 3(c) is triggered by the adhesion of oil contamination or water drops or partial damage of the symbols caused by scratches after the article passes through many processing processes. In this case, the Data Matrix to be imaged is partially lost, and when the finder pattern, which is particularly important for reading, is damaged, the code becomes unreadable. The situation shown in Fig. 3(d) occurs when printing with a highspeed inkjet printer or pin-dot matrix printing, and the dot array is not maintained in a constant manner. In this case, the code becomes unreadable owing to uneven dot intervals or waving of straight lines.
As stated above, there are many cases where the Data Matrix becomes unreadable because the finder pattern is subjected to stresses that make the contrast unstable, damaged or distorted, thus resulting in failure to detect the Data Matrix.

3. Technical Details

3.1　Corrective measures for finder pattern

With the proposed technology, Data Matrices can be read with high accuracy by executing corrective measures when the finder pattern is subjected to stress. Fig. 4 shows the reading flow chart of the proposed technology. The portions shown in red show the differences from the conventional technology^6）.

It was arranged that various corrective measures for finder patterns can be implemented by adding retries to image processing, “L” shape detection and grid estimation.
An example of a retry for grid estimation is the case of a partial loss or contaminated clock track. The number of modules that can be arranged on one edge of the Data Matrix is defined by ISO/IEC 16022, which will be shown in Table 1^5）. Therefore, when the number of modules of the detected clock track does not agree with the defined number, the module center position of the clock track is estimated based on the defined number.

When executing grid estimation, the central coordinates of the modules which constitute a clock track are required. Fig. 5 shows some examples for cases where a clock track module is missing.

As shown in Fig. 5(a), when there is no stress, the respective centers of the light/dark modules can be obtained. As shown in Fig. 5(b), when the second dark module from the left end is missing owing to halation, the light module space expands, and the center-to-center clearance of modules becomes partially non-dense. If this is the case, the center positions of the clock track are added, as shown in Fig. 5(c), by referring to the publicly known number of modules.
Fig. 6 shows some examples when the clock track is contaminated.

As shown in Fig. 6(a), when there is no stress, the respective centers of the light/dark modules can be obtained. As shown in Fig. 6(b), when the third light module from the left end is contaminated, the center-to-center clearance of modules becomes partially dense because the number of dark modules increases. If this is the case, the center positions of the clock track are deleted, as shown in Fig. 6(c), by referring to the publicly known number of modules.
By applying such corrective measures, a more accurate grid position can be obtained, thereby ensuring higher reading accuracy.

3.2　Determination of retry sequence

While the advantage that higher reading accuracy can be ensured by repeating retries many times, there is a disadvantage in that the processing time is increased.
As shown in Fig. 4, the proposed technology is designed so that retry is repeated while varying the parameter values including the binarization threshold values of the modules and types of images to be created, and the processing can be completed when reading is completed once. Therefore, it is assumed that the processing time can be shortened stochastically by implementing retry starting from the parameter that is highly likely to be read, then implementing retry of a parameter that is not likely to be read later.
The method for determining the parameter order in retries will be explained using Fig. 7.

First obtain combinations of parameters for implementing retries, including the binarization threshold values of modules and types of images to be created [Parameter Rows A to C in Fig. 7(a)]. Next, prepare a sample image, evaluate the possibility of reading for the respective combinations, and obtain the score [Score Row of Fig. 7(a)]. Finally, determine the parameter order based on the score so that retrying can be implemented starting from the parameter that is more likely to be read [Fig. 7(b)].
In general, the operation to acquire the information necessary to read symbols using sample images is called “teaching.” With the proposed technology, the optimization of the parameter order in retries is also implemented during teaching.

4. Effects

4.1　Verification Method

To verify the improvement in performance according to the proposed technology, verification was implemented using an evaluation image database that simulates the situation of production sites. The evaluation image database consists of images covering stressful conditions and types of work that are supposed to be present at production sites as closely as possible. Among such images, several images presenting stressful conditions including low contrast, low S/N ratio, halation, oil contamination, and variation of printing are shown in Table 2. Teaching was applied to one sampled image for each stressful condition and measurement was carried out for the images sampled.

4.2　Verification Result

Fig. 8 shows the recognition rate of the conventional technology^6） and the proposed technology for the images sampled.

The figure reveals that, for all stressful conditions, the recognition rate is improved by about twice as much, showing a very high value of approximately 100%.
For halation and oil contamination, since the specular ingredients of lighting become partially intensified, the illuminance value of the clock track significantly varies toward the white side along the way. As a result, the binarization result of the clock track is partially judged to be the light module and it is thus assumed that there is an omission in the clock track. Therefore, it can be assumed that grid estimation processing when the clock track is missing as described earlier becomes effective, thereby realizing a high recognition rate.
For printing variations, incorrect judgment occurs when estimating the module centers of the clock track because the centers are not aligned. Deviation of the dark module position of the clock track results in the judgment that it is a light module, and the protrusion of dark modules in the rows/columns that are adjacent to the clock modules results in the judgment that there are dark modules. Therefore, it can be assumed that the corrective measures of clock modules as described earlier are effective, thereby enabling the realization of a high recognition rate.
Fig. 9 shows the processing time of the conventional technology and the proposed technology for the images extracted.

The processing time is the average value of the times that are required for processing the images that could be successfully read with the Omron Image Sensor (Model FH-5050). The figure reveals that the processing time could be shortened by approximately one quarter for all stressful conditions. Although there are more combinations of retries than the conventional technology, it is assumed that the optimization of retries according to stressful conditions works well because teaching is implemented for each stressful condition.
For example, the factors of low contrast and low S/N ratio will be trade-off stresses from the viewpoint of processing time. When threshold values related to contrast are reduced to speed-up the processing time for the factor of low contrast, a lot of small amplitude noise are likely to be detected and the speed decreases in the low S/N ratio images, and vice versa. Even for trade-off stresses, on the other hand, a faster processing time can be realized, while securing a high recognition rate, because the scores corresponding to the respective stresses are correctly calculated.

5. Conclusion

There is the problem whereby the printing of Data Matrices on all sorts of objects and places makes such codes unreadable in a stable manner on production lines. Regarding this situation, we proposed reading technology to optimize the retry sequence through teaching by increasing the number of combinations of retries to implement corrective measures for finder patterns. We implemented performance comparisons with the conventional technology and verified the improvement in performance, wherein high accuracy and high speed are satisfied. Therefore, we believe the proposed technology can read Data Matrices in a stable manner even under highly stressful conditions.
In the future, it is expected that the number of parts intended for individual article management will further increase and Data Matrices for the image becomes smaller, resulting in longer processing time. On the other hand, the size of parts is small, and the tact time allowed for inspection becomes shorter. To realize high-precision and high-speed reading even under such circumstances, we will improve the technology for “L” shape detection and grid estimation.

References

1）

Takayama, I. Current Status and Prospects of Automatic Recognition Technology Application in Traceability. Monthly Automatic Recognition. 2013, Vol.26, No.10, p.11-15.

2）

Ikematsu, Y. Quality control “panacea”. Nikkei Business. 2017, No.1916, p.48-52.

3）

Kokuryou, J. Digital ID Revolution. Nikkei Inc., 2004, 356p.

4）

Cognex Co., Ltd.. Barcode reader for traceability and worldwide procurement. Monthly Automatic Recognition. 2014, Vol.27, No.7, p.5-9.

5）

ISO/IEC 16022, Information technology – Automatic identification and data capture techniques – Data Matrix bar code symbology specification, 2006, p.3-5, p.16.

6）

Sakacho, K., Murata, T., Shiramizu, G. Two-dimensional symbol reader required in the manufacturing process of the automotive industry. OMRON TECHNICS. 2002, Vol.42, No.3, p.248-252.

7）

Sateesh Gopalakrishna Nadabar; Ranjit Desai; Method and apparatus using intensity gradients for visual identification of 2D matrix symbols. 2005, United States Patent, US6941026.

The names of products in the text may be trademarks of each company.