Test Data Analytics - Exploring Hidden Correlations in Production Test Data

Post-silicon validation involves multiple stages of quality checking (Fig. 1). Due to the growing chip complexity and the close-to-zero Defective Parts Per Million (DPPM) quality requirements, more tests have been added to each stage in post-silicon validation. Beyond the traditional pass/fail information of each test item, there exist various correlations in the collected data from the test measurements. Exploring such hidden correlations in test data could lead to a lower test time/cost by removing redundant tests, a higher test quality by identifying additional defective chips that escape all the test items, a more efficient screening by reordering the test items, and a diagnosis of the measurement integrity.

Fig. 1: Post-silicon validation stages provide a huge amount of data for analysis.

Source: International Technology Roadmap for Semiconductors (ITRS)

For test cost reduction, a technique based on compressive sensing, named Virtual Probe (VP), can recover the spatial pattern on wafer of a test item from a small subset of sampled chips. Fig. 2 shows the prediction result based on only 10% sampled chips on wafer. While VP exploits the spatial correlation in production test data, another technique called Group LASSO (GL) identifies test items that can be predicted by the others and therefore can be skipped during test application for test cost reduction. Fig. 3 shows a test item identified by GL that can be predicted accurately by the linear combination of other test items.

Fig. 2: The original measurement, 10% sampled chips, and the prediction result from VP based on the 10% sampled chips for one test item.

Fig. 3: Test item 87 can be predicted by a linear combination of other test items with corresponding coefficients.

For improving the test quality, our target is to identify test escapes (chips that pass the entire test program but fail at system level) through statistical analysis based on existing test data without adding more physical measurements. Feature engineering is essential for extracting useful information from the production test data. Fig. 4 shows the standardized measurement values of two test items on two different wafers, each with a test escape. To detect the intrinsically abnormal test escapes, we can compare the measurement value of the test escape on the left wafer with the measurement mean (green-yellow ish color) of the wafer and expose the test escape as an outlier. This comparison, however, will not work for the test escape on the right wafer. Instead, if we compare the measurement value of the test escape on the right wafer with the spatial pattern of the wafer, the test escape can be exposed as an outlier for it is very different from its neighbors. The above mentioned comparisons are two possible features for the multivariate analysis to screen test escapes. Since there are diverse root causes for test escapes, it is important to develop multiple feature sets to reveal different aspects of the abnormalities of the test escapes.

Fig. 4: Real test escapes that are exposed as outliers when comparing their measurement values to different references.

After generating multiple feature sets that could potentially reveal some abnormalities of the test escape, the next step is to efficiently and effectively utilize the features for identifying the test escapes. One example is using the canonical analysis to project the feature data into another space in which the separation between normal chips and test escapes are compacted into the first few dimensions. Fig. 5 shows the distributions of good chips and test escapes in the projected space using principal component analysis (PCA) and using canonical analysis. There is a clear separation between the two populations in the 3-dimensional canonical space, which is not present in the 3-dimensional principal component space, which shows that the canonical analysis successfully compacts the multi-dimensional difference between the two classes of samples into the first three dimensions in the projected space.

Fig. 5: Distributions of the good chips and test escapes in the 3-dimensional projected space using principal component analysis (PCA) and canonical analysis.

Publications (as of 4/9/2015):


F. LinHsu, C. - K., and Cheng, K. - T. TimFeature engineering with canonical analysis for effective statistical tests screening test escapes, in Test Conference (ITC), 2014 IEEE International, 2014.


F. LinHsu, C. - K., and Cheng, K. - T. TimLearning from Production Test Data: Correlation Exploration and Feature Engineering, in Test Symposium (ATS), 2014 IEEE 23rd Asian, 2014.


S. ZhangLin, F.Hsu, C. - K.Cheng, K. - T. Tim, and Wang, H.Joint Virtual Probe: Joint exploration of multiple test items' spatial patterns for efficient silicon characterization and test prediction, in Design, Automation and Test in Europe Conference and Exhibition (DATE), 2014, 2014.


C. - K. HsuLin, F.Cheng, K. - T. TimZhang, W.Li, X.Carulli, J. M., and Butler, K. M.Test data analytics - Exploring spatial and test-item correlations in production test data, in Test Conference (ITC), 2013 IEEE International, 2013.