• Issue 4,2022 Table of Contents
    Select All
    Display Type: |
    • Preface to Special Issue on Analysis and Verification of Intelligent Systems

      2022, 12(4):351-353. DOI: 10.21655/ijsi.1673-7288.00279

      Abstract (61) HTML (0) PDF 606.92 K (97) Comment (0) Favorites

      Abstract:Preface to Special Issue on Analysis and Verification of Intelligent Systems

    • Minimal-unsatisfiable-core-driven Local Explainability Analysis for Random Forest

      2022, 12(4):355-376. DOI: 10.21655/ijsi.1673-7288.00280

      Abstract (40) HTML (0) PDF 3.06 M (90) Comment (0) Favorites

      Abstract:As Machine Learning (ML) is widely applied in security-critical fields, the requirements for the interpretability of ML also increase. The interpretability aims at helping people understand internal operation principles and decision principles of models, so as to improve models' credibility. However, research on the interpretability of ML models such as Random Forest (RF) is still in the infant stage. Considering the strict and standardized characteristics of formal methods and their wide application in the field of ML in recent years, this study leverages formal methods and logical reasoning to develop an ML interpretability method for interpreting the prediction of RF. Specifically, the decision-making process of RF is encoded into a first-order logical formula, and a Minimal Unsatisfiable Core (MUC) is taken as the core. Local interpretation of feature importance and counterfactual sample generation methods are provided. Experimental results on several public datasets illustrate the high quality of the proposed feature importance measurement, and the counterfactual sample generation method outperforms the existing state-of-the-art methods. Moreover, from the perspective of user-friendliness, the user report can be generated according to the analysis results of counterfactual samples, which can provide suggestions for users to improve their situation in real-life applications.

    • Multi-path Back-propagation Method for Neural Network Verification

      2022, 12(4):377-401. DOI: 10.21655/ijsi.1673-7288.00281

      Abstract (72) HTML (0) PDF 5.49 M (105) Comment (0) Favorites

      Abstract:Symbolic propagation methods based on linear abstraction play a significant role in neural network verification. This paper proposes the notion of multi-path back-propagation for such methods. Existing methods are viewed as using only a single back-propagation path to calculate the upper and lower bounds of each node in a given neural network, so they are specific instances under the proposed notion. Leveraging multiple back-propagation paths effectively improves the accuracy of this kind of methods. For evaluation, the proposed multi-path back-propagation method is quantitatively compared with the state-of-the-art tool DeepPoly on benchmarks ACAS Xu, MNIST, and CIFAR10. The experiment results show that the proposed method achieves significant accuracy improvement while introducing only a low extra time cost. In addition, the multi-path back-propagation method is compared with the Optimized LiRPA, a tool based on global optimization, on the dataset MNIST. The results show that the proposed method still has an accuracy advantage.

    • Deep Learning Test Optimization Method Using Multi-objective Optimization

      2022, 12(4):403-436. DOI: 10.21655/ijsi.1673-7288.00282

      Abstract (52) HTML (0) PDF 1.93 M (85) Comment (0) Favorites

      Abstract:With the rapid development of deep learning technology, research on its quality assurance is raising more attention. Meanwhile, it is no longer difficult to collect test data owing to the mature sensor technology, but it costs a lot to label the collected data. To reduce the cost of labeling, the existing studies attempt to select a test subset from the original test set. The test subset, however, only ensures that the overall accuracy (the accuracy of the target deep learning model on all test inputs of the test set) of the test subset is similar to that of the original test set; it cannot maintain other test properties similar to those of the original test set. For example, it cannot fully cover all kinds of test input in the original test set. This study proposes a method based on multi-objective optimization called Deep Multi-Objective Selection (DMOS). It firstly analyzes the data distribution of the original test set by Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN). Then, it designs multiple optimization objectives given the characteristics of the clustering results and then carries out multi-objective optimization to find out the appropriate selection solution. Massive experiments are carried out on eight pairs of classic deep learning test sets and models. The results reveal that the best test subset selected by the DMOS method (the test subset corresponding to the Pareto optimal solution with the best performance) can not only cover more test input categories in the original test set but also estimate the accuracy of each test input category extremely close to that of the original test set. Meanwhile, it can also ensure that the overall accuracy and test adequacy are close to those of the original test set: the average error of the overall accuracy estimation is only 1.081%, which is 0.845% lower than that of Practical ACcuracy Estimation (PACE), an improvement of 43.87%. The average error of the accuracy estimation of each test input category is only 5.547%, which is 2.926% less than that of PACE, an improvement of 34.53%. The average estimation error of the five test adequacy measures is only 8.739%, which is 7.328% lower than that of PACE, an improvement of 45.61%.

    • Transfer-based Adversarial Attack with Rectified Adam and Color Invariance

      2022, 12(4):437-452. DOI: 10.21655/ijsi.1673-7288.00283

      Abstract (51) HTML (0) PDF 2.44 M (84) Comment (0) Favorites

      Abstract:Deep Neural Networks (DNNs) have been widely used in object detection, image classification, natural language processing, speech recognition, and other fields. Nevertheless, DNNs are vulnerable to adversarial examples which are formed by adding imperceptible perturbations to original samples. Moreover, the same perturbation can deceive multiple classifiers across models and even across tasks. The cross-model transfer characteristics of adversarial examples limit the application of DNNs in real life, and the threat of adversarial examples to DNNs has stimulated researchers' interest in adversarial attacks. Recently, researchers have proposed several adversarial attack methods, but most of these methods (especially the black-box attack) have poor cross-model attack ability for defense models with adversarial training or input transformation in particular. Therefore, this study proposes a method to improve the transferability of adversarial examples, namely, RLI-CI-FGSM. RLI-CI-FGSM is a transfer-based attack method, which employs the gradient-based white-box attack RLI-FGSM to generate adversarial examples on the substitution model and adopts CIM to expand the source model so that RLI-FGSM can attack both the substitution model and the extended model at the same time. Specifically, RLI-FGSM integrates the RAdam optimization algorithm into the Iterative Fast Gradient Sign Method (I-FGSM) and makes use of the second-derivative information of the objective function to generate adversarial examples, which prevents the optimization algorithm from falling into a poor local optimum. Based on the color invariance property of DNNs, CIM optimizes the perturbations of image sets with color transformation to generate adversarial examples that can be transferred and are less sensitive to the attacked white-box model. Experimental results show that the proposed method has a high success rate on both normal and adversarial network models.

    • Safe Reinforcement Learning Algorithm and Its Application in Intelligent Control for CPS

      2022, 12(4):453-483. DOI: 10.21655/ijsi.1673-7288.00284

      Abstract (52) HTML (0) PDF 24.92 M (99) Comment (0) Favorites

      Abstract:The design of a safe controller for a Cyber-Physical System (CPS) is a hot research topic. The existing safety controller design based on formal methods has problems such as excessive reliance on models and poor scalability. Intelligent control based on Deep Reinforcement Learning (DRL) can handle high-dimensional nonlinear complex systems and uncertain systems and is becoming a very promising CPS control technology, but it lacks safety guarantees. This study addresses the safety issues of Reinforcement Learning (RL) control by analyzing a typical case of an industrial oil pump control system and carries out research on a Safe Reinforcement Learning (SRL) algorithm and intelligent control application. First, the SRL issue of the industrial oil pump control system is formalized, and a simulation environment of the oil pump is built. Then, by designing the structure and activation function of the output layer, an oil pump controller in the form of a neural network is constructed to satisfy the linear inequality constraints of the on-off operations of the oil pump. Finally, in order to better balance the safety and optimality control objectives, a new SRL algorithm is designed based on the Augmented Lagrange Multiplier (ALM) method. A comparative experiment on the industrial oil pump shows that the controller synthesized by the proposed algorithm surpasses existing similar algorithms both in safety and optimality. During the evaluation, the neural network controllers synthesized in this study pass rigorous formal verification with a probability of 90%. Meanwhile, compared with the theoretically optimal controller, neural network controllers achieve an optimal objective value loss of 2%. The proposed method is expected to be applied in more scenarios, and the case study scheme may provide a reference for other researchers in the field of safe intelligent control and formal verification.

    • Database Translation Mechanism: Generating Data Dictionary for Relational Database

      2022, 12(4):485-499. DOI: 10.21655/ijsi.1673-7288.00291

      Abstract (93) HTML (0) PDF 1.41 M (215) Comment (0) Favorites

      Abstract:In order to optimize workflow and improve efficiency, modern enterprises usually entrust software providers to build the Enterprise Information System (EIS). However, the software providers generally do not provide the data dictionary for the EIS relational database, which brings great difficulties for enterprises to use the data stored in the EIS database. This paper proposes a method named database translation mechanism for generating the data dictionary of the EIS relational database which only utilizes the data collected from the interfaces of EIS. Inspired by the great success that Graph Neural Networks (GNNs) have achieved, the study builds a graph structure for the EIS relational database by mining the relationships between columns and then train a GNN-based classifier to predict to which column the value belongs. In addition, a table-based sampling method is designed to construct graph datasets for mini-batch training on the large-scale graph structure. Furthermore, a uniform encoding method and a hybrid aggregator function are proposed to improve the performance of the GNN-based classifier. The trained GNN-based classifier can be used to predict the matching relationships between the columns of the EIS database and the tables extracted from the EIS interfaces given the fact that the data in the database is entered at the interfaces. In this way, the detailed information at the interface to translate the database can be used. Experimental results on a real-world ERP relational database demonstrate the superior performance of the proposed method, which efficiently exploits and utilizes the graph structure information in the relational database.

    • An Extensible Heterogeneous Network Embedding Framework for Knowledge Tracing

      2022, 12(4):501-515. DOI: 10.21655/ijsi.1673-7288.00292

      Abstract (72) HTML (0) PDF 3.48 M (104) Comment (0) Favorites

      Abstract:Knowledge tracing is of great significance for providing better personalized learning guidance and has thus attracted extensive research attention in recent years. The task of knowledge tracing is to model students' learning process on the basis of historical exercise records and trace students' knowledge proficiency, thereby predicting students' performance on future exercises or recommending exercises for better proficiency. Existing methods focus on either the skill level or the exercise level, ignoring the relationships among exercises and Knowledge Components (KCs). The classical single-factor models include the Deep Knowledge Tracing (DKT) {model} and the Dynamic Key-Value Memory Network (DKVMN) model. Although a few models, such as the Bayesian Knowledge Tracing (BKT) model and the Knowledge Proficiency Tracing (KPT) model, utilize the Q-matrix to improve model performance, most of them ignore the interaction among KCs, not to mention models that do not use the Q-matrix. Inspired by the recent success of network embedding, this paper presents a heterogeneous network embedding framework for knowledge tracing called HNEKT that takes both exercises and KCs into account. To adapt to the application of knowledge tracing, this paper also proposes several meta-paths to generate meaningful node sequences for network embedding. Besides, it explores other side information as well to improve the extensibility and effectiveness of the proposed model. Extensive experiments on three real-world datasets demonstrate the effectiveness of the HNEKT model.