RU2745371C1 - Method and a system for prediction of cyber security risks during the development of software products - Google Patents

Abstract

FIELD: computing.SUBSTANCE: invention relates to computing. Disclosed is a computer-implemented method for predicting cybersecurity risks in the development of software products, in which data is obtained containing information about teams of software developers and software products being developed; processing the received data using a machine learning model trained on the basis of expert data on cybersecurity, in the course of which the following is carried out: division of the received data into categorical and numerical variables; processing the obtained variables, performing vectorization of categorical variables and normalization of numerical variables; concatenation of processed variables and building a vector based on them; assessment using the vector of the degree of occurrence of cybersecurity risks for each software product, and the classification of development teams with the assignment of the degree of probability of occurrence of the cybersecurity risk based on the assessment of the developed software products.EFFECT: invention improves speed and accuracy of predicting cybersecurity risks and classifying agile teams according to the degree of cybersecurity requirements fulfillment.7 cl, 4 dwg

Description

FIELD OF TECHNOLOGY

[0001] The claimed technical solution generally relates to the computing field, and in particular to an automated method and system for predicting cybersecurity risks when developing software products using machine learning algorithms.

LEVEL OF TECHNOLOGY

[0002] Software development for large financial institutions (banks) is always laborious and painstaking work. In addition, when developing a software product, it is necessary to take into account all the risks of cybersecurity defects. Cybersecurity experts are involved in these audits. which manually check for critical defects in the developed software product. The required number of cybersecurity experts working with development teams (for example, Agile teams) developing banking products depends linearly on the number of such teams. With the limited staffing of cybersecurity experts and the growing number of Agile teams, it becomes impossible for cybersecurity experts to fully and qualitatively participate in the work of all Agile teams.

[0003] The work of cybersecurity-savvy Agile software development teams is slowed by the involvement of cybersecurity experts, although such involvement is not required. At the same time, the cybersecurity expert does not have the opportunity to motivatedly assess whether the team is experienced, which entails the need to participate in their work and complicates the interchangeability of cybersecurity experts.

[0004] The work of inexperienced cybersecurity Agile teams developing software products requires a closer involvement of cybersecurity experts, as lack of attention to cybersecurity issues in the early stages of product development entails the need to make adjustments to the cybersecurity code at a later stage or prohibit implementation into industrial operation by a cybersecurity expert during acceptance tests. This entails an increase in the development time of software products, an increase in cybersecurity risks (as a result of the inorganic nature of cybersecurity functions and their inconsistency with the application architecture), demotivation of Agile team members and reputational losses for experts and the cybersecurity function in general.

[0005] From the prior art patent US 8631384 B2 "Creating a test progression plan" is known, patentee: IBM, published: 01.12.2011. This solution describes an automated process for generating software test plans. The known solution provides automatic creation of a software test execution plan by calculating for each unit of the testing period x the efforts to execute the ATTx test blocks and the efforts to complete the execution of the CCx test block. The calculation introduces three variables to characterize the testing strategy: performance, which represents the performance of the test team, the defect density ratio, and the validation ratio value. When choosing a test strategy, the test manager determines the values of three variables that affect the development plan. During test execution, the cumulative “try” curve of ATTx values and the cumulative curve “completion” of CCx values allow the test manager to compare the efforts already made with the expected efforts made for test blocks that have been made and for test items that have been completed, that is, when defects found in the code have been fixed.

[0006] A disadvantage of the known solution in the art is the lack of the ability to automatically predict cybersecurity risk and classify Agile teams according to the degree of fulfillment of cybersecurity requirements in software development.

DISCLOSURE OF THE INVENTION

[0007] The claimed technical solution proposes a new approach to predicting cybersecurity risks and classifying agile teams according to the degree of fulfillment of cybersecurity requirements in software development. This solution uses a machine learning algorithm that automates the process of predicting cybersecurity risks and classifying agile teams according to the degree of fulfillment of cybersecurity requirements.

[0008] Thus, the technical problem of automated cybersecurity risk prediction and classification of agile teams according to the degree of fulfillment of cybersecurity requirements is solved.

[0009] The technical result achieved in solving this problem is to increase the speed and accuracy of predicting cybersecurity risks and classifying agile teams according to the degree of fulfillment of cybersecurity requirements.

[0010] The specified technical result is achieved through the implementation of a computer-implemented method for predicting cybersecurity risks in the development of software products, performed using at least one processor and containing the stages, which are:

- receive data containing information at least about the teams of software developers and developed software products, each of the mentioned teams;

- carry out the processing of the obtained data using a machine learning (ML) model trained on the basis of expert data on cybersecurity on compliance with cybersecurity requirements by teams in the development of software products, and during this processing, the following is carried out:

- division of the received data into categorical and numerical variables;

- processing of the obtained variables, in which vectorization of categorical variables and normalization of numerical variables is performed;

- concatenation of processed variables and construction of a vector based on them;

- using the aforementioned vector to evaluate the degree of occurrence of cybersecurity risks for each software product, and - classification of development teams with the assignment of the degree of probability of occurrence of cybersecurity risk based on the assessment of the developed software products.

[0011] In one of the particular embodiments of the method, the data is processed using a machine learning model based on a random forest classifier.

[0012] In another particular embodiment of the method, the classification of commands is carried out with the assignment of high, medium and low probability of the occurrence of the cybersecurity risk.

[0013] In another particular embodiment of the method, information on development teams classified with a medium and high degree is automatically sent to the automated workstation of cybersecurity experts interacting with the development teams, with an increased control mark.

[0014] In another particular embodiment of the method, the data on the classified commands contain at least:

i. data on the tasks of the team members during the development of a software product in the task management system for the development of software products;

ii. data on the structure of the team and the professional qualities of the team members;

iii. data on communications between team members and cybersecurity experts in the development of software products for the entire lifetime of the team;

iv. data on the source code of software products released by the team.

[0015] In another particular embodiment of the method, the data on the developed software products contain at least:

v. data on the number of critical cybersecurity defects identified during acceptance tests of the team's software products released into commercial operation over the entire period of the team's existence;

vi. data on the number of critical non-cybersecurity defects identified during acceptance tests of the team's products released into commercial operation over the entire period of the team's existence;

vii. data on testing the finished software product of the team, before its release into industrial operation;

viii. data on the passage of checks by the static and dynamic analysis system for vulnerabilities in the finished software products of the team, before their release into industrial operation;

ix. data on vulnerabilities in the team's software products discovered after the release into industrial operation.

[0016] In addition, the claimed technical result is achieved through a system for predicting cybersecurity risks in the development of software products, containing:

- at least one processor;

- at least one memory coupled to the processor, which contains machine-readable instructions that, when executed by at least one processor, provide a method for assessing the likelihood of critical cybersecurity defects in acceptance tests of product releases.

BRIEF DESCRIPTION OF DRAWINGS

[0017] The features and advantages of the present technical solution will become apparent from the following detailed description and accompanying drawings.

[0018] FIG. 1 illustrates a block diagram of the implementation of the claimed method.

[0019] FIG. 2 illustrates the ROC curve (error curve) for a command classifier based on a random forest.

[0020] FIG. 3 illustrates an error matrix (without normalization) for a command classifier based on a random forest.

[0021] FIG. 4 illustrates an example of a general view of a computing system that provides an implementation of the claimed solution.

CARRYING OUT THE INVENTION

[0022] The following will describe the concepts and terms necessary to understand this technical solution.

[0023] A model in machine learning (ML) is a set of artificial intelligence methods, a characteristic feature of which is not a direct problem solution, but learning in the process of applying solutions to many similar problems.

[0024] The F-1 measure is a joint assessment of accuracy and completeness.

[0025] ROC-curve is a graphical characteristic of the quality of a binary classifier, reflecting the dependence of the proportion of true-positive classifications on the proportion of false-positive classifications when varying the threshold of the decision rule.

[0026] An error matrix is a way to categorize objects to be classified into four categories based on a combination of the real class and the response of the classifier.

[0027] Connectors are software components that collect data from information sources (Task management system / System for collaborative work on releases / Version control system / Project management system / Enterprise service management system / etc.) and bringing data to the required structure and format.

[0028] Storage - a system for storing large volumes of data collected and processed by connectors, as well as generated by other system components.

[0029] This technical solution can be implemented on a computer in the form of an automated information system (AIS) or a computer-readable medium containing instructions for performing the above method.

[0030] The technical solution can be implemented as a distributed computer system.

[0031] In this solution, the system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of computing operations (actions, instructions).

[0032] By a command processing device is meant an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0033] A command processor reads and executes machine instructions (programs) from one or more storage devices such as random access memory (RAM) and / or read only memory (ROM). The ROM can be, but is not limited to, hard disks (HDD), flash memory, solid state drives (SSD), optical data carriers (CD, DVD, BD, MD, etc.), etc.

[0034] A program is a sequence of instructions for execution by a computer control device or command processing device.

[0035] The training of the MO model is performed on pre-labeled data. In total, at the time of the creation of the machine learning model, 142 development teams were available, acting as of 06/01/2019. To assess the quality of the model, the dataset was divided into 2 parts: training (92 teams) and control samples (50 teams). Teams from the training sample were classified by interviewing cybersecurity experts working (or working) with these teams, with each team being classified more than 1 time by different experts.

[0036] In case of discrepancy between the opinions of experts, the opinion of a simple majority of experts was chosen. If the number of experts coincided with opposite opinions, the classification was carried out in accordance with the opinion of the expert who worked with the team at the time of the survey.

[0037] The weighted f-1 measure for the classifier is about 0.62, the accuracy is about 0.63.

[0038] FIG. 2 shows the ROC curve (error curve) for the team classifier based on a random forest.

[0039] FIG. 3 shows a matrix of errors (without normalization) for a command classifier based on a random forest.

[0040] Connectors receive the necessary information from the sources (by downloading files, queries to the database, to the API, analyzing web pages, reading event logs, etc.), save it to the storage.

[0041] Connectors extract significant parameters from the loaded data for further calculations, perform their preprocessing and form in the storage a table of values of the specified parameters (or attributes) according to commands with the following columns:

The number of Bug hits for the team;

The number of Feature calls for the command;

The number of hits with minor priority for the team;

The number of calls with the major priority for the team;

The number of calls with priority critical for the command;

The number of communications between team members and the cybersecurity expert;

Average time from creating a Release case to marking it as resolved;

Average time from creating a Feature case to marking it as resolved;

Average time from creating a bug of type Bug to marking it as resolved;

The number of releases released by the team.

[0042] The machine learning algorithm, based on the command parameter values contained in the table, marks the commands present in the table for high, medium and low compliance with cybersecurity requirements. The marking results are saved as a table in the repository

[0043] As shown in FIG. 1, the claimed method for predicting cybersecurity risks in the development of software products (100) consists of several stages performed by at least one processor.

[0044] At step (101), the input of the machine learning model is data containing information about at least the teams of software developers and the developed software products, each of these instructions. Also, the data can contain information about:

- tickets from agile teams in the development task management system: number, type, importance, status, time of creation, time of commissioning, time of marking as resolved, list of dependent tickets and types of dependencies, list of dependent tickets and types of dependencies, responsible agile -team, responsible team member

- the number of critical cybersecurity defects identified during acceptance tests of all previous releases of software products of agile teams

- the number of critical non-cybersecurity defects identified in acceptance tests of all previous releases of software products of agile teams

- data about agile teams and their members: teams, team members, their roles in the team, positions, training completed, exams passed and their results, data on previous employee transitions between agile teams and changes in positions, a list of releases, above which employees worked

- data on documentation for product releases of agile teams: its volume and hierarchy of pages, the number of attempts and the dates of its approval by cybersecurity experts and other employees

- communications between members of agile teams and cybersecurity experts over the periods of product existence: date, participants, duration of calls (from the telephony system), meetings (from corporate calendars), video conferencing (from the video conferencing management system); date and participants of email correspondence (from corporate mail system and corporate instant messaging system)

- data on the source code of product releases of agile teams: languages used, number of modules, amount of code, number of functions, methods, classes, variables, files

- data on the coding of product releases of agile teams: the number of build attempts, the number of errors and warnings that occurred when trying to build, the amount of code sent to the build, the number of functions, methods, classes, variables, files

- data on testing product releases of agile teams: the number of attempts to pass autotests, load and functional testing, the amount of code sent for testing, the number of functions, methods, classes, variables, files

- data on the passage of checks by the static and dynamic analysis system for vulnerabilities in product releases of agile teams: the number and types of vulnerabilities discovered, the results of their marking by release developers in the system as true-positive / false-positive, the amount of code sent for assembly, number of functions, methods, classes, variables, files

- data on vulnerabilities found in software products of agile teams after the release into industrial operation: release number, date of detection, the developer who created the vulnerable code, type of vulnerability, severity of the vulnerability, who discovered the vulnerability.

[0045] Next, at step (102), the received data is processed using a machine learning (ML) model, for example, but not limited to, using a machine learning algorithm based on a random forest classifier.

[0046] During processing, the machine learning algorithm performs:

- at step (103) dividing the received data into categorical and numerical variables;

- at step (104) processing of the obtained variables, in which vectorization of categorical variables and normalization of numerical variables is performed;

- at step (105), the concatenation of the processed variables and the construction of a vector based on them;

- at step (106), using the above-mentioned vector, the degree of occurrence of cybersecurity risks for each software product is assessed, and

- at stage (107), the classification of the development teams with the assignment of the degree of probability of the occurrence of the cybersecurity risk based on the performed assessment of the developed software products. The vector is compared with a numerical estimate (from 0 to 1) of the probability of the specified risk. Further, the numerical assessment is compared with a qualitative risk assessment, depending on the range in which the numerical assessment turned out to be: from 0 to 0.4 - low risk, 0.4-0.8 - medium risk, 0.8-1 - high risk.

[0047] The claimed technical solution provides a new opportunity for automated assessment of cybersecurity risk levels generated by the activities of agile product teams, and their classification into high, medium and low cybersecurity requirements in the development of their products, allows you to automatically generate a prioritized list of tasks for experts cybersecurity based on an algorithm-calculated risk level, resulting in labor savings for cybersecurity experts and agile team members while reducing the level of enterprise cybersecurity risks posed by agile product teams.

[0048] FIG. 4 shows an example of a general view of a computing system (300) that implements the claimed method or is part of a computer system, for example, a server, a personal computer, a part of a computing cluster that processes the necessary data to implement the claimed technical solution.

[0049] In the General case, the system (300) contains one or more processors (301) united by a common bus of information exchange, memory means such as RAM (302) and ROM (303), input / output interfaces (304), input devices / output (1105), and a device for networking (306).

[0050] The processor (301) (or multiple processors, multi-core processor, etc.) can be selected from a range of devices currently widely used, for example, such manufacturers as: Intel ™, AMD ™, Apple ™, Samsung Exynos ™, MediaTEK ™, Qualcomm Snapdragon ™, etc. Under the processor or one of the processors used in the system (300), it is also necessary to take into account a graphics processor, for example, NVIDIA GPU or Graphcore, the type of which is also suitable for full or partial execution of the method, and can also be used for training and applying machine learning models in various information systems.

[0051] RAM (302) is a random access memory and is intended for storing machine-readable instructions executed by the processor (301) for performing the necessary operations for logical data processing. RAM (302), as a rule, contains executable instructions of the operating system and corresponding software components (applications, software modules, etc.). In this case, the available memory of the graphics card or graphics processor can act as RAM (302).

[0052] ROM (303) is one or more persistent storage devices such as a hard disk drive (HDD), solid state data storage device (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R / RW, DVD-R / RW, BlueRay Disc, MD), etc.

[0053] Various types of I / O interfaces (304) are used to organize the operation of system components (300) and to organize the operation of external connected devices. The choice of appropriate interfaces depends on the specific design of the computing device, which can be, but are not limited to: PCI, AGP, PS / 2, IrDa, Fire Wire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro , mini, type C), TRS / Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0054] To ensure user interaction with the computing system (300), various means (305) I / O information are used, for example, a keyboard, display (monitor), touch display, touch-pad, joystick, mouse manipulator, light pen, stylus, touch panel, trackball, speakers, microphone, augmented reality, optical sensors, tablet, light indicators, projector, camera, biometric identification (retina scanner, fingerprint scanner, voice recognition module), etc.

[0055] The means of networking (306) provides data transmission via an internal or external computer network, for example, Intranet, Internet, LAN, etc. One or more means (306) may be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module, etc.

[0056] The presented materials of the application disclose the preferred examples of the implementation of the technical solution and should not be construed as limiting other, particular examples of its implementation, not going beyond the scope of the claimed legal protection, which are obvious to specialists in the relevant field of technology.

Claims (24)

1. A computer-implemented method for predicting cybersecurity risks in the development of software products, performed using at least one processor and containing the stages at which: - receive data containing information at least about the teams of software developers and developed software products, each of the mentioned teams; - carry out the processing of the obtained data using a machine learning model trained on the basis of expert data on cybersecurity on compliance with cybersecurity requirements by teams in the development of software products, and during this processing, the following is carried out: - division of the received data into categorical and numerical variables; - processing of the obtained variables, in which vectorization of categorical variables and normalization of numerical variables is performed; - concatenation of processed variables and construction of a vector based on them; - using the mentioned vector to evaluate the degree of cybersecurity risks occurrence for each software product, and - classification of development teams with the assignment of the degree of likelihood of cybersecurity risk on the basis of the assessment of the developed software products. 2. The method according to claim 1, characterized in that the processing of the obtained data is carried out using a machine learning model based on a random forest classifier. 3. The method according to claim 1, characterized in that the classification of teams is carried out with the assignment of a high, medium and low degree of likelihood of a cybersecurity risk. 4. The method according to claim 3, characterized in that information on development teams classified with a medium and high degree is automatically sent to the automated workstation of cybersecurity experts interacting with the development teams, with a high control mark. 5. The method according to claim 1, characterized in that the data on the classified commands contain at least: i. data on the tasks of the team members during the development of a software product in the task management system for the development of software products; ii. data on the structure of the team and the professional qualities of the team members; iii. data on communications between team members and cybersecurity experts in the development of software products for the entire lifetime of the team; iv. data on the source code of software products released by the team. 6. The method according to claim 1, characterized in that the data on the developed software products contain at least: i. data on the number of critical cybersecurity defects identified during acceptance tests of the team's software products released into commercial operation over the entire period of the team's existence; ii. data on the number of critical non-cybersecurity defects identified during acceptance tests of the team's products released into commercial operation over the entire period of the team's existence; iii. data on testing the finished software product of the team, before its release into industrial operation; iv. data on the passage of checks by the static and dynamic analysis system for vulnerabilities in the finished software products of the team, before their release into industrial operation; v. data on vulnerabilities in the team's software products discovered after the release into industrial operation. 7. A system for predicting cybersecurity risks in the development of software products, containing: at least one processor; - at least one memory connected to the processor, which contains machine-readable instructions, which, when executed by at least one processor, ensure the execution of the method according to any one of claims. 1-6.

Images