CN111917835A - A system, method and device for monitoring network data - Google Patents

Abstract

The invention discloses a system, method and device for monitoring network data, and relates to the technical field of computers. A specific implementation of the method includes: using a data acquisition and analysis tool to process the network mirror data to be monitored according to a self-defined transmission protocol, and reducing the resource overhead of system cache and mirror data transmission by combining data and adjusting the data transmission frequency, Improve the stability of the system and the efficiency of monitoring data, and improve the transmission efficiency of mirror data by combining multiple connections based on the transmission control protocol; by restoring the mirror data containing the custom transmission protocol to the data of the standard transmission control protocol Format, and establish a virtual connection based on the application layer protocol, so that the mirror data can be compatible with third-party network application layer analysis applications, and improve the flexibility of monitoring network mirror data.

Description

A system, method and device for monitoring network data

technical field

The present invention relates to the field of computer technology, and in particular, to a system, method and device for monitoring network data.

Background technique

In today's Internet era, data security and information security have become issues that enterprises must pay attention to. Usually, the solution for data traffic monitoring is to bypass the mirror image of the network traffic that needs to be analyzed on the backbone network of the enterprise to the network security monitoring equipment for processing. In this way, the normal business traffic of users is not affected, and the network security monitoring device can analyze the network data traffic of the bypass mirror in time, and feed back the risk of the data traffic to the network administrator; the solution for security products to analyze the bypass mirror traffic is usually It is mainly implemented in the way of traditional Linux-based kernel module (kernel-module), that is to realize data traffic identification and capture in the Linux system kernel, and then forward the captured data traffic to network applications for analysis. When there are risks such as data security, a data risk notification is generated to the operation and maintenance personnel.

In the process of realizing the present invention, the inventor found that there are at least the following problems in the prior art:

When the traffic of the mirror data to be monitored is large, the method of using the kernel-module to monitor the network data is to obtain all the bypass mirror data without processing it. Therefore, it is necessary to apply for a large amount of space in the kernel to cache the data that has not been processed in time. Data, the kernel memory usage rate may be too high, causing system stability problems due to high resource consumption, affecting the efficiency of monitoring mirrored data; and mirrored data is mainly transmitted based on the transmission control protocol, so in the kernel module in order to To judge the validity of the data, it is necessary to process multiple connections of the data, which increases the complexity of mirror data processing; the kernel module runs on the Linux system, resulting in poor portability, which in turn increases the complexity of monitoring mirror data.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present invention provide a system, method and device for monitoring network data, using data acquisition and analysis tools to process the monitored mirror data according to a self-defined transmission protocol, by combining data and adjusting the data transmission frequency. It reduces the resource overhead of system cache and mirror data transmission, improves the stability of the system and the efficiency of monitoring data, and improves the transmission efficiency of mirror data by combining multiple connections based on the transmission control protocol; The mirror data of the transmission protocol is the data format of the standard transmission control protocol, and the virtual connection based on the application layer protocol is established, so that the mirror data can be compatible with the third-party network application layer analysis application, and the flexibility of monitoring the mirror data is improved.

In order to achieve the above object, according to an aspect of the embodiments of the present invention, a system for monitoring network data is provided, which is characterized by comprising: a data acquisition unit and a data monitoring unit module, wherein:

The data acquisition unit is configured to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; based on the self-defined transmission protocol, the mirror data is Data is converted into intermediate data containing the custom transmission protocol format;

The data monitoring unit is configured to receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data is When it matches the set monitoring policy, send monitoring prompt information.

Optionally, the system for monitoring network data is characterized in that:

Based on the application layer protocol identifier contained in the target mirror data, a virtual connection based on the application layer protocol of the target mirror data is established.

Optionally, the system for monitoring network data is characterized in that:

According to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting a data filtering unit in the data acquisition unit, where the data filtering unit is configured to set data based on a network address of data transmission Get strategy.

Optionally, the system for monitoring network data is characterized in that:

Based on a custom transmission protocol, converting the mirror data into intermediate data in the format of the custom transmission protocol includes: combining at least two sets of the mirror data based on the custom transmission protocol, and generating a set containing the custom transmission protocol The intermediate data in the protocol format.

Optionally, the system for monitoring network data is characterized in that:

According to the size of a set of the intermediate data including the custom transmission protocol format, the sending frequency of sending a set of the intermediate data to the data monitoring unit is determined.

Optionally, the system for monitoring network data is characterized in that:

The multiple connections based on the transmission control protocol contained in the mirror data are combined and processed.

Optionally, the system for monitoring network data is characterized in that:

The data acquisition unit and the data processing unit run on the same physical device.

Optionally, the system for monitoring network data is characterized in that:

The data acquisition unit and the data processing unit use a local long connection for data transmission.

In order to achieve the above object, according to a second aspect of the embodiments of the present invention, a method for monitoring network data is provided, characterized in that, the method includes: acquiring, according to a data acquisition policy, a mirror image matching the data acquisition policy from a data exchange server data; the mirror data includes the data format of the transmission control protocol; based on a custom transmission protocol, convert the mirror data into intermediate data including the custom transmission protocol format; and send the intermediate data; receive the intermediate data data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format to form target mirror data; monitor the target mirror data, when the target mirror data matches the set monitoring strategy, send monitoring prompt information.

Optionally, the method for monitoring network data is characterized in that:

Based on the application layer protocol identifier contained in the target mirror data, a virtual connection based on the application layer protocol of the target mirror data is established.

Optionally, the method for monitoring network data is characterized in that:

According to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting the data acquisition policy based on the network address of data transmission.

Optionally, the method for monitoring network data is characterized in that:

Processing the mirror data based on a custom transmission protocol to generate intermediate data in the format of the custom transmission protocol, comprising: combining at least two sets of the mirror data based on the custom transmission protocol, and generating a set of data containing the custom transmission protocol format of the intermediate data.

Optionally, the method for monitoring network data is characterized in that:

According to the size of a set of the intermediate data including the custom transmission protocol format, the sending frequency of sending a set of the intermediate data is determined.

Optionally, the method for monitoring network data is characterized in that:

The multiple connections based on the transmission control protocol contained in the mirror data are combined and processed.

Optionally, the method for monitoring network data is characterized in that:

The intermediate data is sent or received using a local persistent connection.

In order to achieve the above object, according to a third aspect of the embodiments of the present invention, a device for monitoring network data is provided, which is characterized by comprising: a data acquisition module and a data monitoring module; wherein,

The data acquisition module is used to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; based on the self-defined transmission protocol, the mirror data is converting the data into intermediate data containing the custom transmission protocol format; and sending the intermediate data;

The data monitoring module is configured to receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data is When it matches the set monitoring policy, send monitoring prompt information.

Optionally, the device for monitoring network data is characterized in that:

Based on the application layer protocol identifier contained in the target mirror data, a virtual connection based on the application layer protocol of the target mirror data is established.

Optionally, the device for monitoring network data is characterized in that:

According to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting the data acquisition policy based on the network address of data transmission.

Optionally, the device for monitoring network data is characterized in that:

Processing the mirror data based on a custom transmission protocol to generate intermediate data in the format of the custom transmission protocol, comprising: combining at least two sets of the mirror data based on the custom transmission protocol, and generating a set of data containing the custom transmission protocol format of the intermediate data.

Optionally, the device for monitoring network data is characterized in that:

According to the size of a set of the intermediate data including the custom transmission protocol format, the sending frequency of sending a set of the intermediate data is determined.

Optionally, the device for monitoring network data is characterized in that:

The multiple connections based on the transmission control protocol contained in the mirror data are combined and processed.

Optionally, the device for monitoring network data is characterized in that:

The intermediate data is sent or received using a local persistent connection.

In order to achieve the above object, according to a fourth aspect of the embodiments of the present invention, an electronic device for monitoring network data is provided, which is characterized by comprising: one or more processors; and a storage device for storing one or more programs , when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method described in any one of the above methods for monitoring network data.

In order to achieve the above object, according to a fifth aspect of the embodiments of the present invention, a computer-readable medium is provided on which a computer program is stored, wherein the program is executed by the processor to realize the above-mentioned monitoring network data. The method of any one of the methods.

An embodiment of the above invention has the following advantages or beneficial effects: using data acquisition and analysis tools, processing network mirror data to be monitored according to a self-defined transmission protocol, and reducing system cache and mirror data by merging data and adjusting data transmission frequency The resource overhead of transmission improves the stability of the system, and improves the transmission efficiency of mirror data by combining the three-way handshake and four-way wave process based on the transmission control protocol; by restoring the mirror data containing the custom transmission protocol as standard transmission Control the data format of the protocol, and establish a virtual connection based on the application layer protocol, so that the mirror data can be compatible with third-party network application layer data analysis applications, and improve the portability and flexibility of monitoring network mirror data.

Further effects of the above non-conventional alternatives will be described below in conjunction with specific embodiments.

Description of drawings

The accompanying drawings are used for better understanding of the present invention and do not constitute an improper limitation of the present invention. in:

1 is a schematic structural diagram of a system for monitoring network data provided by an embodiment of the present invention;

2 is a schematic diagram of a system for monitoring network data provided by an embodiment of the present invention;

3 is a schematic flowchart of a method for monitoring network data provided by an embodiment of the present invention;

4 is a schematic diagram of an apparatus for monitoring network data provided by an embodiment of the present invention;

5 is an exemplary system architecture diagram to which an embodiment of the present invention may be applied;

FIG. 6 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

As shown in FIG. 1 , an embodiment of the present invention provides a schematic structural diagram of a system 100 for monitoring network data, including a data acquisition unit 101 and a data monitoring unit 102 .

The data acquisition unit 101 is configured to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; The mirror data is converted into intermediate data containing the custom transmission protocol format.

Specifically, the security product server usually obtains data traffic by bypassing mirror traffic to analyze and monitor data security. In an embodiment of the present invention, the data acquisition unit running in the security product server, according to the data acquisition unit, Obtain the mirror data from the exchange server, for example: obtain the mirror data through the network card of the switch; the data acquisition strategy is a preset strategy, which is used to determine the data acquired by the data acquisition unit and the source of the data, for example, it can be set The strategy is to acquire mirror data based on the network address of data transmission, and further, the data acquisition strategy is set by the data filtering unit; for example, the data acquisition strategy is to acquire data from a network segment 172.168.*.*, or to acquire data associated with a certain IP (Internet Protocol) address port number (for example: port number 80) data, etc.; the data acquisition strategy is set according to the specific business scenario of the enterprise, and the present invention does not limit the specific content of the mirror data acquisition strategy. That is, according to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting a data filtering unit in the data acquisition unit, where the data filtering unit is used for a network address based on data transmission Set the data acquisition policy.

Further, an embodiment of the present invention uses the combination of the driver provided by the PF_RING tool, the Libpcap software package, and the BRO tool (which is a packet flow analysis tool) to achieve the acquisition of mirror data. It can be understood that using This solution, instead of the commonly used Kernel-Module data acquisition method, overcomes the defects of the existing Kernel-Module solution.

Specifically, PF_RING is a packet capture technology toolkit. Using the underlying socket buffer mechanism provided by the driver included in PF_RING, it can efficiently receive the network card data of the switch, and can cache a large amount of data by adapting the size of the buffer. , so that the process of copying mirror data from the socket buffer to the user space buffer in the system kernel module (Kernel-Module) scheme is not required, which improves the receiving efficiency of mirror data. Further, the image data is obtained by using the interface of the Libpcap software package. The Libpcap software package is recompiled based on the driver interface contained in PF_RING, and then provides various standard image data acquisition interfaces, thereby solving the need for BRO tools to be adapted. This improves the flexibility and ease of use of the BRO tool to ensure the stability of the data acquisition application; further, the BRO tool is used to filter traffic according to the data acquisition strategy; for example, the data acquisition strategy is: tcp port 80, that is The instruction is to acquire mirror data with a TCP (Transmission Control Protocol) protocol port of 80; that is, according to the data acquisition policy, acquire mirror data matching the data acquisition policy from the data exchange server.

Further, the mirror data is a data format based on a transmission control protocol; the mirror data is processed based on a self-defined transmission protocol, and the intermediate data based on the self-defined transmission protocol format is generated; It is TCP (Transmission Control Protocol, TCP, Transmission Control Protocol). Generally, the data traffic of the application layer monitored and detected by the enterprise network usually uses the TCP protocol as the transmission protocol. The protocol types of the application layer include: HTTP protocol (Hypertext Transfer Protocol, hypertext). Transmission Protocol), FTP Protocol (File Transfer Protocol, File Transfer Protocol), SNMP Protocol (Simple Network Management Protocol, Simple Network Management Protocol), IMAP Protocol (Internet Message Access Protocol, Internet Information Access Protocol), POP3 Protocol (Post Office Protocol) 3, Post Office Protocol version 3) etc.; by obtaining the mirror data of the data format based on the TCP protocol (that is, the transmission control protocol), and sending the data monitoring unit to further monitor and analyze the mirror data; The data acquisition unit converts the mirror data into intermediate data including the self-defined transmission protocol format based on the self-defined transmission protocol; specifically, the self-defined transmission protocol, for example, an embodiment of the present invention is a self-defined TCP-Replay The protocol is used to merge and recombine the obtained multiple sets of mirror data to generate intermediate data including the custom transmission protocol format; and transmit the data to the data through the serial packet method of the local long connection (for example: unix-socket). The processing module performs further data processing, which reduces the consumption of network socket resources; improves the transmission efficiency of mirrored data; that is, the data acquisition module and the data processing module use a local long connection for data transmission. That is, processing the mirror data based on a custom transmission protocol, and generating intermediate data based on the custom transmission protocol format, includes: combining at least two sets of the mirror data based on the custom transmission protocol, and generating a set of data based on the custom transmission protocol. The intermediate data in a transport protocol format. It can be understood that, by combining multiple sets of mirror data, the overhead of network resources for transmitting mirror data is reduced. Further, the frequency of sending the intermediate data to the data processing module is determined according to the size of the intermediate data and the availability rate of network resources, wherein the size of the network data is the size of the network data (for example, 1GB, 500MB, etc.); for example, when When the size of the intermediate data is large, the frequency of sending the intermediate data can be reduced, so as to reduce the congestion of network resources; that is, the mirror data is processed based on the custom transmission protocol, and the intermediate data based on the format of the custom transmission protocol is generated, The method also includes: determining a sending frequency for sending the intermediate data to the data processing module according to a set of sizes of the intermediate data based on the custom transmission protocol format.

Further, in the BRO tool, it is possible to identify and combine the data packets based on the transmission control protocol about the three-way handshake and the four-way wave (ie, multiple connections), which reduces the processing complexity of the application receiving the mirror data and improves the monitoring. Efficiency of mirrored data; that is, the combined processing of multiple Transmission Control Protocol-based connections contained in the mirrored data.

The data monitoring unit 102 is configured to receive the intermediate data, convert the custom transmission protocol format included in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror When the data matches the set monitoring strategy, send monitoring prompt information.

Specifically, the data monitoring unit 102 receives the intermediate data, converts the custom transmission protocol format included in the intermediate data into a transmission control protocol format, and forms target mirror data; in an embodiment of the present invention, the data monitoring unit Runs on ATS (Apache Traffic Server, ATS), receives intermediate data from BRO through local long connection (unix-socket), first converts intermediate data into target mirror data based on TCP protocol format according to TCP-Replay protocol, the process of data conversion Including depacketizing and restoring data; further, based on the application layer protocol identification contained in the target mirror data, establish a virtual connection based on the application layer protocol of the target mirror data; specifically, the data monitoring unit includes an application layer protocol. A network analysis application for monitoring and analyzing target mirror data. Therefore, based on the application layer protocol identification (for example, HTTP protocol identification) contained in the target mirror data, a virtual connection based on the application layer protocol (for example, the HTTP protocol) of the target mirror data is established; thus, it can be compatible with the ATS. The third-party network analysis application on the network improves the compatibility and flexibility of the monitoring data; wherein, when generating the virtual connection of the application layer protocol, the event simulation function included in the ATS can be used to indicate that the target mirror data is the mirror data attribute . The method used can add code indicating the attributes of the mirrored data by adapting the asynchronous event handling system existing in the ATS. The network analysis application analyzes the target mirror data, and according to the set monitoring strategy, for example, including the analysis strategy, security risk trigger conditions, etc., when the target mirror data matches the set monitoring strategy, generates and sends the monitoring strategy. Prompt information, monitoring prompt information such as log information, alarm text, etc. And send monitoring prompt information to relevant operation and maintenance personnel through email, event notification, short message, etc.; that is, monitor the target mirror data, and send monitoring prompt information when the target mirror data matches the set monitoring strategy. The present invention does not limit the specific content of the set monitoring strategy, the specific content of the monitoring prompt information, and the specific manner of sending the monitoring prompt information.

Preferably, the data acquisition unit and the data processing unit run on the same physical device. For example, the data acquisition unit and the data processing unit run on the same physical device for monitoring network data (network data is mirror data), such as a personal computer, a server, and the like. It can be understood that, by running the data acquisition unit and the data processing unit on the same physical device, and using the local long connection to transmit the mirror data, the transmission efficiency and transmission stability of the mirror data are improved, thereby improving the monitoring data. efficiency. Further, FIG. 2 shows a schematic structural diagram of an embodiment based on the BRO and ATS described above.

As shown in FIG. 3 , an embodiment of the present invention provides a schematic flowchart of a method for monitoring network data. The method includes the following steps:

Step S301: According to the data acquisition policy, acquire mirror data matching the data acquisition policy from the data exchange server; the mirror data includes the data format of the transmission control protocol; based on the custom transmission protocol, convert the mirror data to include intermediate data in the custom transmission protocol format; and sending the intermediate data.

Specifically, the description about acquiring and converting the mirror data is consistent with the description of the data acquiring unit 101 , and details are not repeated here.

Step S302: Receive the intermediate data, convert the custom transmission protocol format included in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data matches the set When the monitoring policy is specified, the monitoring prompt information is sent.

Specifically, the description of monitoring mirror data, including receiving and processing mirror data, is consistent with the description of the data monitoring unit 102 , and details are not repeated here.

As shown in FIG. 4, an embodiment of the present invention provides a schematic structural diagram of an apparatus 400 for monitoring network data, including: a data acquisition module 401 and a data monitoring module 402, wherein,

The data acquisition module 401 is configured to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; Converting the mirror data into intermediate data including the custom transmission protocol format; and sending the intermediate data;

The data monitoring module 402 is configured to receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; based on the application layer protocol identifier contained in the target mirror data , establishing a virtual connection of the target mirror data based on an application layer protocol; and performing data analysis and monitoring based on the target mirror data.

Optionally, the data acquisition module 401 is further configured to establish a virtual connection of the target mirror data based on the application layer protocol based on the application layer protocol identifier included in the target mirror data.

Optionally, the data acquisition module 401 is further configured to acquire mirror data matching the data acquisition policy from the data exchange server according to the data acquisition policy, including: setting the data acquisition policy based on the network address of data transmission.

Optionally, the data acquisition module 401 is further configured to process the mirror data based on a custom transmission protocol, and generate intermediate data including the format of the custom transmission protocol, including: combining at least two groups of data based on the custom transmission protocol. The mirror data is generated, and a set of intermediate data including the custom transmission protocol format is generated.

Optionally, the data acquisition module 401 is further configured to determine the sending frequency for sending a set of the intermediate data according to the size of the set of intermediate data including the custom transmission protocol format.

Optionally, the data acquisition module 401 is further configured to combine and process multiple connections based on the transmission control protocol contained in the mirror data.

Optionally, the data acquisition module 401 is further configured to send or receive the intermediate data by using a local persistent connection.

An embodiment of the present invention further provides an electronic device for monitoring network data, including: one or more processors; and a storage device for storing one or more programs, when the one or more programs are used by the one or more programs Multiple processors execute, so that the one or more processors implement the method provided by any of the above embodiments.

An embodiment of the present invention further provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, implements the method provided by any of the foregoing embodiments.

FIG. 5 shows an exemplary system architecture 500 of a method for monitoring network data or an apparatus for monitoring network data to which embodiments of the present invention may be applied.

As shown in FIG. 5 , the system architecture 500 may include terminal devices 501 , 502 , and 503 , a network 504 and a server 505 . The network 504 is a medium used to provide a communication link between the terminal devices 501 , 502 , 503 and the server 505 . Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the terminal devices 501, 502, 503 to interact with the server 505 through the network 504 to receive or send messages and the like. Various client applications may be installed on the terminal devices 501 , 502 and 503 , such as web browser applications, search applications, instant messaging tools, and email clients.

The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The server 505 may be a server that provides various services, for example, a background management server that provides monitoring for the network data generated by the terminal devices 501 , 502 , and 503 for users. The background management server can analyze and monitor the received mirror data, and feed back the monitoring prompt information to the terminal device.

It should be noted that the method for monitoring network data provided by the embodiment of the present invention is generally executed by the server 505 , and a corresponding apparatus for monitoring network data is generally set in the server 505 .

It should be understood that the numbers of terminal devices, networks and servers in FIG. 5 are only illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

Referring to FIG. 6 below, it shows a schematic structural diagram of a computer system 600 suitable for implementing a terminal device according to an embodiment of the present invention. The terminal device shown in FIG. 6 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present invention.

As shown in FIG. 6, a computer system 600 includes a central processing unit (CPU) 601, which can be loaded into a random access memory (RAM) 603 according to a program stored in a read only memory (ROM) 602 or a program from a storage section 608 Instead, various appropriate actions and processes are performed. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

The following components are connected to the I/O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc. ; and a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I/O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

In particular, the processes described above with reference to the flowcharts may be implemented as computer software programs in accordance with the disclosed embodiments of the present invention. For example, embodiments disclosed herein include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 609 and/or installed from the removable medium 611 . When the computer program is executed by the central processing unit (CPU) 601, the above-described functions defined in the system of the present invention are performed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The modules and/or units involved in the embodiments of the present invention may be implemented in a software manner, and may also be implemented in a hardware manner. The described modules and/or units can also be provided in the processor, for example, it can be described as: a processor includes a data acquisition module and a data monitoring module. Among them, the names of these modules do not constitute a limitation of the module itself under certain circumstances. For example, the data monitoring module can also be described as "receiving and processing mirror data, and monitoring the mirror data according to the set monitoring strategy. module".

As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or may exist alone without being assembled into the device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by a device, the device includes: using a data acquisition and analysis tool to process the image to be monitored according to a self-defined transmission protocol Data, by combining data and adjusting the data transmission frequency, the resource overhead of system cache and mirror data transmission is reduced, the stability of the system and the efficiency of monitoring network data are improved, and multiple connections based on the transmission control protocol are combined to improve the performance. Mirror data transmission efficiency; by restoring the mirror data containing the custom transmission protocol to the data format of the standard transmission control protocol, and establishing a virtual connection based on the application layer protocol, the mirror data can be compatible with third-party network application layer analysis applications, improving the performance. Flexibility to monitor mirrored data.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (18)

1. a system for monitoring network data, comprising: a data acquisition unit and a data monitoring unit, wherein: The data acquisition unit is configured to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; based on the self-defined transmission protocol, the mirror data is Data is converted into intermediate data containing the custom transmission protocol format; The data monitoring unit is configured to receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data is When it matches the set monitoring policy, send monitoring prompt information. 2. The system of claim 1, wherein: Based on the application layer protocol identifier contained in the target mirror data, a virtual connection based on the application layer protocol of the target mirror data is established. 3. The system of claim 1, wherein: According to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting a data filtering unit in the data acquisition unit, where the data filtering unit is configured to set data based on a network address of data transmission Get strategy. 4. The system of claim 1, wherein: Based on the custom transmission protocol, the image data is converted into intermediate data including the format of the custom transmission protocol, including: Combine at least two sets of the mirror data based on a custom transmission protocol to generate a set of the intermediate data in the format of the custom transmission protocol. 5. The system of claim 4, wherein: According to the size of a set of the intermediate data including the custom transmission protocol format, the sending frequency of sending a set of the intermediate data to the data monitoring unit is determined. 6. The system of claim 1, wherein: The multiple connections based on the transmission control protocol contained in the mirror data are combined and processed. 7. The system of claim 1, wherein: The data acquisition unit and the data processing unit run on the same physical device. 8. The system of claim 7, wherein: The data acquisition unit and the data processing unit use a local long connection for data transmission. 9. A method for monitoring network data, comprising: According to the data acquisition strategy, the mirror data matching the data acquisition strategy is obtained from the data exchange server; the mirror data includes the data format of the transmission control protocol; based on the custom transmission protocol, the mirror data is converted to include the self Define the intermediate data in the format of the transmission protocol; and send the intermediate data; Receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data matches the set monitoring strategy , send monitoring prompt information. 10. The method of claim 9, wherein: Based on the application layer protocol identifier contained in the target mirror data, a virtual connection based on the application layer protocol of the target mirror data is established. 11. The method of claim 9, wherein: According to the data acquisition policy, acquiring mirror data matching the data acquisition policy from the data exchange server includes: setting the data acquisition policy based on the network address of data transmission. 12. The method of claim 9, wherein: Process the mirror data based on the custom transmission protocol, and generate intermediate data including the format of the custom transmission protocol, including: Combine at least two sets of the mirror data based on a custom transmission protocol to generate a set of the intermediate data in the format of the custom transmission protocol. 13. The method of claim 12, wherein According to the size of a set of the intermediate data including the custom transmission protocol format, the sending frequency of sending a set of the intermediate data is determined. 14. The method of claim 9, wherein: The multiple connections based on the transmission control protocol contained in the mirror data are combined and processed. 15. The method of claim 9, wherein: The intermediate data is sent or received using a local persistent connection. 16. A device for monitoring network data, comprising: a data acquisition module and a data monitoring module; wherein, The data acquisition module is used to acquire mirror data matching the data acquisition strategy from the data exchange server according to the data acquisition strategy; the mirror data includes the data format of the transmission control protocol; based on the self-defined transmission protocol, the mirror data is converting the data into intermediate data containing the custom transmission protocol format; and sending the intermediate data; The data monitoring module is configured to receive the intermediate data, convert the custom transmission protocol format contained in the intermediate data into a transmission control protocol format, and form target mirror data; monitor the target mirror data, when the target mirror data is When it matches the set monitoring policy, send monitoring prompt information. 17. An electronic device, characterized in that, comprising: one or more processors; storage means for storing one or more programs, The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 9-15. 18. A computer-readable medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method according to any one of claims 9-15 is implemented.

Images