CN115580409A - Distributed data storage access method and system based on object storage - Google Patents

Abstract

The invention relates to a distributed data storage access method and a system based on object storage, wherein the system comprises a server and a client, and the server authenticates user login information based on a site information table; receiving a storage position request, and generating a first feedback instruction, wherein the first feedback instruction is provided with a signature and a validity period; when the successful storage information is received, generating and storing a data record; the client establishes connection with the server through user login information and a server address, sends a storage position request when data storage is needed, and stores nearby data after receiving a first feedback instruction; and when data access is needed, sending a query instruction, if a data record matched with the query instruction is received, sending an access instruction, and after a second feedback instruction is received, performing nearby data access. Compared with the prior art, the method has the advantages of solving the problem of cross-region storage performance, meeting the requirement of unified management and the like.

Description

A distributed data storage access method and system based on object storage

technical field

The present invention relates to a distributed storage system, in particular to a distributed data storage access method and system based on object storage.

Background technique

With the acceleration of digital transformation, the unified management of data by institutions is becoming more and more urgent. Traditional storage systems use centralized storage servers to store all data. Storage servers become the bottleneck of system performance and the focus of reliability and security, which cannot meet the needs of large-scale storage applications. The distributed storage system adopts a scalable system structure, uses multiple storage servers to share the storage load, uses location servers to locate and store information, and disperses and stores data on multiple independent devices in the LAN, which not only improves the reliability of the system , usability and access efficiency, and easy to expand. The distributed storage system can well solve the problem of massive file storage, with good scalability and reliability, it can meet the storage requirements of massive data.

However, for a large organization, each branch is scattered in different regions, and each branch may need to store a large amount of data. If the data is stored centrally at the headquarters, the cost of network bandwidth is high and the storage efficiency is low; if each branch is stored independently, And lost the unified management of data.

Contents of the invention

The purpose of the present invention is to provide an object storage-based distributed data storage access method and system that can solve the problem of cross-regional storage performance and meet the requirements of unified management in order to overcome the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved through the following technical solutions:

In the first aspect, the present invention provides a method for accessing distributed data storage based on object storage, which runs on a server and includes the steps of nearby storage, nearby access and unified management, wherein,

Described nearest storage step comprises:

101) receiving user login information, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 102), if not, feedback login failure information;

102) Receive a storage location request, determine the corresponding object storage according to the site information table, and generate a first feedback instruction, the first feedback instruction has a signature and a validity period;

103) Judging whether the storage success information is received, if so, then generate and save the data record, the data record includes the data ID, the user and the storage location, if not, then end;

The nearby visit steps include:

201) receiving user login information, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 202), if not, feedback login failure information;

202) Receive a query instruction, find out whether there is a data record matching the query instruction, if so, then feed back the data record, if not, then end;

203) Receive an access instruction, the access instruction includes data ID and user login information, query and obtain the corresponding storage address based on the access instruction, and generate a second feedback instruction, the second feedback instruction has a signature and a validity period;

The unified management steps include:

Receive a record query instruction, the record query instruction includes administrator login information, determine whether the administrator login information is authenticated, and if so, feed back all data records.

Further, the site information table includes a main site and multiple sub-sites, and each site is configured with object storage information and user login information of a corresponding site.

Further, the user login information includes username and password.

In the second aspect, the present invention provides a method for accessing distributed data storage based on object storage, which runs on a client and includes a nearby storage step and a nearby access step, wherein,

Described nearest storage step comprises:

101) After establishing a connection with the server through the user login information and the server address, send a storage location request;

102) Real-time detection of whether the first feedback instruction is received, if yes, then perform step 103), if not, then return to step 101), the first feedback instruction is generated according to the corresponding object storage determined by the site information table, the The site information table is stored on the server, and the first feedback command has a signature and a validity period;

103) performing data storage in the object storage in step 102) according to the first feedback instruction;

The nearby visit steps include:

201) After establishing a connection with the server through the user login information and the server address, sending a query command;

202) Real-time detection whether the data records matched with the query instruction are received, if so, then perform step 203), if not, then end;

203) Send an access instruction, the access instruction includes data ID and user login information, detects in real time whether the second feedback instruction is received, if so, then execute step 204), if not, then return to step 203), the second feedback instruction Generated based on the corresponding storage address obtained from the query, the second feedback instruction has a signature and a validity period;

204) Perform data access according to the second feedback instruction.

Further, the site information table includes a main site and multiple sub-sites, and each site is configured with object storage information and user login information of a corresponding site.

Further, the user login information includes username and password.

In a third aspect, the present invention provides a server, including one or more processors, memory, and one or more programs stored in the memory, and the one or more programs are used to execute the above-mentioned object-based Stored instructions for distributed data storage access methods.

In a fourth aspect, the present invention provides a client, including one or more processors, memories, and one or more programs stored in the memories, the one or more programs including being used to execute the above-mentioned object-based Stored instructions for distributed data storage access methods.

In the fifth aspect, the present invention provides a distributed data storage access system based on object storage, including a server and a client, wherein,

The server maintains a site information table, and authenticates user login information based on the site information table; receives a storage location request, determines the corresponding object storage according to the site information table, and generates a first feedback instruction, the first feedback instruction With a signature and a validity period; when the successful storage information is received, a data record is generated and saved, and the data record includes the data ID, the user and the storage location; the record query command is received, and the record query command includes the login information of the administrator, and the judgment Whether the administrator login information is authenticated, if so, all data records are fed back;

The client establishes a connection with the server through the user login information and the address of the server, sends a storage location request when data storage is required, and stores the data according to the first feedback instruction after receiving the first feedback instruction, wherein , the first feedback instruction is generated according to the corresponding object storage determined by the site information table, and the first feedback instruction has a signature and a validity period; when data access is required, a query instruction is sent, and if a query matching the query instruction is received data records, send an access instruction, the access instruction includes data ID and user login information, after receiving the second feedback instruction, perform data access according to the second feedback instruction, the second feedback instruction is based on the query obtained A corresponding storage address is generated, and the second feedback command has a signature and a validity period.

Further, the client is replaced by an API module.

Compared with the existing technology, the present invention not only realizes unified management of data, but also improves storage efficiency through nearby storage and access, and solves the difficulties of unified data management, high cost of cross-regional bandwidth, and low efficiency faced by cross-regional and multi-branch organizations. And other issues. Specifically, the present invention has the following beneficial effects

1) Through the corresponding relationship between sites, users, and object storage, the user's data is stored in the object storage of the user's site nearby to improve storage efficiency.

2) The user's data is stored in the object storage of the user's site, and access to these data does not need to cross regions, and the access efficiency is high.

3) The main site records all user data, which facilitates the unified management of data.

Description of drawings

Fig. 1 is a schematic diagram of the principle of nearby storage or nearby access in the present invention;

Fig. 2 is a schematic diagram of the principle of unified management in the present invention.

detailed description

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is carried out on the premise of the technical solution of the present invention, and detailed implementation and specific operation process are given, but the protection scope of the present invention is not limited to the following embodiments.

On the one hand, the present invention provides a method for accessing distributed data storage based on object storage, which runs on a server and includes the steps of nearby storage, nearby access and unified management. A site information table is pre-deployed in the server, and the site information table includes a main site and multiple sub-sites, and each site is configured with object storage information and user login information of the corresponding site. The main site can correspond to the headquarters or employees of the headquarters, and the sub-sites can correspond to branches or employees of branches.

The above steps are specifically described as follows.

Nearby storage steps include:

101) receiving user login information, including user name and password, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 102), if not, feedback login failure information;

102) Receive a storage location request, determine the corresponding object storage according to the site information table, and generate a first feedback instruction, the first feedback instruction has a signature and a validity period;

103) Judging whether the storage success information is received, if yes, then generate and save the data record, the data record includes the data ID, the user and the storage location, if not, then end.

Nearby visit steps include:

201) receiving user login information, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 202), if not, feedback login failure information;

202) Receive the query instruction, find out whether there is a data record matching the query instruction, if so, feed back the data record, if not, then end;

203) Receive an access instruction, the access instruction includes the data ID and user login information, query and obtain the corresponding storage address based on the access instruction, and generate a second feedback instruction, the second feedback instruction has a signature and a validity period.

The unified management step includes: receiving a record query instruction, the record query instruction includes administrator login information, judging whether the administrator login information is authenticated, and if so, feeding back all data records.

If the above functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. . Specifically, the above-mentioned computer-readable storage medium may be stored in a server.

In the second aspect, the present invention also provides an object-storage-based distributed data storage access method running on a client, including a nearby storage step and a nearby access step, and the client communicates and interacts with the server.

The above steps are specifically described as follows.

Nearby storage steps include:

101) After establishing a connection with the server through the user login information and the server address, send a storage location request;

102) detect in real time whether to receive the first feedback instruction, if so, then perform step 103), if not, then return to step 101), the first feedback instruction is generated according to the corresponding object storage determined by the site information table, and the site information table stores On the server side, the first feedback command has a signature and an expiration date;

103) Perform data storage in the object storage in step 102) according to the first feedback instruction.

Nearby visit steps include:

201) After establishing a connection with the server through the user login information and the server address, sending a query command;

202) detect in real time whether to receive the data record matched with the query instruction, if so, then perform step 203), if not, then end;

203) Send an access instruction, the access instruction includes data ID and user login information, detects in real time whether the second feedback instruction is received, if so, then execute step 204), if not, then return to step 203), the second feedback instruction is based on the query The obtained corresponding storage address is generated, and the second feedback command has a signature and a validity period;

204) Perform data access according to the second feedback instruction.

If the above functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. . Specifically, the above-mentioned computer-readable storage medium may be stored in a client.

In the third aspect, the present invention also provides a distributed data storage and access system based on object storage, which can be referred to as an OSS gateway for short, and can manage object storage scattered in various regions in a unified manner, satisfying the need for nearby data storage and unified management in each region requirements. As shown in Figure 1, the OSS gateway includes a server and a client, and the configuration of the server and client is as follows:

Server deployment and configuration methods:

a. Deploy the OSS gateway server program at the headquarters.

b. Configure the information of the headquarters and each branch on the server side, referred to as the site, the headquarters is the main site, and each branch is the sub-site.

c. Configure the object storage, tenant information, user names and passwords of branch users and other information of each branch to the site corresponding to the branch.

The server maintains a site information table, and authenticates user login information based on the site information table; receives the storage location request, determines the corresponding object storage according to the site information table, and generates a first feedback command with a signature and Validity period; when receiving the successful storage information, generate and save the data record, the data record includes the data ID, the user and the storage location; receive the record query command, the record query command includes the administrator login information, and judge whether the administrator login information is authenticated Passed, if yes, all data records are fed back.

Client configuration method:

a. Deploy OSS gateways where data needs to be stored and accessed.

b. Configure the OSS gateway server address on the client.

The client establishes a connection with the server through the user login information and the server address. When data storage is required, the client sends a storage location request. After receiving the first feedback instruction, the client performs data storage according to the first feedback instruction, wherein the first feedback The instruction is generated according to the corresponding object storage determined by the site information table, and the first feedback instruction has a signature and a validity period; when data access is required, a query instruction is sent, and if a data record matching the query instruction is received, an access instruction is sent , the access instruction includes data ID and user login information, after receiving the second feedback instruction, data access is performed according to the second feedback instruction, the second feedback instruction is generated based on the corresponding storage address obtained from the query, and the second feedback instruction carries With signature and expiry date.

Through the above interaction between the server and the client, nearby storage, nearby access and unified management can be realized conveniently and efficiently.

As shown in Figure 1 and Figure 2, the specific process of realizing nearby storage, nearby access and unified management is described as follows.

1. Nearby storage:

S1. The user uses the user name and password to log in to the OSS gateway server through the OSS gateway client or API.

S2. When the user needs to store data, the user requests a suitable storage location from the OSS gateway server.

S3. The OSS gateway server receives the request, and according to the relationship between the user and the site in the configuration, selects the object storage of the site where the user is located, generates a URL with signature and validity period, and returns it to the OSS gateway client or API caller.

S4. The OSS gateway client or API caller directly uses the URL returned by the OSS gateway server to store data.

S5. After the data is successfully stored, the OSS gateway server is notified, and the server records the data ID, the user to which it belongs, the storage location and other information into the database.

2. Nearby visit:

V1. The user uses the user name and password to log in to the OSS gateway server through the OSS gateway client or API.

V2. Query the data information stored by the user to the OSS gateway server through the OSS gateway client or API, and the OSS gateway server queries the user's data records stored in the database and returns them to the client or API caller.

V3. When the user needs to access data, the user requests the storage location of the data ID from the OSS gateway server through the data ID in the data record returned by the server.

V4. After receiving the request, the server queries the corresponding storage address from the database according to the data ID and user information, generates a URL with signature and validity period, and returns it to the OSS gateway client or API caller.

V5. The OSS gateway client or API caller directly uses the URL returned by the OSS gateway server to access data.

3. Unified management:

T1. According to S5, all data stored in this system are recorded in the database of the OSS gateway server of the main site.

T2. The system administrator can query all data according to the information in the database to realize the unified management of data.

In other embodiments, the client can be replaced by an API module, and users can use the OSS gateway client or API to store and access data.

Example 1

In this embodiment, the above method is applied to the following scenario: a large enterprise X has its headquarters in city A, branch 1 in city B, and branch 2 in city C. Enterprise X needs to undergo digital transformation. The headquarters and branch offices have a large amount of data that needs to be managed in a unified manner. However, based on cost considerations, the public network bandwidth of the headquarters and branch offices can only meet normal office needs. Transferring data to the headquarters for storage and downloading data from the headquarters will occupy a large amount of public network bandwidth, seriously affecting normal office work.

In this scenario, the headquarters and branch companies plan to save data in the computer rooms of their respective office areas. Through the solution of the present invention, efficient distributed storage is realized and unified management requirements are met. The specific process is as follows:

A. The headquarters, branch 1, and branch 2 deploy object storage respectively.

B. Deploy the OSS gateway server at the headquarters.

C. Configure 3 sites including headquarters, branch 1, and branch 2 in the OSS gateway server.

D. Configure information such as the object storage information of the headquarters, the user names and passwords of employees in the headquarters to the headquarters site.

E. Configure information such as the object storage information of branch 1, the user name and password of branch 1 employees to the branch 1 site.

F. Configure information such as the object storage information of branch 2, the user name and password of branch 2 employees to the branch 2 site.

G. The employee deploys the OSS gateway client on the office equipment and configures the address of the OSS gateway server. The employee uses the user name and password to store the data in the object storage of the corresponding site of the employee through the OSS gateway client, and the OSS gateway server records this data. information.

H. The enterprise leader logs in to the OSS gateway server as an administrator to view the data saved by all employees.

Example 2

In this embodiment, the above method is applied to the following scenario: large organization Z, headquartered in city A, branch 1 in city B, and branch 2 in city C. Organization Z needs to undergo digital transformation. The headquarters and branch offices have a set of office systems. The office system will generate a large amount of data that needs to be managed in a unified manner. However, based on cost considerations, the public network bandwidth of the headquarters and branch offices can only meet normal office needs. If the office system of the branch office transmits data to the headquarters for storage and downloads data from the headquarters through the network, it will occupy a large amount of public network bandwidth and seriously affect normal office work.

In this scenario, the headquarters and branches plan to save data in the computer rooms of their respective office areas. Through the solution of the present invention, efficient distributed storage can be realized and the requirements of unified management can be met:

I. The headquarters, branch 1, and branch 2 deploy object storage respectively.

J. Deploy the OSS gateway server at the headquarters.

K. Configure 3 sites including headquarters, branch 1, and branch 2 in the OSS gateway server.

L. Configure information such as the object storage information of the headquarters, the user name and password of the office system of the headquarters to the headquarters site.

M. Configure information such as the object storage information of branch 1, the user name and password of the office system of branch 1 to the site of branch 1.

N. Configure information such as the object storage information of branch 2, the user name and password of the office system of branch 2 to the site of branch 2.

O. Perform secondary development on the office system, configure the address of the OSS gateway server, use the user name and password of the office system, and log in to the OSS gateway server through the OSS gateway API. Store the data in the object storage of the corresponding site of the office system, and the OSS gateway server records the data information.

P. Organization leaders can log in to the OSS gateway server as an administrator to view the data saved in all office systems.

The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection defined by the claims.

Claims (10)

1. A method for accessing distributed data storage based on object storage, running on a server, characterized in that it includes a nearby storage step, a nearby access step and a unified management step, wherein, Described nearest storage step comprises: 101) receiving user login information, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 102), if not, feedback login failure information; 102) Receive a storage location request, determine the corresponding object storage according to the site information table, and generate a first feedback instruction, the first feedback instruction has a signature and a validity period; 103) Judging whether the storage success information is received, if so, then generate and save the data record, the data record includes the data ID, the user and the storage location, if not, then end; The nearby visit steps include: 201) receiving user login information, based on the pre-stored site information table, judging whether the user login information is passed through the authentication, if so, then perform step 202), if not, feedback login failure information; 202) Receive a query instruction, find out whether there is a data record matching the query instruction, if so, then feed back the data record, if not, then end; 203) Receive an access instruction, the access instruction includes data ID and user login information, query and obtain the corresponding storage address based on the access instruction, and generate a second feedback instruction, the second feedback instruction has a signature and a validity period; The unified management steps include: Receive a record query instruction, the record query instruction includes administrator login information, determine whether the administrator login information is authenticated, and if so, feed back all data records. 2. The distributed data storage access method based on object storage according to claim 1, wherein the site information table includes a main site and a plurality of sub-sites, and each site is configured with object storage information and User login information. 3. The method for accessing distributed data storage based on object storage according to claim 1, wherein the user login information includes a user name and a password. 4. A distributed data storage access method based on object storage, running on a client, characterized in that it includes a nearby storage step and a nearby access step, wherein, Described nearest storage step comprises: 101) After establishing a connection with the server through the user login information and the server address, send a storage location request; 102) Real-time detection of whether the first feedback instruction is received, if yes, then perform step 103), if not, then return to step 101), the first feedback instruction is generated according to the corresponding object storage determined by the site information table, the The site information table is stored on the server, and the first feedback command has a signature and a validity period; 103) performing data storage in the object storage in step 102) according to the first feedback instruction; The nearby visit steps include: 201) After establishing a connection with the server through the user login information and the server address, sending a query instruction; 202) Real-time detection whether the data records matched with the query instruction are received, if so, then perform step 203), if not, then end; 203) Send an access instruction, the access instruction includes data ID and user login information, detects in real time whether the second feedback instruction is received, if so, then execute step 204), if not, then return to step 203), the second feedback instruction Generated based on the corresponding storage address obtained from the query, the second feedback instruction has a signature and a validity period; 204) Perform data access according to the second feedback instruction. 5. The distributed data storage access method based on object storage according to claim 4, wherein the site information table includes a main site and a plurality of sub-sites, and each site is configured with object storage information and User login information. 6. The method for accessing distributed data storage based on object storage according to claim 4, wherein the user login information includes a user name and a password. 7. A server, characterized in that it comprises one or more processors, memory and one or more programs stored in the memory, the one or more programs including the one or more programs used to execute claims 1-3 Instructions of any one of the distributed data storage access methods based on object storage. 8. A client, characterized in that it comprises one or more processors, memory and one or more programs stored in the memory, the one or more programs include a program used to execute claims 4-6 Instructions of any one of the distributed data storage access methods based on object storage. 9. A distributed data storage access system based on object storage, comprising a server and a client, wherein, The server maintains a site information table, and authenticates user login information based on the site information table; receives a storage location request, determines the corresponding object storage according to the site information table, and generates a first feedback instruction, the first feedback instruction With a signature and a validity period; when the successful storage information is received, a data record is generated and saved, and the data record includes the data ID, the user and the storage location; the record query command is received, and the record query command includes the login information of the administrator, and the judgment Whether the administrator login information is authenticated, if so, all data records are fed back; The client establishes a connection with the server through the user login information and the address of the server, sends a storage location request when data storage is required, and stores the data according to the first feedback instruction after receiving the first feedback instruction, wherein , the first feedback instruction is generated according to the corresponding object storage determined by the site information table, and the first feedback instruction has a signature and a validity period; when data access is required, a query instruction is sent, and if a query matching the query instruction is received data records, send an access instruction, the access instruction includes data ID and user login information, after receiving the second feedback instruction, perform data access according to the second feedback instruction, the second feedback instruction is based on the query obtained A corresponding storage address is generated, and the second feedback command has a signature and a validity period. 10. The distributed data storage access system based on object storage according to claim 9, wherein the client is replaced by an API module.

Images