CN110600004A - Voice synthesis playing method and device and storage medium - Google Patents

Abstract

The embodiment of the present invention discloses a voice synthesis playback method, device and storage medium, wherein the user terminal can receive the voice synthesis request, obtain the text to be synthesized that needs to be synthesized according to the voice synthesis request, and then send the text to be synthesized to The speech synthesis server performs speech synthesis, obtains the corresponding synthesized speech, then plays the synthesized speech, receives the pronunciation correction request for the synthesized speech, receives the correction data corresponding to the synthesized speech according to the pronunciation correction request, and sends the correction data to the speech synthesis The server is used to update the synthesized voice, so as to obtain the updated synthesized voice, and replace the currently played synthesized voice with the updated synthesized voice for playback. Compared with related technologies, the present invention can correct and update the played synthesized voice in real time during the process of playing the synthesized voice, so that even when the pronunciation prediction of polyphonic characters is wrong, the pronunciation can be corrected in time.

Description

A speech synthesis playback method, device and storage medium

technical field

The present invention relates to the technical field of speech, in particular to a speech synthesis playback method, device and storage medium.

Background technique

Speech synthesis technology, also known as text-to-speech technology (Text To Speech, TTS), its goal is to allow machines to convert text information into voice output through recognition and understanding, so that machines can speak, which is the future of human-computer interaction. important branch.

Speech synthesis technology is widely used, such as web page content reading aloud, novel audio reading, e-mail reading and so on. Take the audio reading of novels as an example. Through speech synthesis, user terminals such as mobile phones and tablet computers can read the novels read by users aloud, so that users can "read" novels with their eyes closed.

During the research and practice of the prior art, the inventors of the present invention found that the polyphonic character processing ability of the existing speech synthesis technology has defects, and it is often impossible to accurately predict the polyphonic character when faced with an uncommon context. pronunciation.

Contents of the invention

Embodiments of the present invention provide a speech synthesis playback method, device and storage medium, capable of timely correcting the pronunciation of a polyphone when the pronunciation prediction of a polyphonic character is wrong.

An embodiment of the present invention provides a speech synthesis playback method, including:

receiving a speech synthesis request, and obtaining text to be synthesized that needs to be speech synthesized according to the speech synthesis request;

Sending the text to be synthesized to a speech synthesis server for speech synthesis, so that the speech synthesis server returns a synthesized speech corresponding to the text to be synthesized;

Playing the synthesized voice, and receiving a pronunciation correction request for the synthesized voice;

receiving input correction data corresponding to the synthesized speech according to the pronunciation correction request, and sending the correction data to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and return the updated synthesized speech;

The currently played synthesized voice is replaced with the updated synthesized voice for playback.

The embodiment of the present invention also provides a speech synthesis playback method, including:

When receiving the text to be synthesized from the user terminal, performing speech synthesis on the text to be synthesized according to a pre-trained speech synthesis model to obtain a synthesized speech;

returning the synthesized voice to the user terminal for playing, and receiving correction data corresponding to the synthesized voice returned by the user terminal;

updating the synthesized speech according to the correction data to obtain the updated synthesized speech;

returning the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playing.

The embodiment of the present invention also provides a speech synthesis playback device, comprising:

A text acquisition module, configured to receive a speech synthesis request, and obtain text to be synthesized that requires speech synthesis according to the speech synthesis request;

A speech synthesis module, configured to send the text to be synthesized to a speech synthesis server for speech synthesis, so that the speech synthesis server returns the synthesized speech corresponding to the text to be synthesized;

A voice playback module, configured to play the synthesized voice and receive a pronunciation correction request for the synthesized voice;

A text correction module, configured to receive input correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis server, so that the speech synthesis server updating the synthesized speech, and returning the updated synthesized speech;

The voice playing module is also used to replace the currently played synthesized voice with the updated synthesized voice for playback.

In one embodiment, after receiving the correction data corresponding to the synthesized speech according to the pronunciation correction request, the text correction module is used for:

According to the pronunciation correction request, a pronunciation correction interface is displayed, and the pronunciation correction interface includes a word input control and a pronunciation control;

receiving a target word to be corrected in the text to be synthesized based on the word input control;

receiving a target pronunciation corresponding to the target word based on the pronunciation control;

The target word and the target pronunciation are set as the correction data.

In one embodiment, when the target pronunciation of the corresponding target word is received based on the pronunciation control, the text correction module is used for:

Check whether the target word is a polyphonic word;

When determining that the target word is a polyphonic character, according to the preset correspondence between polyphonic characters and pronunciations, obtain a plurality of pronunciations corresponding to the target word;

displaying the plurality of pronunciations based on the pronunciation control, and receiving a selection operation on the displayed pronunciations;

The pronunciation corresponding to the selection operation is set as the target pronunciation of the target word.

In one embodiment, when obtaining the text to be synthesized that needs to be synthesized according to the speech synthesis request, the text acquisition module is used for:

Extracting the text in the display content of the foreground application according to the speech synthesis request to obtain the extracted text;

Dividing the extracted text into multiple clauses according to a preset clause strategy;

Set the clause as the text to be synthesized.

In one embodiment, in the process of playing synthesized speech, the speech playback module is also used for:

The clauses corresponding to the synthesized speech are identified according to preset rules.

In one embodiment, when receiving a pronunciation correction request for synthesized speech, the speech playback module is used to:

Displaying a pronunciation correction control within a preset range of a sentence corresponding to the synthesized speech;

A pronunciation correction request for synthesized speech is received based on the pronunciation correction control.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a data storage module for:

storing the text to be synthesized, the synthesized speech and/or the updated synthesized speech in a distributed system.

An embodiment of the present invention also provides a speech synthesis playback device, including a speech synthesis module, a speech delivery module, and a speech update module, wherein,

The speech synthesis module is configured to, when receiving the text to be synthesized from the user terminal, perform speech synthesis on the text to be synthesized according to a pre-trained speech synthesis model to obtain synthesized speech;

The voice sending module is used to return the synthesized voice to the user terminal for playback, and receive correction data corresponding to the text to be synthesized returned by the user terminal;

The voice update module is used to update the synthesized voice according to the correction data to obtain the updated synthesized voice;

The voice sending module is further configured to return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a model update module, which is used for:

The speech synthesis model is updated according to the text to be synthesized and the correction data.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a data storage module for:

storing the text to be synthesized, the synthesized speech and/or the updated synthesized speech in a distributed system.

In addition, an embodiment of the present invention also provides a storage medium, the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute any speech synthesis playback method provided in the embodiments of the present invention.

In the present invention, the speech synthesis request is received by the user terminal, and the text to be synthesized that needs to be synthesized is obtained according to the speech synthesis request, and then the text to be synthesized is sent to the speech synthesis server for speech synthesis to obtain the corresponding synthesized speech, and then the text is played. Synthesizing speech, and receiving a pronunciation correction request for the synthesized speech, receiving correction data corresponding to the synthesized speech according to the pronunciation correction request, sending the correction data to the speech synthesis server for updating the synthesized speech, thereby obtaining the updated synthesized speech, and The currently playing synthesized voice is replaced with the updated synthesized voice for playback. Compared with related technologies, the present invention can correct and update the played synthesized voice in real time during the process of playing the synthesized voice, so that even when the pronunciation prediction of polyphonic characters is wrong, the pronunciation can be corrected in time.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a schematic diagram of the architecture of a speech synthesis playback system in an embodiment of the present invention;

Fig. 2a is a schematic flow chart of a speech synthesis playback method provided by an embodiment of the present invention;

Fig. 2b is a schematic diagram showing speech synthesis controls in an embodiment of the present invention;

Fig. 2c is a schematic diagram of a clause 1 corresponding to the identified synthesized speech in the embodiment of the present invention;

Fig. 2d is another schematic diagram of the sentence corresponding to the identification synthesized speech in the embodiment of the present invention;

Fig. 2e is a schematic diagram showing pronunciation correction controls in an embodiment of the present invention;

Fig. 2f is a schematic diagram showing a pronunciation correction interface in an embodiment of the present invention;

Fig. 2g is a schematic structural diagram of a distributed system in an embodiment of the present invention;

Fig. 2h is a schematic diagram of a block structure in an embodiment of the present invention;

Fig. 3 is another schematic flow chart of the speech synthesis playback method provided by the embodiment of the present invention;

Fig. 4 is another schematic flow chart of the speech synthesis playback method provided by the embodiment of the present invention;

5 is a schematic structural diagram of a speech synthesis playback device provided by an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a speech synthesis playback device provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a user terminal in an embodiment of the present invention;

Fig. 8 is a schematic structural diagram of the speech synthesis server in the embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

The key branches of Speech Technology include Automatic Speech Recognition (ASR), Text To Speech (TTS) and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the development direction of human-computer interaction in the future. Among them, speech synthesis technology has become one of the most promising methods of human-computer interaction in the future.

Early speech synthesis was generally implemented using dedicated chips, such as Texas Instruments' TMS50C10/TMS50C57, Philips' PH84H36, etc., but it was mainly used in household appliances and children's toys.

Today's speech synthesis is generally implemented by pure software. The text-to-speech conversion process is as follows: firstly, the text is preprocessed, word segmentation, part-of-speech tagging, polyphonic word prediction, prosodic level prediction, etc., and then through the acoustic model, each unit is predicted Corresponding to the acoustic features, finally use the acoustic parameters to directly synthesize the sound through the vocoder, or select units from the recorded lexicon for splicing to generate the speech corresponding to the text.

For Chinese speech synthesis, the current key research directions are Chinese prosodic processing, symbolic numbers, polyphonic word prediction, and word formation, etc., which need continuous research to improve the naturalization of Chinese speech synthesis.

Among them, the prediction of polyphonic characters is one of the foundations of Chinese speech synthesis. Whether the pronunciation of polyphonic characters is correct or not greatly affects the semantic understanding of the synthesized voice by the listener. If the prediction accuracy of polyphonic characters is high, the user experience will be greatly improved. Make the synthesized speech easier to understand and sound more natural and smooth.

At present, for polyphonic characters, the existing speech synthesis mostly adopts the following synthesis strategy:

If the polyphonic character can be combined with the context to form a word, the speech synthesis is carried out according to the polyphonic character in the fixed collocation, such as heavy (zhong4) point, heavy (chong2) new, wherein the number after the pinyin represents the tone;

If the polyphonic character appears in the form of a single character, use the speech synthesis model trained by using a large number of sample data in advance to predict its pronunciation, such as (wei4) serving the people, the result is (wei2) zero.

Among them, commonly used methods for training speech synthesis models include but are not limited to: Conditional Random Field (Conditional Random Field, CRF) method, Hidden Markov Model (Hidden Markov Model, HMM) method, decision tree method and so on. The characteristic of these methods is that the pronunciation of a large number of polyphonic characters is required for training. The advantage is that the pronunciation of polyphonic characters can be predicted based on the text alone, and the prediction accuracy for polyphonic characters that appear in common contexts is high. The disadvantage is that the processing ability for polyphonic characters in uncommon contexts is poor.

Based on the above defects in the prior art, embodiments of the present invention provide a speech synthesis playback method, device and storage medium. Among them, it includes a speech synthesis playing method, device and storage medium suitable for a user terminal, and a speech synthesis playing method, device and storage medium suitable for a speech synthesis server.

Please refer to Fig. 1, the embodiment of the present invention also provides a kind of speech synthesis playing system, this speech synthesis playing system comprises user terminal 10, speech synthesis server 20 and network 30 (can be wired network, also can be wireless network), user terminal 10 interacts with speech synthesis server 20 over network 30. Wherein, the network 30 includes network entities such as routers and gateways, which are not shown in FIG. 1 .

Based on the speech synthesis playback system shown in Figure 1, the user terminal 10 can receive the speech synthesis request, and obtain the text to be synthesized that needs to be synthesized according to the speech synthesis request, and then send the text to be synthesized to the speech synthesis server 20; the speech synthesis server 20. After receiving the text to be synthesized from the user terminal 10, perform speech synthesis on the text to be synthesized to obtain corresponding synthesized speech, and return the synthesized speech to the user terminal 10; After the synthesized speech, play this synthesized speech, and receive the pronunciation correction request to synthesized speech, then receive the correction data corresponding to synthesized speech according to the pronunciation correction request, and send this corrected data to speech synthesis server 20; Speech synthesis server 20 After receiving the correction data from the user terminal 10, update the synthesized speech according to the correction data, and return the updated synthesized speech to the user terminal 10; After the speech, replace the currently played synthesized speech with the updated synthesized speech for playback.

It should be noted that the foregoing FIG. 1 shows only an example of a system architecture for implementing the embodiment of the present invention, and the embodiment of the present invention is not limited to the system architecture shown in the foregoing FIG. 1 . Based on the system architecture, detailed descriptions are given below respectively. It should be noted that the order of the following examples is not intended to limit the preferred order of the examples.

Embodiment one,

An embodiment of the present invention provides a speech synthesis playback method, which is suitable for a user terminal, including: receiving a speech synthesis request, and obtaining text to be synthesized that needs to be synthesized according to the speech synthesis request; sending the text to be synthesized to a speech synthesis server for speech Synthesize, so that the speech synthesis server returns the synthesized speech corresponding to the text to be synthesized; play the synthesized speech, and receive the pronunciation correction request for the synthesized speech; receive the correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis The server enables the speech synthesis server to update the synthesized speech according to the correction data, and return the updated synthesized speech; replace the currently played synthesized speech with the updated synthesized speech for playback.

Please refer to Figure 2a, the flow of the speech synthesis playback method can be as follows:

201. Receive a speech synthesis request, and acquire text to be synthesized that requires speech synthesis according to the speech synthesis request.

In the embodiment of the present invention, the user terminal may receive an externally input speech synthesis request in real time, thereby triggering speech synthesis, and converting the corresponding text into speech for output. Wherein, the user terminal may receive the speech synthesis request directly input by the user, or may receive the speech synthesis request input by other user terminals.

Exemplarily, for the user, the speech synthesis request can be input to the user terminal in many different ways.

For example, the user can say "please read the current interface/full text" in the form of a voice command, so as to input to the user terminal to indicate the text in the current interface (for example, web page browsing interface, text browsing interface, etc.) or the full text. Speech synthesis request for speech synthesis.

For another example, the user terminal is provided with a speech synthesis control for inputting a speech synthesis request. As shown in FIG. The terminal input is used to indicate a speech synthesis request for performing speech synthesis on the text in the current interface.

It should be noted that those skilled in the art can configure the user terminal according to actual needs, so that the user terminal can also receive speech synthesis requests input in other ways not shown above.

After receiving the speech synthesis request, the user terminal further acquires the text to be synthesized that needs to be speech synthesized according to the speech synthesis request.

In an embodiment, "obtaining the text to be synthesized that needs to be synthesized according to the speech synthesis request" includes:

(1) Extract the text in the display content of the foreground application according to the speech synthesis request, and obtain the extracted text;

(2) Divide the extracted text into multiple clauses according to the preset clause strategy;

(3) Set the clause as the text to be synthesized.

In the embodiment of the present invention, the speech synthesis request is used to instruct the user terminal to perform speech synthesis on the text in its foreground application, and the foreground application is the application currently displayed by the user.

Correspondingly, when the user terminal obtains the text to be synthesized that needs to be synthesized according to the speech synthesis request, it first extracts the text in the display content of the foreground application according to the received speech synthesis request, and records the extracted text as the extracted text .

For example, when a user terminal receives a speech synthesis request while browsing webpages through a browser application, it receives the DOM tree of the webpage according to the speech synthesis request, and extracts the text in the webpage based on the text density calculation method, such as the text of news articles or novels. Chapter content, etc.;

For another example, when the user terminal receives a speech synthesis request during browsing of local documents (such as txt, word, etc.) through a text reading application, the local document is encoded and decoded according to the speech synthesis request, and the GB2312-encoded text is parsed out. Plain text content.

After the extracted text corresponding to the foreground application is extracted, the user terminal further divides the extracted text into multiple clauses according to the preset sentence segmentation strategy. It should be noted that the embodiment of the present invention does not specifically limit the configuration of the preset clause strategy, and it can be configured by those skilled in the art according to actual needs. For example, in the embodiment of the present invention, the configured preset clause strategy To divide sentences according to punctuation and length.

For the multiple divided sentences, the user terminal sequentially sets each sentence as the text to be synthesized, so as to perform speech synthesis on each sentence.

202. Send the text to be synthesized to the speech synthesis server for speech synthesis, so that the speech synthesis server returns synthesized speech corresponding to the text to be synthesized.

After obtaining the text to be synthesized that needs to be synthesized, the user terminal constructs a speech synthesis capability request carrying the text to be synthesized according to a predetermined data format, and sends the speech synthesis energy request to the speech synthesis server, instructing the speech synthesis server Perform speech synthesis.

Exemplarily, the data format of a speech synthesis capability request is as follows:

Among them, header indicates the request header of the speech synthesis capability request, header.guid indicates the unique identifier of the user terminal, header.qua indicates the device and application information of the user terminal, header.user indicates user information, and header.user.user_id indicates the user's unique ID, header.lbs indicates user location information, header.lbs.longitude indicates longitude, header.lbs.latitude indicates latitude, header.ip indicates the IP address of the user terminal, and header.device.network indicates the network type of the user terminal.

payload indicates the request content of the speech synthesis capability request, payload.speech_meta indicates the voice configuration information, payload.speech_meta.compress indicates the compression type, payload.speech_meta.person indicates the speaker, payload.speech_meta.volume indicates the pronunciation volume, and payload.speech_meta.speed Indicates the pronunciation speed, payload.speech_meta.pitch indicates the pitch, payload.session_id indicates the session ID, payload.index indicates the serial number of the requested voice film, payload.single_request indicates the type of speech synthesis, payload.content indicates the content of speech synthesis, payload.content .text is used to fill in the text to be synthesized.

On the other hand, when the speech synthesis server receives the speech synthesis capability request from the user terminal, it performs speech synthesis according to the speech synthesis energy request, obtains the synthesized speech corresponding to the text to be synthesized, and returns the synthesized speech to the user terminal.

For example, corresponding to the data format of the speech synthesis energy request shown above, the data format of the synthesized speech returned by the speech synthesis server is as follows:

Among them, header indicates the message header, header.session indicates the session, header.session.session_id indicates the session ID, payload indicates the message body, payload.speech_finished indicates whether it is finished, and payload.speech_base64 indicates the Base64 data of the synthesized speech.

203. Play the synthesized voice, and receive a pronunciation correction request for the synthesized voice.

After receiving the synthesized voice returned by the speech synthesis server, the user terminal plays the synthesized voice, and receives a pronunciation correction request for the synthesized voice during the process of playing the synthesized voice.

In one embodiment, during the process of playing synthesized speech, the speech synthesis playback method provided by the embodiment of the present invention further includes:

The clauses corresponding to the synthesized speech are identified according to preset rules.

In the embodiment of the present invention, after extracting the extracted text of the foreground application, the user terminal can create a speech synthesis playback interface covering the foreground application, and display the extracted text in the speech synthesis playback interface.

During the process of playing the synthesized voice, the user terminal identifies the clauses corresponding to the synthesized voice according to the preset rules. The purpose of marking the clauses corresponding to the synthesized voice is to highlight the clauses so that the clauses are distinguished from each other. Displayed in other clauses, so that users can quickly locate the clause being played from the extracted text. It should be noted that, in the embodiment of the present invention, there is no specific limitation on the way of configuring the preset rules, and those skilled in the art can configure according to actual needs.

For example, the preset rule can be configured to increase the display ratio. As shown in Figure 2c, the clause set as the text to be synthesized is "Wang Baole didn't understand at the beginning", and this clause has a larger value than other clauses. The display ratio of , making it distinct from other clauses.

For another example, the default rule can be configured to adjust the display color. As shown in Figure 2d, the clause set as the text to be synthesized is "Wang Baole didn't understand at the beginning", and this clause has a different meaning than other clauses. Display the color so that it is clearly distinguished from other clauses.

In an embodiment, "receiving a pronunciation correction request for synthesized speech" includes:

(1) displaying pronunciation correction controls within the preset range of the sentence corresponding to the synthesized speech;

(2) A pronunciation correction request for the synthesized speech is received based on the pronunciation correction control.

In the embodiment of the present invention, during the process of playing the synthesized voice, the user terminal not only identifies the clauses corresponding to the synthesized voice according to the preset rules, but also displays the pronunciation correction control within the preset range of the clauses corresponding to the synthesized voice , so as to receive a pronunciation correction request for the played synthesized speech through the pronunciation correction control. It should be noted that, for the configuration of the preset range, there is no specific limitation in the embodiment of the present invention, and it can be configured by those of ordinary skill in the art according to actual needs.

For example, as shown in Figure 2e, the clause "Wang Baole didn't understand at the beginning" is set as the text to be synthesized. During the process of playing the synthesized speech corresponding to "Wang Baole didn't understand at the beginning", the user terminal changes the display color way to mark "Wang Baole didn't understand at the beginning", and at the same time, the user terminal displays a pronunciation correction control at the end of "Wang Baole did not understand at the beginning", so that the user can click on the pronunciation correction control to input the pronunciation to the user terminal Correction request.

204. Receive correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and returns the updated synthesized speech.

In the embodiment of the present invention, after receiving the input pronunciation correction request, the user terminal further receives the correction data corresponding to the synthesized speech according to the pronunciation correction request, and after receiving the correction data corresponding to the synthesized speech, sends the correction data to to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and returns the updated synthesized speech. Wherein, the correction data includes the word to be corrected and the correct pronunciation.

For example, the user terminal can send correction data in the form of a speech synthesis capability request, but the difference from the data format of the speech synthesis capability request shown above is that two additional fields are added here, namely "report_correction" and "correct_phonetic ", among them, report_correction is used to indicate whether the speech synthesis capability request sent this time is used to update the synthesized speech. When the written value is "true", it means update; when the written value is "false", it means that the speech is performed normally synthetic, correct_phonetic is used to write correction data. When the speech synthesis server receives the speech synthesis capability request from the user terminal, it determines whether it is to update the synthesized speech according to the "report_correction", and if so, extracts the correction data and the text to be synthesized from the "correct_phonetic", and The data and the text to be synthesized are re-synthesized to obtain a new synthesized voice, which is set as the updated synthesized voice and returned to the user terminal.

In an embodiment, "receiving correction data corresponding to synthesized speech according to a pronunciation correction request" includes:

(1) Display the pronunciation correction interface according to the pronunciation correction request, and the pronunciation correction interface includes word input controls and pronunciation controls;

(2) receiving the target word to be corrected in the text to be synthesized based on the word input control;

(3) receiving the target pronunciation of the corresponding target word based on the pronunciation control;

(4) Set the target word and target pronunciation as correction data.

In the embodiment of the present invention, when receiving the correction data corresponding to the synthesized speech according to the pronunciation correction request, the user terminal first displays the pronunciation correction interface according to the received pronunciation correction request, and the pronunciation correction interface includes a character input control and a pronunciation control, wherein, The word input control is used to receive the target word to be corrected in the text to be synthesized, and the pronunciation control is used to receive the target pronunciation of the target word.

Thus, the user terminal can receive the target word to be corrected input by the user based on the character input control, and receive the target pronunciation corresponding to the target word based on the pronunciation control, and set the target word and its corresponding target pronunciation as correction data.

Wherein, before setting the target word and its corresponding target pronunciation as correction data, the user terminal also identifies whether the target word belongs to the text to be synthesized corresponding to the synthesized speech, and only when the target word belongs to the text to be synthesized, the The received target word and target pronunciation are set to correspond to the corrected data of the played synthesized speech, thereby ensuring the accuracy of correcting the synthesized speech.

In one embodiment, "receiving the target pronunciation of the corresponding target word based on the pronunciation control" includes:

(1) check whether the target word is a polyphonic word;

(2) When determining that the target word is a polyphonic word, according to the preset correspondence between the polyphonic word and the pronunciation, obtain a plurality of pronunciations corresponding to the target word;

(3) Display multiple pronunciations based on the pronunciation control, and receive a selection operation on the displayed pronunciations;

(4) Set the pronunciation corresponding to the selection operation as the target pronunciation of the target word.

In the embodiment of the present invention, in order to further ensure the accuracy of the synthetic speech correction, when the user terminal receives the target pronunciation of the corresponding target word based on the pronunciation control, it first checks whether the target word is a polyphonic word, so as to eliminate the user's mistaken input at the source. resulting in miscorrection.

For example, the polyphonic database is pre-configured in the user terminal, and the polyphonic database stores known polyphonic characters. When checking whether the target word is a polyphonic character, the user terminal can query whether there is a target word input by the user in the polyphonic database. If it exists, then the verification is passed, and it is determined that the target word input by the user is a polyphonic word.

When judging that the input target word is a polyphonic character, the user terminal further acquires multiple pronunciations corresponding to the target word according to the preset correspondence between polyphonic characters and pronunciations. Then, the user terminal displays the acquired multiple pronunciations corresponding to the target word based on the pronunciation control, receives a selection operation on the displayed pronunciations, and sets the pronunciation corresponding to the selection operation as the target pronunciation of the target word.

For example, please refer to Figure 2f, the pronunciation correction interface shows:

The clause set as the text to be synthesized is "Wang Baole didn't understand at the beginning";

The word input control in the form of an input box and the first prompt message "Please enter the word to be corrected" prompt the user to input the target word to be corrected, for example, the user has entered "乐" in the illustration;

The pronunciation control in the form of a selection box and the second prompt message "Please check the correct pronunciation" prompt the user to select the correct pronunciation as the target pronunciation, wherein the number of pronunciation controls is the same as the number of pronunciations of the corresponding target word obtained, as shown in Fig. The pronunciation control showing the pronunciation "le4" and the pronunciation control showing the pronunciation "yue4" in the demo;

The "report correction" control is used to indicate the completion of the input. When the user completes the input, the "report correction" control can be clicked, so that the user terminal can obtain the target word "le" and the corresponding target pronunciation "le4".

205. Replace the currently played synthesized voice with the updated synthesized voice for playback.

Wherein, when the user terminal receives the updated synthesized voice returned by the speech synthesis server, it replaces the currently played synthesized voice with the updated synthesized voice and plays it, so as to realize the pronunciation correction of the synthesized voice.

In one embodiment, the speech synthesis playback method provided by the embodiment of the present invention further includes:

The text to be synthesized, the synthesized speech and/or the updated synthesized speech are stored in the distributed system.

Taking the distributed system as a block chain system as an example, please refer to FIG. 2g. FIG. 2g is an optional structural diagram of the distributed system 100 applied to the block chain provided by the embodiment of the present invention, which consists of multiple nodes (this The user terminal mentioned in the above embodiments of the invention, other user terminals and speech synthesis server) and the client are formed, and a point-to-point (P2P, Peer To Peer) network is formed between the nodes, and the P2P protocol is a transmission control protocol (TCP) that runs on , the application layer protocol above the TransmissionControl Protocol) protocol. Nodes include hardware layer, middle layer, operating system layer and application layer.

Referring to the functions of each node in the blockchain system shown in Figure 2g, the functions involved include:

1) Routing, the basic function of nodes, is used to support communication between nodes.

In addition to the routing function, nodes can also have the following functions:

2) Application, which is used to deploy in the blockchain, realize specific business according to actual business needs, record the data related to the realization function to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data to To other nodes in the blockchain system, for other nodes to add the record data to the temporary block when they verify the source of the record data and the integrity is successful.

For example, the services implemented by the application include:

2.1) Wallet, which is used to provide the function of conducting electronic currency transactions, including initiating transactions (that is, sending the transaction records of the current transaction to other nodes in the blockchain system, and after other nodes are successfully verified, they will be recognized as a valid response to the transaction , store the transaction record data in the temporary block of the blockchain; of course, the wallet also supports querying the remaining electronic currency in the electronic currency address;

2.2) Shared account book, which is used to provide the functions of storage, query and modification of account data, and send the record data of the operation of account data to other nodes in the blockchain system. Respond with valid data, store the recorded data in the temporary block, and send a confirmation to the node that initiated the operation.

2.3) Smart contracts, computerized agreements, can execute the terms of a certain contract, implemented by code deployed on the shared ledger for execution when certain conditions are met, and codes are used to complete automated transactions according to actual business needs. For example, query the logistics status of the goods purchased by the buyer, and transfer the buyer's electronic money to the merchant's address after the buyer signs for the goods; of course, smart contracts are not limited to the execution of contracts for transactions, but can also execute the received information. Processed contracts.

3) Blockchain, including a series of blocks (Block) that are successively connected to each other according to the time sequence of generation. Once a new block is added to the blockchain, it will not be removed again. The blockchain system is recorded in the block. The record data submitted by the middle node.

Referring to Fig. 2h, Fig. 2h is an optional schematic diagram of the block structure (Block Structure) provided by the embodiment of the present invention, each block includes the hash value of the block storage transaction record (the hash value of the block ), and the hash value of the previous block, each block is connected through the hash value to form a blockchain. In addition, the block may also include information such as a time stamp when the block was generated. Blockchain (Blockchain), essentially a decentralized database, is a series of data blocks associated with each other using cryptographic methods. Each data block contains relevant information to verify the validity of its information. (anti-counterfeiting) and generate the next block.

In the embodiment of the present invention, the user terminal may also store the text to be synthesized and its corresponding synthesized voice and/or updated synthesized voice in the distributed system where it is located in the above voice synthesis process for recording.

As can be seen from the above, in the embodiment of the present invention, the user terminal can receive the speech synthesis request, and obtain the text to be synthesized that needs to be synthesized according to the speech synthesis request, and then send the text to be synthesized to the speech synthesis server for speech synthesis, and obtain the corresponding Synthesize voice, then play the synthesized voice, and receive a pronunciation correction request for the synthesized voice, receive correction data corresponding to the synthesized voice according to the pronunciation correction request, and send the correction data to the speech synthesis server for updating the synthesized voice, thereby obtaining an update The updated synthesized voice will replace the currently played synthesized voice with the updated synthesized voice for playback. Compared with related technologies, the present invention can correct and update the played synthesized voice in real time during the process of playing the synthesized voice, so that even when the pronunciation prediction of polyphonic characters is wrong, the pronunciation can be corrected in time.

Embodiment two,

An embodiment of the present invention also provides a speech synthesis playback method, which is suitable for a speech synthesis server, including: when receiving text to be synthesized from a user terminal, performing speech synthesis on the text to be synthesized according to a pre-trained speech synthesis model, and obtaining Synthesize voice; return the synthesized voice to the user terminal for playback, and receive the correction data corresponding to the synthesized voice returned by the user terminal; update the synthesized voice according to the correction data to obtain an updated synthesized voice; return the updated synthesized voice to the user terminal , so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

Please refer to Fig. 3, the flow process of this speech synthesis playing method can be as follows:

301. When receiving text to be synthesized from a user terminal, perform speech synthesis on the text to be synthesized according to a pre-trained speech synthesis model to obtain synthesized speech.

Wherein, when receiving an input speech synthesis request, the user terminal acquires the text to be synthesized that needs to be speech synthesized according to the speech synthesis request. Correspondingly, the speech synthesis server receives the text to be synthesized from the user terminal, and when receiving the text to be synthesized from the user terminal, performs speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model to obtain synthesized speech.

It should be noted that the speech synthesis model can be pre-trained using a conditional random field (Conditional Random Field, CRF) method, a hidden Markov model (Hidden Markov Model, HMM) method, a decision tree method, etc. Do repeat.

302. Return the synthesized speech to the user terminal for playing, and receive correction data corresponding to the synthesized speech returned by the user terminal.

After the speech synthesis server synthesizes the synthesized speech, it returns the synthesized speech to the user terminal for playback.

On the other hand, during the process of playing the synthesized speech, the user terminal receives a pronunciation correction request for the synthesized speech, receives input correction data according to the pronunciation correction request, and sends the correction data to the speech synthesis server. Correspondingly, the speech synthesis server also receives the correction data corresponding to the text to be synthesized returned by the user terminal.

303. Update the synthesized speech according to the correction data to obtain the updated synthesized speech.

After receiving the correction data returned by the user terminal, the speech synthesis server updates the previously synthesized speech according to the correction data to obtain the updated synthesized speech.

For example, for the text to be synthesized "Wang Baole didn't understand at the beginning", the pronunciation of "Le" in the synthesized voice is "yue4". After updating, the pronunciation of "Le" in the updated synthesized voice is "le4".

304. Return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

After obtaining the updated synthesized voice, the voice synthesis server returns the updated synthesized voice to the electronic device, so that the electronic device replaces the currently played synthesized voice with the updated synthesized voice to be played mechanically and realizes pronunciation correction.

It should be noted that, for the part that is specifically described in the embodiment of the present invention, reference may be made to the relevant description in the above embodiment of the speech synthesis playback method applicable to the user terminal, and details are not repeated here.

In one embodiment, the speech synthesis playback method provided by the embodiment of the present invention further includes:

The speech synthesis model is updated according to the text to be synthesized and the correction data.

For example, each time the speech synthesis server receives correction data from the user terminal, it stores the correction data and its corresponding text to be synthesized as training corpus in a pre-created database, continuously enriches the training corpus, and waits for the training corpus in the database to When the preset amount is accumulated, the speech synthesis model is updated by means of supervised learning according to the stored training corpus, so that the speech synthesis model can more accurately predict the pronunciation of polyphonic characters.

In one embodiment, the speech synthesis method provided by the embodiment of the present invention further includes:

The text to be synthesized, the synthesized speech and/or the updated synthesized speech are stored in the distributed system.

Wherein, the distributed system is taken as an example of a block chain system, which is formed by multiple nodes (user terminals and speech synthesis servers mentioned in the above embodiments of the present invention, etc.) and clients.

In the embodiment of the present invention, the speech synthesis server may also store the text to be synthesized during the speech synthesis process and its corresponding synthesized speech and/or updated synthesized speech in its distributed system for recording.

Embodiment three,

According to the methods described in the previous embodiments, examples will be given below for further description.

As shown in Figure 4, the flow of the speech synthesis playback method can be as follows:

401. The user terminal receives the speech synthesis request, and extracts the text in the display content of the foreground application according to the speech synthesis request, obtains the extracted text, and divides the extracted text into multiple clauses according to the preset sentence segmentation strategy, and sequentially divides the divided text The obtained sentence is set as the text to be synthesized and sent to the speech synthesis server.

In the embodiment of the present invention, the user terminal may receive an externally input speech synthesis request in real time, thereby triggering speech synthesis, and converting the corresponding text into speech for output.

After receiving the speech synthesis request, the user terminal further acquires the text to be synthesized that needs to be speech synthesized according to the speech synthesis request. Wherein, firstly, according to the received speech synthesis request, the text in the display content of the foreground application is extracted, and the extracted text is recorded as the extracted text.

For example, when a user terminal receives a speech synthesis request while browsing webpages through a browser application, it receives the DOM tree of the webpage according to the speech synthesis request, and extracts the text in the webpage based on the text density calculation method, such as the text of news articles or novels. Chapter content, etc.;

For another example, when the user terminal receives a speech synthesis request during browsing of local documents (such as txt, word, etc.) through a text reading application, the local document is encoded and decoded according to the speech synthesis request, and the GB2312-encoded text is parsed out. Plain text content.

After the extracted text corresponding to the foreground application is extracted, the user terminal further divides the extracted text into multiple clauses according to the preset sentence segmentation policy. It should be noted that the embodiment of the present invention does not specifically limit the configuration of the preset clause strategy, and it can be configured by those skilled in the art according to actual needs. For example, in the embodiment of the present invention, the configured preset clause strategy To divide sentences according to punctuation and length.

For the multiple divided sentences, the user terminal sequentially sets each sentence as the text to be synthesized, so as to perform speech synthesis on each sentence.

After obtaining the text to be synthesized that needs to be synthesized, the user terminal constructs a speech synthesis capability request carrying the text to be synthesized according to a predetermined data format, and sends the speech synthesis energy request to the speech synthesis server, instructing the speech synthesis server Perform speech synthesis.

402. The speech synthesis server performs speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model, obtains the synthesized speech, and returns it to the user terminal.

It should be noted that the speech synthesis model can be pre-trained using a conditional random field (Conditional Random Field, CRF) method, a hidden Markov model (Hidden Markov Model, HMM) method, a decision tree method, etc. Do repeat.

Wherein, the speech synthesis server receives the text to be synthesized from the user terminal, when receiving the text to be synthesized from the user terminal, performs speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model, obtains the synthesized speech, and returns to the user terminal .

403. The user terminal plays the synthesized voice, identifies the clause corresponding to the synthesized voice according to a preset rule, and displays a pronunciation correction control within a preset range of the clause.

404. The user terminal receives a pronunciation correction request for the synthesized voice based on the pronunciation correction control.

After receiving the synthesized voice returned by the voice synthesis server, the user terminal plays the synthesized voice, and in the process of playing the synthesized voice, identifies the clauses corresponding to the synthesized voice according to preset rules.

Among them, by marking the clauses corresponding to the synthesized speech, the purpose is to highlight the clauses so that the clauses are displayed differently from other clauses, so that users can quickly locate the clauses being played from the extracted text. It should be noted that, in the embodiment of the present invention, there is no specific limitation on the way of configuring the preset rules, and those skilled in the art can configure according to actual needs.

In addition to identifying the sentence corresponding to the synthesized speech according to the preset rules, the user terminal also displays the pronunciation correction control within the preset range of the sentence corresponding to the synthesized speech, so as to receive feedback on the played synthesized speech through the pronunciation correction control. Pronunciation correction request. It should be noted that, for the configuration of the preset range, there is no specific limitation in the embodiment of the present invention, and it can be configured by those of ordinary skill in the art according to actual needs.

405. The user terminal displays a pronunciation correction interface according to the pronunciation correction request, and the pronunciation correction interface includes a character input control and a pronunciation control;

406. The user terminal receives the target word to be corrected in the text to be synthesized based on the word input control, and receives the target pronunciation of the corresponding target word based on the pronunciation control, and sends the target word and the target pronunciation as correction data to the speech synthesis server.

In the embodiment of the present invention, when receiving the correction data corresponding to the synthesized speech according to the pronunciation correction request, the user terminal first displays the pronunciation correction interface according to the received pronunciation correction request, and the pronunciation correction interface includes a character input control and a pronunciation control, wherein, The word input control is used to receive the target word to be corrected in the text to be synthesized, and the pronunciation control is used to receive the target pronunciation of the target word.

For example, please refer to Figure 2f, the pronunciation correction interface shows:

The clause set as the text to be synthesized is "Wang Baole didn't understand at the beginning";

The word input control in the form of an input box and the first prompt message "Please enter the word to be corrected" prompt the user to input the target word to be corrected, for example, the user has entered "乐" in the illustration;

The pronunciation control in the form of a selection box and the second prompt message "Please check the correct pronunciation" prompt the user to select the correct pronunciation as the target pronunciation, wherein the number of pronunciation controls is the same as the number of pronunciations of the corresponding target word obtained, as shown in Fig. The pronunciation control showing the pronunciation "le4" and the pronunciation control showing the pronunciation "yue4" in the demo;

The "report correction" control is used to indicate the completion of the input. When the user completes the input, the "report correction" control can be clicked, so that the user terminal can obtain the target word "le" and the corresponding target pronunciation "le4".

After the user terminal receives the target word that needs to be corrected input by the user based on the character input control, and the target pronunciation corresponding to the target word is received based on the pronunciation control, the target word and its corresponding target pronunciation are set as correction data and sent to the speech synthesis server.

407. The speech synthesis server updates the synthesized speech according to the correction data, obtains the updated synthesized speech, and returns it to the user terminal.

After receiving the correction data returned by the user terminal, the speech synthesis server updates the previously synthesized speech according to the correction data, obtains the updated synthesized speech, and returns it to the user terminal. ,

For example, for the text to be synthesized "Wang Baole didn't understand at the beginning", the pronunciation of "Le" in the synthesized voice is "yue4". After updating, the pronunciation of "Le" in the updated synthesized voice is "le4".

408. The user terminal replaces the currently played synthesized voice with the updated synthesized voice for playback.

Wherein, when the user terminal receives the updated synthesized voice returned by the speech synthesis server, it replaces the currently played synthesized voice with the updated synthesized voice and plays it, so as to realize the pronunciation correction of the synthesized voice.

Embodiment four,

In order to better implement the above speech synthesis playback method, an embodiment of the present invention further provides a speech synthesis playback device, and the speech synthesis playback device may specifically be integrated in a user terminal.

For example, as shown in Figure 5, the speech synthesis playback device may include a text acquisition module 501, a speech synthesis module 502, a speech playback module 503 and a text correction module 504, as follows:

The text obtaining module 501 is configured to receive a speech synthesis request, and obtain text to be synthesized that needs to be speech synthesized according to the speech synthesis request.

The speech synthesis module 502 is used for sending the text to be synthesized to the speech synthesis server for speech synthesis, so that the speech synthesis server returns the synthesized speech corresponding to the text to be synthesized;

Voice playing module 503, used for playing synthesized voice, and receiving the pronunciation correction request to synthesized voice;

The text correction module 504 is used to receive correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and returns the updated synthesized speech;

The voice playing module 503 is also used to replace the currently played synthesized voice with the updated synthesized voice for playing.

In one embodiment, after receiving correction data corresponding to the synthesized speech according to the pronunciation correction request, the text correction module 504 is used to:

Display the pronunciation correction interface according to the pronunciation correction request, and the pronunciation correction interface includes word input controls and pronunciation controls;

Receiving target words that need to be corrected in the text to be synthesized based on the word input control;

receiving the target pronunciation of the corresponding target word based on the pronunciation control;

The target word and the target pronunciation are set as correction data.

In one embodiment, when receiving the target pronunciation of the corresponding target word based on the pronunciation control, the text correction module 504 is used to:

Check whether the target word is a polyphonic word;

When it is determined that the target word is a polyphonic character, according to the preset correspondence between polyphonic characters and pronunciations, multiple pronunciations corresponding to the target word are obtained;

Display multiple pronunciations based on the pronunciation control, and receive selection operations on the displayed pronunciations;

The pronunciation corresponding to the selection operation is set as the target pronunciation of the target word.

In one embodiment, when obtaining the text to be synthesized that needs to be synthesized according to the speech synthesis request, the text acquisition module 501 is used to:

Extract the text in the display content of the foreground application according to the speech synthesis request to obtain the extracted text;

Divide the extracted text into multiple clauses according to the preset clause strategy;

Set the clause as the text to be synthesized.

In one embodiment, during the process of playing synthesized voice, the voice playing module 503 is also used for:

The clauses corresponding to the synthesized speech are identified according to preset rules.

In one embodiment, when receiving a pronunciation correction request for synthesized speech, the speech playback module 503 is used to:

Display pronunciation correction controls within the preset range of the sentence corresponding to the synthesized speech;

A pronunciation correction request for the synthesized speech is received based on the pronunciation correction control.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a data storage module for:

The text to be synthesized, the synthesized speech and/or the updated synthesized speech are stored in the distributed system.

Embodiment five,

In order to better implement the above intelligent retrieval method, an embodiment of the present invention further provides a speech synthesis playback device, which may specifically be integrated in a speech synthesis server.

For example, as shown in Figure 6, the speech synthesis playback device may include a speech synthesis module 601, a speech delivery module 602 and a speech update module 603, as follows:

Speech synthesis module 601, for when receiving the text to be synthesized from the user terminal, perform speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model, to obtain synthesized speech;

The voice sending module 602 is used to return the synthesized voice to the user terminal for playback, and receive correction data corresponding to the synthesized voice returned by the user terminal;

Voice updating module 603, for updating the synthesized voice according to the correction data, to obtain the updated synthesized voice;

The voice delivery module 602 is also used to return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a model update module, which is used for:

The speech synthesis model is updated according to the text to be synthesized and the correction data.

In one embodiment, the speech synthesis playback device provided by the embodiment of the present invention also includes a data storage module for:

The text to be synthesized, the synthesized speech and/or the updated synthesized speech are stored in the distributed system.

Embodiment six,

The embodiment of the present invention also provides a user terminal, and the user terminal may be a device such as a mobile phone, a tablet computer, or a notebook computer. As shown in FIG. 7, it shows a schematic structural diagram of a user terminal involved in an embodiment of the present invention, specifically:

The user terminal may include a processor 701 of one or more processing cores, a memory 702 of one or more computer-readable storage media, a power supply 703, an input unit 704 and other components. Those skilled in the art can understand that the structure of the user terminal shown in FIG. 7 does not constitute a limitation on the user terminal, and may include more or less components than those shown in the figure, or combine some components, or arrange different components. in:

The processor 701 is the control center of the user terminal, and uses various interfaces and lines to connect various parts of the entire user terminal, by running or executing software programs and/or modules stored in the memory 702, and calling the Data, perform various functions of the user terminal and process data.

The memory 702 can be used to store software programs and modules, and the processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702 . In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 702 may further include a memory controller to provide the processor 701 with access to the memory 702 .

The user terminal also includes a power supply 703 for supplying power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so that functions such as charging, discharging, and power consumption management can be implemented through the power management system.

The user terminal can also include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

Although not shown, the user terminal may also include a display unit, etc., which will not be repeated here. Specifically, in this embodiment, the processor 701 in the user terminal will load the executable file corresponding to the process of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the executable file stored in the The application program in memory 702, thus realizes various functions, as follows:

Receive the speech synthesis request, and obtain the text to be synthesized that needs to be synthesized according to the speech synthesis request; send the text to be synthesized to the speech synthesis server for speech synthesis, so that the speech synthesis server returns the synthesized speech corresponding to the text to be synthesized; play the synthesized speech, And receive the pronunciation correction request for the synthesized speech; receive the correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and returns the updated synthesized speech Speech; replace the currently playing synthesized voice with the updated synthesized voice for playback.

It should be noted that the user terminal provided by the embodiment of the present invention belongs to the same concept as the speech synthesis playback method applicable to the user terminal in the above embodiment, and its specific implementation process is detailed in the above method embodiment, and will not be repeated here.

Embodiment seven,

The embodiment of the present invention also provides a speech synthesis server, as shown in FIG. 8 , which shows a schematic structural diagram of the user terminal involved in the embodiment of the present invention. Specifically:

The speech synthesis server may include a processor 801 of one or more processing cores, a memory 802 of one or more computer-readable storage media, a power supply 803, an input unit 804 and other components. Those skilled in the art can understand that the speech synthesis server structure shown in Figure 8 does not constitute a limitation to the speech synthesis server, and may include more or less components than those shown in the illustration, or combine certain components, or different components layout. in:

The processor 801 is the control center of the speech synthesis server, and uses various interfaces and lines to connect the various parts of the entire speech synthesis server, by running or executing software programs and/or modules stored in the memory 802, and calling the software programs stored in the memory 802 The data within, perform various functions of the speech synthesis server and process the data.

The memory 802 can be used to store software programs and modules, and the processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802 . In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 802 may further include a memory controller to provide the processor 801 with access to the memory 802 .

The speech synthesis server also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that functions such as charging, discharging, and power consumption management can be realized through the power management system.

Specifically in this embodiment, the processor 801 in the speech synthesis server will load the executable file corresponding to the process of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the stored The application program in memory 802, thus realizes various functions, as follows:

When receiving the text to be synthesized from the user terminal, perform speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model to obtain a synthesized voice; return the synthesized voice to the user terminal for playback, and receive the corresponding synthesized voice returned by the user terminal the correction data; update the synthesized voice according to the correction data to obtain the updated synthesized voice; return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

It should be noted that the speech synthesis server provided by the embodiment of the present invention belongs to the same concept as the speech synthesis playback method applicable to the speech synthesis server in the above embodiments, and its specific implementation process is detailed in the above method embodiments, and will not be repeated here. repeat.

Embodiment eight,

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, and the instructions can be stored in a computer-readable storage medium, and loaded and executed by the processor.

To this end, the embodiment of the present invention provides a storage medium, which stores a plurality of instructions, which can be loaded by the processor of the user terminal to execute the speech synthesis playback method suitable for the user terminal provided by the embodiment of the present invention , for example, the command can perform the following steps:

Receive the speech synthesis request, and obtain the text to be synthesized that needs to be synthesized according to the speech synthesis request; send the text to be synthesized to the speech synthesis server for speech synthesis, so that the speech synthesis server returns the synthesized speech corresponding to the text to be synthesized; play the synthesized speech, And receive the pronunciation correction request for the synthesized speech; receive the correction data corresponding to the synthesized speech according to the pronunciation correction request, and send the correction data to the speech synthesis server, so that the speech synthesis server updates the synthesized speech according to the correction data, and returns the updated synthesized speech Speech; replace the currently playing synthesized voice with the updated synthesized voice for playback.

In addition, an embodiment of the present invention provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by the processor of the speech synthesis server to execute the speech synthesis playback method suitable for the server provided by the embodiment of the present invention, For example, the command can perform the following steps:

When receiving the text to be synthesized from the user terminal, perform speech synthesis on the text to be synthesized according to the pre-trained speech synthesis model to obtain a synthesized voice; return the synthesized voice to the user terminal for playback, and receive the corresponding synthesized voice returned by the user terminal the correction data; update the synthesized voice according to the correction data to obtain the updated synthesized voice; return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playback.

Wherein, the storage medium may include: a read only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and the like.

The storage medium provided by the embodiment of the present invention can achieve the beneficial effect achieved by the corresponding speech synthesis playback method provided by the embodiment of the present invention. For details, refer to the previous embodiments, and details will not be repeated here.

A speech synthesis playback method, device, and storage medium provided by the embodiments of the present invention have been described in detail above. In this paper, specific examples are used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for To help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation and application range. In summary, the content of this specification is not It should be understood as a limitation of the present invention.

Claims (10)

1. A method for synthesizing and playing speech, comprising:

receiving a voice synthesis request, and acquiring a text to be synthesized, which needs to be subjected to voice synthesis, according to the voice synthesis request;

sending the text to be synthesized to a voice synthesis server for voice synthesis, so that the voice synthesis server returns a synthesized voice corresponding to the text to be synthesized;

playing the synthesized voice and receiving a pronunciation correction request for the synthesized voice;

receiving input correction data corresponding to the synthesized voice according to the pronunciation correction request, and sending the correction data to the voice synthesis server, so that the voice synthesis server updates the synthesized voice according to the correction data and returns the updated synthesized voice;

and replacing the currently played synthesized voice with the updated synthesized voice for playing.

2. The speech synthesis playing method according to claim 1, wherein the step of receiving correction data corresponding to the synthesized speech input according to the utterance correction request includes:

displaying a pronunciation correction interface according to the pronunciation correction request, wherein the pronunciation correction interface comprises a character input control and a pronunciation control;

receiving target words needing to be corrected in the text to be synthesized based on the word input control;

receiving a target pronunciation corresponding to the target character based on the pronunciation control;

setting the target word and the target pronunciation as the correction data.

3. The speech synthesis playing method according to claim 2, wherein the step of receiving a target pronunciation corresponding to the target word based on the pronunciation control comprises:

checking whether the target character is a polyphone character;

when the target character is judged to be a polyphone character, acquiring a plurality of pronunciations corresponding to the target character according to a preset corresponding relation between the polyphone character and the pronunciations;

displaying the plurality of pronunciations based on the pronunciation control, and receiving selection operation of the displayed pronunciations;

and setting the pronunciation corresponding to the selection operation as the target pronunciation of the target character.

4. The method for playing speech synthesis according to claims 1-3, wherein the step of obtaining the text to be synthesized that needs to be speech-synthesized according to the speech synthesis request comprises:

extracting a text in the display content of the foreground application according to the voice synthesis request to obtain an extracted text;

dividing the extracted text into a plurality of clauses according to a preset clause strategy;

and setting the clauses as the texts to be synthesized.

5. The speech synthesis playing method according to any one of claims 1 to 3, wherein the speech synthesis playing method further comprises:

and storing the text to be synthesized, the synthesized voice and/or the updated synthesized voice into a distributed system.

6. A method for synthesizing and playing speech, comprising:

when a text to be synthesized from a user terminal is received, carrying out voice synthesis on the text to be synthesized according to a pre-trained voice synthesis model to obtain synthesized voice;

returning the synthesized voice to the user terminal for playing, and receiving correction data corresponding to the synthesized voice returned by the user terminal;

updating the synthesized voice according to the correction data to obtain updated synthesized voice;

and returning the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice for playing.

7. The speech synthesis playing method according to claim 6, further comprising:

and updating the voice synthesis model according to the text to be synthesized and the correction data.

8. A speech synthesis playback apparatus, comprising:

the text acquisition module is used for receiving a voice synthesis request and acquiring a text to be synthesized, which needs voice synthesis, according to the voice synthesis request;

the voice synthesis module is used for sending the text to be synthesized to a voice synthesis server for voice synthesis, so that the voice synthesis server returns the synthesized voice corresponding to the text to be synthesized;

the voice playing module is used for playing the synthesized voice and receiving a pronunciation correction request of the synthesized voice;

the text correction module is used for receiving input correction data corresponding to the synthesized voice according to the pronunciation correction request and sending the correction data to the voice synthesis server, so that the voice synthesis server updates the synthesized voice according to the correction data and returns the updated synthesized voice;

the voice playing module is further configured to replace the currently played synthesized voice with the updated synthesized voice for playing.

9. A speech synthesis playing device is characterized in that the device comprises a speech synthesis module, a speech sending module and a speech updating module, wherein,

the voice synthesis module is used for carrying out voice synthesis on the text to be synthesized according to a pre-trained voice synthesis model when receiving the text to be synthesized from the user terminal to obtain synthesized voice;

the voice issuing module is used for returning the synthesized voice to the user terminal for playing and receiving the correction data corresponding to the text to be synthesized returned by the user terminal;

the voice updating module is used for updating the synthesized voice according to the correction data to obtain the updated synthesized voice;

the voice issuing module is further configured to return the updated synthesized voice to the user terminal, so that the user terminal replaces the synthesized voice with the updated synthesized voice to play the synthesized voice.

10. A storage medium storing instructions adapted to be loaded by a processor to perform a speech synthesis playing method according to any one of claims 1 to 5 or to perform a speech synthesis playing method according to claim 6 or 7.