JP2008185805A - Technology for creating high quality synthesis voice - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently create high quality synthesis voice by connecting a plurality of phonemes. <P>SOLUTION: A system comprises: a phoneme storage section for storing a plurality of phoneme data; a synthesis section for creating a voice data which indicates synthesis voice of a text by reading and connecting a phoneme data corresponding to each phoneme, which indicates pronunciation of the input text, from the phoneme storage section; a calculation section for calculating an index value which indicates unnaturalness of the synthesis voice of the text, based on the voice data; a paraphrase storage section for storing a second notation which is paraphrasing of a first notation by relating it to each of the plurality of first notations; a replacing section for replacing the searched notation with the second notation corresponding to the first notation, by searching notation which corresponds to any of the first notation from the text; and a determination section in which the created voice data is output on condition that the calculated index value is smaller than a reference value, and in which the text is input to the synthesis section so that the voice data of the replaced text may be further created, on condition that the index value is the reference value or more. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for generating synthesized speech. In particular, the present invention relates to a technique for generating a synthesized speech by connecting a plurality of phonemes.

  Conventionally, a speech synthesis technique employing a waveform editing synthesis method has been used with the goal of generating a synthesized speech that is natural to the listener. In this method, a speech synthesizer records in advance the speech of a human speaker and stores it in a database as speech waveform data. Then, the speech synthesizer reads out a plurality of speech waveform data based on the input text and connects them to generate synthesized speech. In order for such a synthesized voice to be heard naturally by the listener, it is desirable that the frequency and tone color of the voice change continuously. For example, if the sound frequency or tone changes greatly at the connection portion of the sound waveform data, the synthesized sound will sound unnatural.

JP 2003-131679 A Wael Hamza, Raimo Bakis, and Ellen Eide, "RECONCILING PRONUNCIATION DIFFERENCES BETWEEN THE FRONTEND AND BACK-END IN THE IBM SPEECH SYNTHESIS SYSTEM", Proceedings of ICSLP, Jeju, South Korea, 2004, pp.2561-2564

  However, the types of audio waveform data that can be recorded in advance are limited due to cost and time constraints, and the storage capacity and processing capability of computers. For this reason, appropriate speech waveform data is not registered in the database, and as a result of using the substitute speech waveform data, the frequency or the like changes greatly at the connection portion, and the synthesized speech may become unnatural. This is likely to occur when the notation content of the input text is significantly different from the speech content recorded in advance to generate speech waveform data.

  Patent Document 1 and Non-Patent Document 1 are listed as reference technical documents. The voice output device described in Patent Document 1 makes it easy for the listener to understand the content by converting text composed of written words into spoken text and then reading it out. However, this apparatus converts text in order to change its expression, and the conversion is performed irrespective of information such as frequency change of speech waveform data. Therefore, the quality of the synthesized speech cannot be improved by this conversion. The technique of Non-Patent Document 1 stores phonemes having the same notation and different pronunciations in advance, and selecting appropriate phonemes so as to improve the quality of synthesized speech. However, even if such a selection is attempted, the synthesized speech becomes unnatural unless there is an appropriate phoneme segment.

  Therefore, an object of the present invention is to provide a system, a method, and a program that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

In order to solve the above-described problem, in a first embodiment of the present invention, a system for generating synthesized speech, a phoneme unit storage unit that stores a plurality of phoneme unit data each indicating speech of different phonemes, and A text synthesizing unit that inputs text, reads out and connects phoneme data corresponding to each phoneme indicating the pronunciation of the input text from the phoneme storage unit, and generates voice data indicating a synthesized voice of the text; A paraphrase memory that stores a second notation that is a paraphrase of the first notation in association with a calculation unit that calculates an index value indicating unnaturalness of speech based on the speech data and each of the plurality of first notations. Part, a replacement part that searches the text for a notation that matches any first notation, and replaces the searched notation with a second notation corresponding to the first notation, and the calculated index value Predetermined The generated voice data is output on condition that the value is smaller than the reference value, and the text is synthesized to further generate voice data for the replaced text on condition that the index value is equal to or higher than the reference value. A system including a determination unit that inputs to a unit is provided. In addition, a method for generating synthesized speech by the system and a program for causing an information processing apparatus to function as the system are provided.
The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows the overall configuration of a speech synthesis system 10 and data related thereto. The speech synthesis system 10 includes a phoneme piece storage unit 20 that stores a plurality of phoneme piece data. These phoneme piece data are generated in advance by dividing the data for each phoneme from the target speech data indicating the synthesized speech to be generated. The target audio data is, for example, a recording of a voice read out by the announcer. Then, the speech synthesis system 10 inputs text, performs processing such as application of morphological analysis and prosodic model to the input text, and the prosody and timbre of each phoneme to be generated as a read-out speech of the text. Generate data for Then, the speech synthesis system 10 selects and reads a plurality of phoneme piece data from the phoneme piece storage unit 20 based on the generated data such as frequencies, and connects these read phoneme piece data. The connected plurality of phoneme piece data is output as voice data indicating the synthesized voice of this text on condition that the user approves it.

  Here, the type of phoneme piece data that can be stored in the phoneme piece storage unit 20 is limited by constraints such as cost, required time, or calculation processing capability of the speech synthesis system 10. For this reason, even if the speech synthesis system 10 obtains a frequency to be generated as a pronunciation of each phoneme as a result of processing such as application of the prosody model, phoneme piece data of that frequency is not stored in the phoneme piece storage unit 20. There is a case. In this case, there is a possibility that synthesized speech having low quality may be generated as a result of the speech synthesis system 10 selecting inappropriate phoneme piece data. On the other hand, the speech synthesis system 10 according to the present embodiment, when the speech data once generated does not have sufficient quality, outputs the text by changing the notation of the text within a range that does not change its meaning. The purpose is to improve the quality of synthesized speech.

  FIG. 2 shows an example of the data structure of the phoneme piece storage unit 20. The phoneme piece storage unit 20 stores a plurality of phoneme piece data each representing a different phoneme sound. Specifically, the phoneme piece storage unit 20 stores, for each phoneme, notation of the phoneme, speech waveform data of the phoneme, and timbre data of the phoneme. As an example, the phoneme piece storage unit 20 stores, as speech waveform data, information indicating a change in fundamental frequency over time for a phoneme having the notation “A”. Here, the fundamental frequency of a phoneme means the frequency component with the loudest sound among the frequency components constituting the phoneme. In addition, the phoneme piece storage unit 20 uses, as timbre data, vector data indicating the magnitude or strength of speech for each of a plurality of frequency components including the fundamental frequency for a phoneme having the same notation “A”. Remember as. In FIG. 2, for convenience of explanation, the timbre data at the head part and the tail part of each phoneme is illustrated, but actually, the phoneme piece storage unit 20 is data indicating the time change of the magnitude or strength of each frequency component. Is remembered.

  Thus, since the phoneme piece storage unit 20 stores the speech waveform data of each phoneme, the speech synthesis system 10 can generate speech having a plurality of phonemes by connecting these speech waveform data. it can. FIG. 2 shows an example of the contents of the phoneme piece data, and the data structure and data format of the phoneme piece data stored in the phoneme piece storage unit 20 are not limited to those shown in this figure. As another example, for example, the phoneme piece storage unit 20 may store phoneme recording data itself as phoneme piece data, or may store data obtained by performing a predetermined calculation on the recording data. . The calculation is, for example, discrete cosine transform and the like, whereby a desired frequency component in the recorded data can be referred to, so that the fundamental frequency and timbre can be analyzed.

  FIG. 3 shows a functional configuration of the speech synthesis system 10. The speech synthesis system 10 includes a phoneme unit storage unit 20, a synthesis unit 310, a calculation unit 320, a determination unit 330, a display unit 335, a paraphrase storage unit 340, a replacement unit 350, and an output unit 370. . First, the relationship between these members and hardware resources will be described. The phoneme piece storage unit 20 and the paraphrase storage unit 340 are realized by a storage device such as a RAM 1020 and a hard disk drive 1040 described later, for example. The synthesis unit 310, the calculation unit 320, the determination unit 330, and the replacement unit 350 are realized by the operation of the CPU 1000, which will be described later, according to instructions of the installed program. The display unit 335 is realized by a pointing device or a keyboard for receiving input from the user, in addition to the graphic controller 1075 and the display device 1080 described later. 370 is realized by a speaker or an input / output chip 1070.

  The phoneme piece storage unit 20 stores a plurality of phoneme piece data as described above. The synthesizing unit 310 inputs text from the outside, reads out phoneme piece data corresponding to each phoneme indicating the pronunciation of the input text from the phoneme piece storage unit 20, and connects them. Specifically, first, the synthesis unit 310 performs a morphological analysis on the text to detect a boundary between words included in the text and a part of speech of each word. The synthesizing unit 310 should pronounce each phoneme in what frequency and in what tone when reading the text, based on data stored in advance about how to read each phrase. Ask for. The synthesizing unit 310 then reads out and connects the phoneme piece data close to the frequency and the timbre from the phoneme piece storage unit 20 and outputs them to the calculating unit 320 as voice data indicating the synthesized voice of the text.

  The calculation unit 320 calculates an index value indicating the unnaturalness of the synthesized speech of the text based on the speech data received from the synthesis unit 310. This index value is, for example, the first phoneme piece data and the second phoneme piece data at the boundary between the first phoneme piece data included in the voice data and the second phoneme piece data connected to the first phoneme piece data. It shows the difference in pronunciation between the phoneme piece data. The difference in pronunciation is the difference in timbre and fundamental frequency. That is, the greater the difference is, the more suddenly the voice frequency changes, and the synthesized voice feels unnatural to the listener.

  The determination unit 330 determines whether or not the calculated index value is smaller than a predetermined reference value. The determination unit 330 instructs the replacement unit 350 to generate voice data for the replaced text by replacing the notation in the text on condition that the index value is equal to or greater than the reference value. On the other hand, on the condition that the index value is smaller than the reference value, the determination unit 330 displays the text that is the target for generating the voice data to the user, and the synthesized voice is generated based on the text. Is displayed to inquire the user as to whether or not to generate the file. This text may be the text itself input from the outside, or may be text generated as a result of the replacement processing performed several times by the replacement unit 350.

  The determination unit 330 outputs the generated audio data to the output unit 370 on condition that the input for approval is received. In response to this, the output unit 370 generates a synthesized voice based on the voice data and outputs it to the user. On the other hand, the replacement unit 350 starts processing upon receiving an instruction from the determination unit 330 when the index value is greater than or equal to the reference value. The paraphrase storage unit 340 stores a second notation that is a paraphrase of the first notation in association with each of the plurality of first notations. When the replacement unit 350 receives an instruction from the determination unit 330, the replacement unit 350 first acquires the text that was the object of speech synthesis last time from the synthesis unit 310. Next, replacement unit 350 searches the text for a notation that matches any of the first notations. The replacement unit 350 replaces the searched notation with the second notation corresponding to the first notation on the condition that the search is performed. The text in which the notation is replaced is input to the synthesis unit 310, and voice data is further generated based on the text.

  FIG. 4 shows a functional configuration of the synthesis unit 310. The synthesis unit 310 includes a phrase storage unit 400, a phrase search unit 410, and a phoneme piece search unit 420. And the synthetic | combination part 310 produces | generates audio | speech data based on it, after producing | generating how to read the text by the method known as an n-gram model. Specifically, the phrase storage unit 400 first stores, for each of a plurality of previously registered phrases, how to read the phrase in association with the notation of the phrase. The notation is a character string that constitutes a phrase, and the reading is, for example, a symbol indicating pronunciation, a symbol indicating accent, or an accent type. The phrase storage unit 400 may store a plurality of different readings in association with the same notation. And in that case, the phrase memory | storage part 400 has further memorize | stored the probability value read by the reading about each reading.

  In detail, the phrase storage unit 400, for each combination of a predetermined number of phrases (for example, a combination of two phrases in the bi-gram model), the probability value that the phrase of the combination is read in each combination of readings. Is remembered. For example, not only memorize the probability value with the accent in the first syllable and the probability value with the accent in the second syllable for the single phrase “my”, but “my” is “near” When consecutively written as a phrase, a probability value having an accent in the first syllable and a probability value having an accent in the second syllable are stored for this combination of consecutive phrases. And apart from this, when the word “my” is written consecutively with other words that are not “near”, there is also a probability value that each syllable has an accent for this continuous word combination. Each is remembered.

  The information on the notation, how to read and the probability value stored here is generated by recognizing previously recorded target speech data and counting the frequency with which the combination of readings appears for each combination of phrases. That is, high probability values are stored for combinations of words and readings that appear frequently in the target speech data. In order to further improve the accuracy of speech synthesis, it is desirable that the phoneme segment storage unit 20 further stores information on the part of speech of the phrase. The part-of-speech information may also be generated by speech recognition of the target speech data, or may be manually added to the speech-recognized text data.

  The phrase search unit 410 searches the phrase storage unit 400 for a phrase that has the same notation as each phrase included in the input text, and reads and connects the reading corresponding to each searched phrase from the phrase storage unit 400. Generates how to read text. For example, in the bi-gram model, the phrase search unit 410 scans the input text from the top, and searches the phrase storage unit 400 for a combination of phrases that matches the phrase of each combination of two consecutive phrases. To do. Then, the phrase search unit 410 reads the combination of readings corresponding to the searched combination of phrases from the phrase storage unit 400 together with the probability value corresponding to the combination. In this way, the phrase search unit 410 searches for a plurality of probability values for each combination of phrases from the beginning to the end of the text.

  For example, when the text includes the phrases A, B, and C in this order, as to how to read the combinations of the phrases A and B, a combination of a1 and b1 (probability value p1), a combination of a2 and b1 (probability value p2), A combination of a1 and b2 (probability value p3) and a combination of a2 and b2 (probability value p4) are searched. Similarly, for the combinations of words B and C, the readings are b1 and c1 combination (probability value p5), b1 and c2 combination (probability value p6), b2 and c1 combination (probability value p7), and b2 And the combination of c2 (probability value p8) is searched. Then, the phrase search unit 410 selects a combination of readings that maximizes the product of the probability values for each combination of phrases, and outputs it to the phoneme piece search unit 420 as a text reading. In this example, p1 * p5, p1 * p7, p2 * p5, p2 * p7, p3 * p6, p3 * p8, p4 * p6, and p4 * p8 are respectively calculated and correspond to the maximum value of them. A combination of readings is output.

  Next, the phoneme segment search unit 420 obtains a target prosody and timbre for each phoneme based on the generated reading, searches the phoneme unit storage unit 20 for the nearest phoneme data, reads out the phoneme data, Audio data is generated by connecting the phoneme piece data and output to the calculation unit 320. For example, when the phoneme segment search unit 420 has an accent that continues in each syllable like LHHHLLH in the generated reading (L indicates that the accent is low and H indicates that the accent is high), The prosody of each phoneme is calculated so that the level is expressed smoothly. The prosody is represented by, for example, a change in fundamental frequency, a length of sound, and a volume. The calculation of the fundamental frequency uses a fundamental frequency model that has been statistically learned in advance from voice data recorded by the announcer. With the fundamental frequency model, the target value of the fundamental frequency of each phoneme can be determined according to the accent environment, the part of speech, the sentence length, and the like. Although an example of the process for obtaining the fundamental frequency from the accent has been described here, the process for obtaining the timbre, the duration time, and the volume from the pronunciation can also be obtained based on a rule learned statistically in advance. As described above, a technique for determining the prosody and tone color of each phoneme based on accents and pronunciations is conventionally known as a technique for predicting a prosody or a tone color, and thus a detailed description thereof will be omitted.

  FIG. 5 shows an example of the data structure of the paraphrase storage unit 340. The paraphrase storage unit 340 stores a second notation that is a paraphrase of the first notation in association with each of the plurality of first notations. Further, the paraphrase storage unit 340 stores the degree of approximation of meaning between the first notation and the second notation in association with each of the first notation and the set of the second notation corresponding thereto. For example, the paraphrase storage unit 340 stores the first notation “my” in association with the second notation “my” and associates it with the set of these notations, and the degree of approximation “65%”. Is further memorized. In this way, the degree of approximation may be expressed, for example, as a percentage, and may be input by an operator whose notation is registered in the paraphrase storage unit 340, or may be given to the user as a result of replacement processing using this paraphrase. It may be calculated based on the probability that the replacement was approved.

  When the number of expressions registered in the paraphrase storage unit 340 increases, a plurality of first expressions that are equal to each other may be stored in association with a plurality of second expressions that are different from each other. That is, as a result of comparison between the input text and the first notation in the paraphrase storage unit 340, the replacement unit 350 may match the notation in the text with each of the plurality of first notations. In this case, the replacement unit 350 replaces the notation in the text with the second notation corresponding to the first notation having the highest degree of approximation among the plurality of first notations. As described above, the degree of approximation stored in association with the notation can be used as a guideline for selecting the notation to be replaced.

  Furthermore, it is desirable that the second notation stored in the paraphrase storage unit 340 is a notation of a phrase included in the text indicating the content of the target speech data. The text indicating the content of the target voice data is, for example, the text in the case where the target voice data indicates a reading voice of the text. In addition, when the target voice data is related to free speech, the target voice data may be a text indicating the result of voice recognition of the target voice data, or the contents of the target voice data are manually recorded in the text. It may be. As a result, the replacement-destination phrase notation is used in the target speech data, and the synthesized speech output for the replaced text can be made more natural.

  In addition to this, when a plurality of second notations corresponding to the first notation in the text are searched, the replacement unit 350, for each, replaces the text replaced with the second notation and the target speech data. You may calculate the distance between the text which shows a content. The distance is a concept known as an index value indicating whether the tendency of expression of text and the tendency of contents are close, and can be calculated by an existing method. In this case, the replacement unit 350 selects the text having the shortest distance as the replaced text. Even with such a method, the voice based on the text after replacement can be made as close as possible to the target voice.

  FIG. 6 shows an example of the data structure of the phrase storage unit 400. The phrase storage unit 400 stores phrase data 600, pronunciation data 610, accent data 620, and part-of-speech data 630 in association with each other. The phrase data 600 indicates the notation of the phrase for each of a plurality of phrases. In the example of FIG. 6, the phrase data 600 includes “Osaka”, “fu”, “resident”, “no”, “how”, “ni”, “limit”, “ri”, “ma” and “su”. Each is included as a phrase. Further, the pronunciation data 610 and the accent data 620 indicate how to read the words for each of the words. The pronunciation data 610 indicates the pronunciation of the reading, and the accent data 620 includes the accent of the reading. The pronunciation is represented by a phonetic symbol using, for example, an alphabet. Accents are obtained by arranging, for each phoneme, a relative high / low relationship of high (H) or low (L) speech. Further, the accent data 620 may include an accent type that identifies a combination of relative height relationships for each phoneme by a number. Moreover, the phrase storage unit 400 may store the part of speech of each phrase as shown as the part of speech data 630. The part of speech includes an extended part of speech suitable for speech synthesis and analysis, not a part of speech with a strict grammatical meaning. For example, it may include an acronym that constitutes the end of the phrase.

  In FIG. 6, the speech waveform data generated by the phrase search unit 410 based on these data is shown in the center of the figure in contrast to these various data. That is, when the phrase search unit 410 inputs the text “limited to those living in Osaka Prefecture”, the relative height relationship (L or H) of each phoneme, and the method using the n-gram model, and The pronunciation of each phoneme (phonetic symbol using alphabets) is obtained. Then, the phoneme segment search unit 420 generates a fundamental frequency that smoothly changes so as not to be unnaturally heard by the user while reflecting the relative height relationship of each phoneme. An example of the fundamental frequency generated in this way is shown in the center of FIG. It is ideal if the fundamental frequency changes in this way, but there may be a case where it is not possible to retrieve phoneme piece data from the phoneme piece storage unit 20 whose frequency values are exactly the same, and as a result, synthesized speech sounds unnaturally. There is also a case. On the other hand, as already described, according to the speech synthesis system 10, by changing the text itself within a range that does not change the meaning, the searchable speech segment data can be used effectively, Quality can be improved.

  FIG. 7 shows the flow of processing in which synthesized speech is generated by the speech synthesis system 10. The synthesizing unit 310 inputs text from outside, reads out phoneme piece data corresponding to each phoneme indicating the pronunciation of the input text from the phoneme piece storage unit 20, and connects them (S700). Specifically, first, the synthesis unit 310 performs a morphological analysis on the text to detect a boundary between words included in the text and a part of speech of each word. Then, based on the data stored in the phrase storage unit 400 in advance on how to read each phrase, the synthesizer 310 reads each phoneme at what frequency and what frequency Find out if the sound should be pronounced. The synthesizing unit 310 then reads out and connects the phoneme piece data close to the frequency and the timbre from the phoneme piece storage unit 20 and outputs them to the calculating unit 320 as voice data indicating the synthesized voice of the text.

  The calculation unit 320 calculates an index value indicating the unnaturalness of the synthesized speech of the text based on the speech data received from the synthesis unit 310 (S710). An example is described. The index value is based on the voice dissimilarity at the connection boundary of the phoneme data, the voice of each phoneme based on how to read the text, and the voice dissimilarity between the phoneme data retrieved by the phoneme segment search unit 420. Calculated based on Hereinafter, it demonstrates in order.

(1) Dissimilarity at Connection Boundary The calculation unit 320 calculates, for each connection boundary of phoneme piece data included in audio data, a fundamental frequency dissimilarity and a timbre dissimilarity at the connection boundary. The difference between the fundamental frequencies may be a difference value between the fundamental frequencies or a change rate of the fundamental frequency. The timbre dissimilarity is a distance between a vector indicating the timbre before the boundary and a vector indicating the timbre after the boundary. For example, it may be the Euclidean distance in the cepstrum space between a vector obtained by discrete cosine transform of speech waveform data before the boundary and a vector obtained by discrete cosine transform of speech waveform data after the boundary. Then, the calculation unit 320 sums up the dissimilarity at each connection boundary.

  However, when an unvoiced consonant such as p or t is generated at the connection boundary of the phoneme piece data, the calculation unit 320 determines that the degree of difference at the connection boundary is zero. This is because the listener is unlikely to feel uncomfortable even if the timbre and the fundamental frequency change greatly before and after the unvoiced consonant. For the same reason, when the reading boundary is included in the connection boundary of the phoneme piece data, the calculation unit 320 determines that the degree of difference at the connection boundary is 0.

(2) The degree of difference between the speech based on the reading and the speech of the phoneme data is calculated based on the prosody of the phoneme data and the reading of the phoneme for each phoneme data included in the speech data. Compare the determined prosody. The prosody may be determined by speech waveform data indicating the fundamental frequency. For example, the calculation unit 320 may compare the sum or average of the frequencies of the respective audio waveform data. The difference value is calculated as the degree of difference. Instead of this, or in addition to this, the calculation unit 320 compares vector data indicating the tone color of each phoneme piece data and vector data determined based on how to read each phoneme. Then, the calculation unit 320 calculates the distance between the vector data as the degree of difference for the timbre at the head or the end of the phoneme. In addition, the calculation unit 320 may use the length of phoneme pronunciation. For example, the phrase search unit 410 calculates a desirable value as the pronunciation length of each phoneme based on how to read. On the other hand, the phoneme segment search unit 420 searches for phoneme segment data indicating a pronunciation having a length closest to this length. In this case, the calculation unit 320 calculates the difference between the lengths of these pronunciations as the degree of difference.

The calculating unit 320 may add the dissimilarities calculated as described above as it is to obtain an index value, or may add the weighted values to obtain an index value. In addition, the calculation unit 320 may input each degree of difference into a predetermined evaluation function and use the output as an index value. That is, this index value may be any value that indicates a difference in voice at the connection boundary and a difference between voice based on reading and voice based on phoneme piece data.
The determination unit 330 determines whether or not the index value calculated in this way is greater than or equal to a predetermined reference value (S720). If it is equal to or greater than the reference value (S720: YES), the replacement unit 350 compares the text with the paraphrase storage unit 340 and searches the text for a notation that matches any of the first notations (S730). Then, the replacement unit 350 replaces the retrieved notation with the second notation corresponding to the first notation.

  The replacement unit 350 may compare all the words in the text with the first notation as candidates for replacement, but some of them may be compared. Preferably, the replacement unit 350 does not replace some sentences in the text even if the first notation is searched. For example, the replacement unit 350 does not replace the notation for a sentence including at least one of a proper noun or a number, and searches for a notation that matches the first notation for a sentence that does not include either a proper noun or a number. . Since sentences containing numbers and proper nouns often require strictness of meaning, avoiding such sentences can prevent a significant change in meaning.

  The replacement unit 350 may compare only a specific part that is a candidate for replacement in the text with the first notation in order to further improve the processing efficiency. For example, the replacement unit 350 sequentially scans the text sequentially from the top, and sequentially selects combinations of a predetermined number of words that are consecutively written in the text. If the words A, B, C, D, and E are included in the text, and the predetermined number is 3, the replacement unit 350 selects ABC, BCD, and CDE in this order. Then, replacement unit 350 calculates an index value indicating the unnaturalness of the synthesized speech corresponding to the selected combination.

Specifically, for each combination of phrases, replacement unit 350 sums the voice dissimilarities at each connection boundary of phonemes included in the combination. Then, the replacement unit 350 calculates an average value of the degrees of difference per connection boundary by dividing the total value by the number of connection boundaries included in the combination. Further, the replacement unit 350 sums the dissimilarities of the synthesized speech and the speech based on how to read in each phoneme included in the combination, and divides by the number of phonemes included in the combination, thereby calculating the difference per phoneme. Find the average degree. Then, the replacement unit 350 calculates an average value of the dissimilarity per connection boundary and the sum of the average dissimilarity per phoneme as an index value. Then, the replacement unit 350 searches the paraphrase storage unit 340 for the first notation that matches the notation of the phrase for the phrase included in the combination with the largest calculated index value. For example, if the index value of BCD is the largest among ABC, BCD, and CDE, replacement unit 350 selects BCD and searches the BCD for a phrase that matches the first notation.
As a result, the vicinity of the most unnatural part can be given priority for replacement, and the replacement process can be made more efficient as a whole.

  Subsequently, the determination unit 330 inputs the text to the synthesis unit 310 and returns the process to S700 in order to further generate voice data for the replaced text. On the other hand, on condition that the index value is less than the reference value (S720: NO), the display unit 335 displays this text with the notation replaced to the user (S740). Then, the determination unit 330 determines whether or not an input to approve replacement is received for the displayed text (S750). On the condition that the input for approving the replacement is received (S750: YES), the determination unit 330 outputs voice data based on the text with the notation replaced (S770). On the other hand, on the condition that an input indicating that the replacement is not approved is received (S750: NO), the determination unit 330 outputs voice data based on the text before the replacement regardless of the size of the index value (S760). ). In response to this, the output unit 370 outputs synthesized speech.

  FIG. 8 shows a specific example of text sequentially generated in the process of generating synthesized speech by the speech synthesis system 10. Text 1 is the text "Please put on the defroster of the window beside me." Even if voice data is generated by the synthesizer 310 based on this text, the synthesized voice is unnatural and the index value is higher than a reference value (for example, 0.55). Text 2 is generated by replacing “defroster” with “defroster”. Since the index value is still higher than the reference value in the text 2, the text 3 is generated by replacing “soba” with “near”. Similarly, “my” is replaced with “my”, “Kureyo” is replaced with “sibling”, and “sibling” is replaced with “please”, and text 6 is generated. Like the last replacement, the once replaced phrase may be replaced again.

  Since the index value is still higher than the reference value in the text 6, “window” is replaced with “window”. As described above, the replacement source or the replacement destination (that is, the first notation or the second notation described above) may include a reading point. In addition, “defroster” is replaced with “defogger”. As a result, the generated text 8 has an index value less than the reference value. Therefore, the output unit 370 outputs synthesized speech based on the text 8.

  FIG. 9 shows an example of the hardware configuration of the information processing apparatus 500 that functions as the speech synthesis system 10. The information processing apparatus 500 includes a CPU peripheral unit including a CPU 1000, a RAM 1020, and a graphic controller 1075 connected to each other by a host controller 1082, a communication interface 1030, a hard disk drive 1040, and the like connected to the host controller 1082 by an input / output controller 1084. And an input / output unit having a CD-ROM drive 1060 and a legacy input / output unit having a ROM 1010 connected to an input / output controller 1084, a flexible disk drive 1050, and an input / output chip 1070.

  The host controller 1082 connects the RAM 1020 to the CPU 1000 and the graphic controller 1075 that access the RAM 1020 at a high transfer rate. The CPU 1000 operates based on programs stored in the ROM 1010 and the RAM 1020, and controls each unit. The graphic controller 1075 acquires image data generated by the CPU 1000 or the like on a frame buffer provided in the RAM 1020 and displays it on the display device 1080. Alternatively, the graphic controller 1075 may include a frame buffer that stores image data generated by the CPU 1000 or the like.

  The input / output controller 1084 connects the host controller 1082 to the communication interface 1030, the hard disk drive 1040, and the CD-ROM drive 1060, which are relatively high-speed input / output devices. The communication interface 1030 communicates with an external device via a network. The hard disk drive 1040 stores programs and data used by the information processing apparatus 500. The CD-ROM drive 1060 reads a program or data from the CD-ROM 1095 and provides it to the RAM 1020 or the hard disk drive 1040.

  The input / output controller 1084 is connected to the ROM 1010 and relatively low-speed input / output devices such as the flexible disk drive 1050 and the input / output chip 1070. The ROM 1010 stores a boot program executed by the CPU 1000 when the information processing apparatus 500 is activated, a program depending on the hardware of the information processing apparatus 500, and the like. The flexible disk drive 1050 reads a program or data from the flexible disk 1090 and provides it to the RAM 1020 or the hard disk drive 1040 via the input / output chip 1070. The input / output chip 1070 connects various input / output devices via a flexible disk 1090 and, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  A program provided to the information processing apparatus 500 is stored in a recording medium such as the flexible disk 1090, the CD-ROM 1095, or an IC card and provided by a user. The program is read from the recording medium via the input / output chip 1070 and / or the input / output controller 1084, installed in the information processing apparatus 500, and executed. The operation that the program causes the information processing apparatus 500 to perform is the same as the operation in the speech synthesis system 10 described with reference to FIGS.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1090 and the CD-ROM 1095, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the information processing apparatus 500 via the network.

  As described above, according to the speech synthesis system 10 according to the present embodiment, by changing the notation in the text sequentially within a range that does not greatly change the meaning, the text notation that makes the combination of phonemes more natural. Search and improve the quality of synthesized speech. Thereby, even if there is a limit in quality depending on acoustic processing such as phoneme coupling and frequency change, it is possible to generate higher quality speech. The sound quality is evaluated with high accuracy by using, for example, the voice dissimilarity at the phoneme connection boundary. As a result, it is possible to accurately determine whether or not to replace, and to determine the location to be replaced.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

FIG. 1 shows the overall configuration of a speech synthesis system 10 and data related thereto. FIG. 2 shows an example of the data structure of the phoneme piece storage unit 20. FIG. 3 shows a functional configuration of the speech synthesis system 10. FIG. 4 shows a functional configuration of the synthesis unit 310. FIG. 5 shows an example of the data structure of the paraphrase storage unit 340. FIG. 6 shows an example of the data structure of the phrase storage unit 400. FIG. 7 shows the flow of processing in which synthesized speech is generated by the speech synthesis system 10. FIG. 8 shows a specific example of text sequentially generated in the process of generating synthesized speech by the speech synthesis system 10. FIG. 9 shows an example of the hardware configuration of the information processing apparatus 500 that functions as the speech synthesis system 10.

Explanation of symbols

10 speech synthesis system 20 phoneme segment storage unit 310 synthesis unit 320 calculation unit 330 judgment unit 335 display unit 340 paraphrase storage unit 350 replacement unit 370 output unit 400 phrase storage unit 410 phrase search unit 420 phoneme segment search unit 500 information processing apparatus 600 Data 610 Pronunciation data 620 Accent data 630 Part of speech data

Claims (12)

A system for generating synthesized speech,
A phoneme piece storage unit for storing a plurality of phoneme piece data each representing speech of a different phoneme;
A synthesizing unit for inputting text, connecting phoneme data corresponding to each phoneme indicating the pronunciation of the input text from the phoneme storage unit, and generating speech data indicating synthesized speech of the text;
A calculation unit that calculates an index value indicating unnaturalness of the synthesized speech of the text based on the speech data;
A paraphrase storage unit that stores a second notation that is a paraphrase of the first notation in association with each of the plurality of first notations;
A replacement unit that searches the text for a notation that matches any of the first notations, and replaces the searched notation with the second notation corresponding to the first notation;
The generated text data is output on the condition that the calculated index value is smaller than a predetermined reference value, and the replaced text on the condition that the index value is not less than the reference value A determination unit that inputs the text to the synthesis unit to further generate voice data. The calculation unit includes the first phoneme piece data and the second phoneme piece data at a boundary between the first phoneme piece data included in the voice data and the second phoneme piece data connected to the first phoneme piece data. The system according to claim 1, wherein a difference in pronunciation between phoneme piece data is calculated as the index value. The phoneme storage unit stores, as the phoneme piece data, data indicating a fundamental frequency and a tone color of speech for each phoneme,
The calculation unit calculates a difference in fundamental frequency and tone color between the first phoneme piece data and the second phoneme piece data at a boundary between the first phoneme piece data and the second phoneme piece data. The system according to claim 2, wherein the system is calculated as the index value. The synthesis unit is
For each of the plurality of phrases, a phrase storage unit that stores how to read the phrase in association with the notation of the phrase;
By searching the phrase storage unit for a phrase whose notation matches with each phrase included in the input text, and reading and connecting from the phrase storage unit the reading corresponding to each searched phrase, A phrase search unit for generating readings;
The phoneme piece data closest to the prosody of each phoneme determined based on the generated reading is retrieved from the phoneme piece storage unit, and the voice data is generated by connecting the read phoneme piece data. Phoneme segment search section and
The calculation unit calculates, as the index value, a difference between a prosody of each phoneme determined based on the generated reading and a prosody indicated by phoneme piece data searched corresponding to each phoneme. The described system. The synthesis unit is
For each of the plurality of phrases, a phrase storage unit that stores how to read the phrase in association with the notation of the phrase;
By searching the phrase storage unit for a phrase whose notation matches with each phrase included in the input text, and reading and connecting from the phrase storage unit the reading corresponding to each searched phrase, A phrase search unit for generating readings;
The phoneme piece data closest to the timbre of each phoneme determined based on the generated reading is retrieved from the phoneme piece storage unit, and the voice data is generated by connecting the read phoneme piece data. Phoneme segment search section and
The calculation unit calculates, as the index value, a difference between a timbre of each phoneme determined based on the generated reading and a timbre indicated by phoneme piece data respectively retrieved corresponding to the phoneme. The described system. The phoneme storage unit acquires in advance target speech data that is speech data indicating a synthesized speech to be generated and generates a plurality of phoneme segment data indicating speech of a plurality of phonemes included in the target speech data. Generated and stored in advance,
The paraphrase storage unit stores, as each of the plurality of second notations, a notation of a phrase included in the text indicating the content of the target speech data,
The replacement unit replaces a notation that matches the first notation in the input text with a second notation that is a notation of a phrase included in the text indicating the content of the target speech data. system. The replacement unit calculates an index value indicating the unnaturalness of the synthesized speech corresponding to each combination of a predetermined number of words written in succession in the input text. The wording included in the combination having the largest calculated index value is searched for the first notation that matches the notation of the word from the paraphrase storage unit, and the notation of the word is replaced with the second notation. The system according to 1. The paraphrase storage unit further stores a degree of approximation of meaning between the first notation and the second notation in association with each of the first notation and the second notation set which is a paraphrase of the first notation. And
The replacement unit replaces the matching notation with the highest degree of approximation among the plurality of first notations on the condition that the notation in the input text matches each of the plurality of first notations. The system according to claim 1, wherein replacement is performed by a second notation corresponding to one notation. The replacement unit does not replace the notation for a sentence that includes at least one of a proper noun or a numeral in the input text, and matches the first notation for a sentence that does not include either a proper noun or a numeral. The system according to claim 1, wherein the notation to be searched is replaced with a second notation corresponding to the first notation. Further comprising a display unit for displaying to the user the text whose notation has been replaced on the condition that the notation has been replaced by the replacing unit;
The determination unit outputs voice data based on the text in which the notation is replaced, and further receives an input that does not approve the replacement, on condition that the input to approve replacement is received for the displayed text. The system according to claim 1, wherein audio data based on the text before replacement is output regardless of the index value. A method for generating synthesized speech,
Storing a plurality of phoneme pieces data each representing a different phoneme speech;
Inputting text, reading and connecting from the phoneme piece data storing the phoneme piece data corresponding to each phoneme indicating the pronunciation of the input text, and generating voice data showing the synthesized voice of the text;
Calculating an index value indicating unnaturalness of the synthesized speech of the text based on the speech data;
Storing a second notation that is a paraphrase of the first notation in association with each of the plurality of first notations;
Searching the text for a notation that matches any of the first notations, and replacing the searched notation with the second notation corresponding to the first notation;
The generated text data is output on the condition that the calculated index value is smaller than a predetermined reference value, and the replaced text on the condition that the index value is equal to or greater than the reference value Generating further synthesized speech of the text to further generate speech data. A program for causing an information processing device to function as a system for generating synthesized speech,
The information processing apparatus;
A phoneme piece storage unit for storing a plurality of phoneme piece data each representing speech of a different phoneme;
A synthesizing unit for inputting text, connecting phoneme data corresponding to each phoneme indicating the pronunciation of the input text from the phoneme storage unit, and generating speech data indicating synthesized speech of the text;
A calculation unit that calculates an index value indicating unnaturalness of the synthesized speech of the text based on the speech data;
A paraphrase storage unit that stores a second notation that is a paraphrase of the first notation in association with each of the plurality of first notations;
A replacement unit that searches the text for a notation that matches any of the first notations, and replaces the searched notation with the second notation corresponding to the first notation;
The generated text data is output on the condition that the calculated index value is smaller than a predetermined reference value, and the replaced text on the condition that the index value is equal to or greater than the reference value A program that functions as a determination unit that inputs the text to the synthesis unit to further generate voice data.

Images