JP4551803B2 - Speech synthesizer and program thereof - Google Patents

Abstract

A speech synthesis system in a preferred embodiment includes a speech unit storage section, a phonetic environment storage section, a phonetic sequence/prosodic information input section, a plural-speech-unit selection section, a fused-speech-unit sequence generation section, and a fused-speech-unit modification/concatenation section. By fusing a plurality of selected speech units in the fused speech unit sequence generation section, a fused speech unit is generated. In the fused speech unit sequence generation section, the average power information is calculated for a plurality of selected M speech units, N speech units are fused together, and the power information of the fused speech unit is so corrected as to be equalized with the average power information of the M speech units.

Description

  The present invention relates to a speech synthesis apparatus and method for text-to-speech synthesis, and more particularly to a speech synthesis apparatus and method for generating a speech signal from a phoneme sequence, prosodic information such as a fundamental frequency and a phoneme duration.

  Artificially creating speech signals from arbitrary sentences is called “text-to-speech synthesis”. Text-to-speech synthesis is generally performed in three stages: a language processing unit, a prosody processing unit, and a speech synthesis unit.

  The input text is first subjected to morphological analysis and syntactic analysis in the language processing unit, and then subjected to accent and intonation processing in the prosody processing unit, and phoneme sequence / prosodic information (basic frequency, phoneme duration length) Etc.) is output. Finally, the speech waveform is synthesized from the phoneme sequence / prosodic information by the speech signal synthesis unit.

  As one of speech synthesis methods, there is a unit selection type speech synthesis method that selects and synthesizes a speech unit sequence from a large number of speech units with the target phoneme sequence / prosodic information as a target. In the unit selection type speech synthesis, a speech unit is selected from a large number of speech units stored in advance based on input phoneme series / prosodic information. As a unit selection method, there is a method in which the degree of distortion of synthesized speech generated by synthesizing speech is defined as a cost function and a unit sequence is selected so as to reduce the cost. For example, the target distortion that represents the difference between the target speech and each speech segment, such as the prosody and phonological environment, and the connection distortion generated by connecting speech segments are quantified as costs, and speech synthesis is performed based on this cost. A method is used in which a speech unit sequence to be used is selected, and a synthesized speech is generated based on the selected speech unit sequence. In the unit selection type speech synthesis, by selecting an appropriate speech unit sequence from a large number of speech units, it is possible to obtain a synthesized speech in which deterioration of sound quality in the editing and connection of the unit is suppressed.

  In addition, for each synthesis unit obtained by dividing the input phoneme sequence with the target phoneme sequence / prosodic information as a target, a plurality of speech segments are selected and selected based on the degree of distortion of the synthesized speech There is a multi-unit selection type speech synthesis method in which a new speech unit is generated by fusing a plurality of speech units and the speech is synthesized by connecting them (see Non-Patent Document 1).

  As the fusion method, for example, a method of averaging pitch waveforms is used. This reduces sound quality degradation caused by mismatch between the target phoneme sequence / prosodic information and the selected speech unit sequence, and sound quality degradation caused by unit connection mismatch. The stability of the synthesized speech can be improved while maintaining the feeling.

Further, as a method for performing power control of synthesized speech, there is a speech synthesis method that divides a speech unit at a phoneme boundary, estimates power for each partial unit, and changes the power of the speech unit based on the estimated power. It is disclosed (see Patent Document 1). In the power estimation step, power is generated using parameters prepared in advance, such as coefficients of quantification class I.
JP 2001-282276 A Tatsuya Mizutani, Takehiko Tsujishima, “Speech synthesis by multiple unit selection and fusion method”, Proceedings of the Acoustical Society of Japan Spring Meeting, March 2004, 1-7-3, pp. 217-218

  In a unit selection type speech synthesizer, a speech unit selected by a cost function is selected from a large number of speech units, but its power is not always appropriate. For this reason, discontinuity of power is perceived, which has been a factor of reducing the quality of synthesized speech. In addition, in the multi-unit selection type speech synthesizer, if the number of units to be fused is increased, the power of synthesized speech becomes stable, but a fused speech unit is created from many speech units having different sound quality characteristics. As a result, sound quality distortion increases. Further, in the process of unit fusion, sound quality may be deteriorated by using speech units that are significantly different from appropriate powers for fusion.

  With a speech synthesis method that has a power estimation process and performs power control using parameters prepared in advance, it is difficult to perform power control that appropriately reflects the power information of a large number of speech segments. Mismatches with speech segments can occur.

  In view of the above problems, the present invention appropriately reflects power information of a large-scale speech unit in speech selection of a unit selection type or a multiple unit selection type, and the power of the speech unit in each speech section. An object of the present invention is to provide a speech synthesizer capable of realizing high-quality speech synthesis that is natural and stable, and a method thereof.

The invention according to claim 1 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech segments for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects N and M (N <M) speech segments, respectively, and the representative speech element In the segment generation means, an average value of power information is obtained from the selected M speech units, a fused speech unit is created by fusing the selected N speech units, and the fusion The speech synthesizer is characterized in that the representative speech element is generated by correcting the power information of the speech element so as to be an average value of the power information obtained from the M speech elements.
The invention according to claim 2 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech units for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects N and M (N <M) speech segments, respectively, and the representative speech element In the segment generation means, an average value of power information is obtained from the selected M speech units, and the power information of each of the selected N speech units becomes the average value of the power information. And the representative speech unit is generated by fusing the corrected N speech units.
The invention according to claim 3 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech units for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects N and M (N <M) speech segments, respectively, and the representative speech element In the segment generation means, an average value of power information of the selected M speech units is obtained, an average value of power information of the selected N speech units is obtained, and the N speech units are obtained. A correction value for correcting the average value of the power information of the pieces to be the average value of the power information of the M speech units is obtained, and the correction value is applied to obtain the correction value of the N speech units. The speech synthesizer is characterized in that the representative speech segment is generated by correcting each of the corrected speech segments and fusing the corrected N speech segments.
The invention according to claim 4 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech segments for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects N and M (N <M) speech segments, respectively, and the representative speech element In the segment generation means, a statistic of power information is obtained from the selected M speech units, and power information of the selected N speech units is obtained, respectively. And determining the weight of each of the N speech units based on the power information of the N speech units, and merging the N speech units based on the weight to obtain the representative speech unit. A speech synthesizer characterized by generating the speech synthesizer.
The invention according to claim 5 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech units for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects N and M (N <M) speech segments, respectively, and the representative speech element In the segment generation means, a segment in which the distribution of the power information is a predetermined probability or more from a statistic of power information of the selected M speech segments, or a segment based on the quartile range of the power information The power information of the selected N speech units is obtained, and when the power information of the N speech units is out of the section, it is excluded from the speech units to be selected , and the selection is performed. The speech synthesizer is characterized in that the representative speech unit is generated by fusing speech units within the section among the N speech units that have been generated.
The invention according to claim 6 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech units for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, and the selection means selects M speech segments and an optimal speech segment with a low degree of distortion of the synthesized speech In the representative speech element generation means, an average value of power information is obtained from the selected M speech elements, and the power information of the optimal speech element becomes the average value of the power information. The speech synthesizer is characterized in that the representative speech segment is generated by correcting as described above.
The invention according to claim 7 divides a phoneme sequence obtained from input text into predetermined synthesis units, obtains representative speech units for each synthesis unit, and connects these representative speech units to generate synthesized speech. And a degree of distortion of the synthesized speech with respect to each synthesis unit of a phoneme sequence obtained from the input text, and storage means for storing a plurality of speech segments corresponding to the synthesis unit A selection unit that selects a plurality of speech units from the speech units stored in the storage unit, and obtains a statistic of power information from the selected plurality of speech units, and the statistics of the power information Representative speech unit generation means for correcting the power information so as to improve the quality of the synthesized speech based on the amount and generating a representative speech unit corresponding to the synthesis unit, and the generated proxy Speech waveform generation means for generating a speech waveform by connecting speech segments, wherein the selection means selects M speech segments, and the representative speech segment generation means selects the selected From the statistic of the power information of the M speech units, a section where the distribution of the power information is a predetermined probability or more, or a section based on the quartile range of the power information is obtained, and the section of the power information The speech synthesizer is characterized in that the representative speech segment is generated by selecting an optimal speech segment with a low degree of distortion of the synthesized speech from speech segments having power information included.

  According to the present invention, the power of synthesized speech can be stabilized in both the unit selection type and the multiple unit selection fusion type. Compared with the method of estimating and controlling power in advance, a large-scale speech unit is selected because multiple units are selected from a large-scale speech unit based on a cost function and average power is generated. It is possible to obtain synthesized speech that appropriately reflects the power information.

  Moreover, weighting at the time of unit fusion based on power information or removal of defective units can be performed, and sound quality can be improved. As a result, the sound becomes stable while maintaining the sound quality, and a more natural synthesized speech can be obtained.

  In an embodiment of the present invention, a phoneme sequence obtained from input text is divided into predetermined synthesis units, representative speech segments are obtained for each synthesis unit, and synthesized speech is obtained by connecting these representative speech segments. In the generated speech synthesizer, a storage means for storing a plurality of speech segments corresponding to the synthesis unit and a degree of distortion of the synthesized speech for each synthesis unit of a phoneme sequence obtained from the input text Based on the selection unit for selecting a plurality of speech units from the speech unit stored in the storage unit, and obtaining statistics of power information from the plurality of speech units, and based on the statistics of the power information Representative speech segment generation means for correcting the power information so as to improve the quality of the synthesized speech and generating a representative speech segment corresponding to the synthesis unit; and the generated representative speech segment And a speech waveform generation means for generating a speech waveform by connecting. With this configuration, when generating synthesized speech, the speech unit is corrected using the statistic of the power information of a plurality of speech units selected from the speech unit group in each speech segment. A synthesized speech that appropriately reflects the power information of the speech unit can be obtained.

  The selecting means selects N and M (N = <M) speech segments, respectively, and the representative speech segment generating means selects power information from the selected M speech segments. , The fused N speech units are created by fusing the selected N speech units, and the power information of the fused speech units is the power obtained from the M speech units. The representative speech segment is generated by correcting it to be an average value of information. With this configuration, in the multiple unit selection / fusion type speech synthesis method, the number of speech units used for unit fusion is limited to N to maintain sound quality, and an average of more M speech units. By correcting using the power, the power of the fused speech unit is stabilized and a natural synthesized speech can be obtained.

  The selection unit selects M speech units and the optimum speech unit, and the representative speech unit generation unit selects an average value of power information from the selected M speech units. And the representative speech unit is generated by correcting the speech unit so that the power information of the optimum speech unit becomes an average value of the power information obtained from the M speech units. With this configuration, in the unit selection type speech synthesis method, the optimum speech unit selected using the average power of a plurality of M speech units is corrected, and the corrected speech is connected, so that high sound quality is maintained. The power of synthesized speech is stabilized.

  The selection unit selects N and M (N = <M) speech units, and the representative speech unit generation unit generates power information from the selected M speech units. A statistic is obtained, power information of each of the selected N speech units is obtained, and a weight of each of the N speech units is determined based on a statistic of power information obtained from the M speech units. And a representative speech unit is generated by fusing N speech units based on the weights. With such a configuration, in the multiple unit selection / fusion type speech synthesis method, the power of each speech unit used for unit fusion is farther from the average power of more M speech units, so that the weight at the time of fusion is higher. Get smaller. As a result, the quality of the fused speech unit is improved and a high-quality synthesized speech can be obtained.

  The selection means selects N and M (N = <M) speech segments, and the representative speech segment generation means selects power information of the selected M speech segments. A section is obtained from statistics, power information of each of the selected N speech units is obtained, and when power information of each of the N speech units is out of the section, it is excluded from the selected unit as an outlier. The representative speech unit is generated by fusing speech units excluding the outlier. With this configuration, in the multiple unit selection / fusion type speech synthesis method, defective units in which the power of each speech unit used for unit fusion deviates greatly from the average power of more M speech units are removed. In other words, by performing the unit fusion by removing the defective unit, the quality of the fused speech unit is improved, and a high-quality synthesized speech can be obtained.

  Also, only when the power information of the fused speech unit obtained by fusing N speech units is larger than the average value of the power information obtained from a plurality of M speech units, the power information of the fused speech unit is The fusion speech unit is corrected so that the average value of the power information is obtained. With this configuration, correction is made only in the direction in which the power information is reduced, so that even if the fusion speech unit contains a noise component, there is no possibility of amplifying it, and sound quality resulting from power correction is eliminated. Degradation can be avoided.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
A text-to-speech synthesizer according to a first embodiment will be described.

[1] Configuration of Text-to-Speech Synthesizer FIG. 1 is a block diagram showing a configuration of a text-to-speech synthesizer according to the first embodiment of the present invention.

  This text-to-speech synthesizer includes a text input unit 11, a language processing unit 12, a prosody processing unit 13, a speech synthesis unit 14, and a speech waveform output unit 15.

  The language processing unit 12 performs morphological analysis / syntax analysis of the text input from the text input unit 11 and sends the result to the prosody processing unit 13.

  The prosody processing unit 13 performs accent and intonation processing from the language analysis result, generates a phoneme sequence (phoneme symbol string) and prosody information, and sends them to the speech synthesis unit 14.

  The speech synthesizer 14 generates a speech waveform from the phoneme sequence and prosodic information. The speech waveform generated in this way is output from the speech waveform output unit 15.

[2] Configuration of Speech Synthesizer 14 FIG. 2 is a block diagram showing a configuration example of the speech synthesizer 14 in FIG.

  In FIG. 2, the speech synthesis unit 14 includes a speech unit storage unit 21, a phoneme environment storage unit 22, a phoneme sequence / prosodic information input unit 23, a multiple speech unit selection unit 24, a fusion speech unit sequence creation unit 25, a fusion The speech unit editing / connecting unit 26 is configured.

[2-1] Speech unit storage unit 21
Speech units are stored in the speech unit storage unit 21, and information on the phoneme environment (phoneme environment information) is stored in the phoneme environment storage unit 22.

  The speech unit storage unit 21 stores speech units in units of speech (synthesis unit) used when generating synthesized speech. A synthesis unit is a phoneme or a combination of phonemes, for example, semiphones, phonemes (C, V), diphones (CV, VC, VV), triphones (CVC, VCV), syllables (CV, V). (V represents a vowel and C represents a consonant), and these may be mixed lengths.

  The phoneme environment of a speech unit is information corresponding to a factor that is an environment for the speech unit. Factors include, for example, the phoneme name of the speech unit, the preceding phoneme, the subsequent phoneme, the subsequent phoneme, the fundamental frequency, the phoneme duration, the presence or absence of stress, the position from the accent core, the time from breathing, the utterance speed, There are emotions.

[2-2] Phoneme sequence / prosodic information input unit 23
The phoneme sequence / prosodic information input unit 23 receives the phoneme sequence and prosodic information corresponding to the input text output from the prosody processing unit 13. The prosodic information input to the phoneme sequence / prosodic information input unit 23 includes a fundamental frequency, a phoneme duration, and the like.

  Hereinafter, the phoneme sequence and the prosody information input to the phoneme sequence / prosodic information input unit 23 are referred to as “input phoneme sequence” and “input prosody information”, respectively. The input phoneme sequence is a sequence of phoneme symbols, for example.

[2-3] Multiple speech unit selection unit 24
The multiple speech segment selection unit 24 estimates the degree of distortion of the synthesized speech based on the input prosodic information and the prosodic information included in the phoneme environment of the fused speech segment for each synthesis unit of the input phoneme sequence, Based on the degree of distortion of the synthesized speech, a plurality of M speech units and a fusion unit for obtaining average power information are obtained from speech units stored in the speech unit storage unit 21. A plurality of N speech units (N = <M) are selected.

  Here, the degree of distortion of the synthesized speech is a distortion based on the difference between the phoneme environment of the speech unit stored in the phoneme unit storage unit 21 and the target phoneme environment sent from the phoneme sequence / prosodic information input unit 23. It is obtained as a weighted sum of a certain target cost and a connection cost that is a distortion based on a difference in phoneme environment between connected speech elements.

  That is, the target cost is distortion caused by using the speech unit stored in the speech unit storage unit 21 under the target text environment of the input text, and the connection cost is the connection cost. This is distortion caused by the discontinuity of the fragment environment of the phoneme conversion. In the present embodiment, a cost function described later is used as the degree of distortion of the synthesized speech.

[2-4] Fusion speech unit sequence creation unit 25
Next, the fused speech element sequence creation unit 25 creates a fused speech element by fusing the selected plurality of elements. Fusion of speech segments can be created, for example, by averaging pitch waveforms as will be described later. In this fused speech unit sequence creation unit 25, average power information of a plurality of selected M speech units is obtained, N speech units are fused, and power information of the generated fused speech unit is obtained. Correction is performed so that the average power information of the M speech segments is obtained. As a result, a sequence of fused speech segments in which power information corresponding to the phoneme symbol sequence of the input phoneme sequence is corrected is obtained. The sequence of fused speech units is transformed and connected based on the input prosodic information in the fused speech unit editing / connecting unit 26 to generate a speech waveform of synthesized speech. The speech waveform generated in this way is output from the speech waveform output unit 15.

  The “power information” is the mean square value or the mean absolute amplitude value of the speech waveform.

[3] Each process of the speech synthesizer 14 Hereinafter, each process of the speech synthesizer 14 will be described in detail.

  Here, it is assumed that the speech unit of the synthesis unit is a phoneme.

[3-1] Speech unit storage unit 21
As shown in FIG. 3, the speech unit storage unit 21 stores a speech waveform of a speech signal of each phoneme together with a speech unit number for identifying the phoneme. In addition, as shown in FIG. 4, the phoneme environment storage unit 22 stores the phoneme environment information of each speech unit stored in the speech unit storage unit 21 in association with the unit number of the phoneme. ing. Here, a phoneme symbol (phoneme name), a fundamental frequency, a phoneme duration, and a connection boundary cepstrum are stored as the phoneme environment.

  Here, although the speech unit is a phoneme unit, the same applies to a semiphoneme, a diphone, a triphone, a syllable, or a combination or variable length thereof.

  Each speech unit stored in the speech unit storage unit 21 is labeled for each phoneme with respect to a large number of separately collected speech data, and a speech waveform cut out for each phoneme is stored as a speech unit. It is a thing. For example, FIG. 5 shows the result of labeling the audio data 51 for each phoneme. In FIG. 5, phoneme symbols are assigned as the label data 53 for the speech data (speech waveform) of each phoneme divided by the label boundary 52. Note that phoneme environment information (eg, phoneme (in this case, phoneme name (phoneme symbol)), fundamental frequency, phoneme duration, etc.) for each phoneme is also extracted from the speech data. In this way, the same unit number is given to each speech waveform obtained from the speech data 51 and the information of the phoneme environment corresponding to the speech waveform, and as shown in FIG. 3 and FIG. The information is stored in the storage unit 21 and the phoneme environment storage unit 22, respectively. Here, the phoneme environment information includes the phoneme of the speech unit, its fundamental frequency, and the phoneme duration.

[3-2] Multiple speech unit selection unit 24
Next, the multiple speech element selection unit 24 will be described.

[3-2-1] Cost Function First, a cost function used when the multiple speech unit selection unit 24 obtains a unit sequence will be described.

Sub cost functions C _n (u _i , u _i−1 , t _i ) (n: 1,..., N, N are sub costs for each factor of distortion generated when speech units are deformed and connected to generate synthesized speech Number of functions). Here, t _i is a portion corresponding to the i-th segment when the target speech (target speech) corresponding to the input phoneme sequence and the input prosodic information is t = (t ₁ ,..., T _I ). The target phoneme environment information of the speech unit is represented, and u _i represents the speech unit having the same phoneme as t _i among the speech units stored in the speech unit storage unit 21.

  The sub-cost function is used to calculate a cost for estimating the degree of distortion of the synthesized speech with respect to the target speech that occurs when the synthesized speech is generated using the speech units stored in the speech unit storage unit 21. Is.

  In order to calculate the cost, a “target cost” that estimates the degree of distortion of the synthesized speech generated by using the speech unit with respect to the target speech, and the speech unit is connected to another speech unit There are two types of sub-costs called “connection costs” for estimating the degree of distortion of the synthesized speech with respect to the target speech.

  The target cost includes a basic frequency cost representing the difference (difference) between the basic frequency of the speech unit stored in the speech unit storage unit 21 and the target basic frequency, the phoneme duration length of the speech unit, and the target The phoneme duration time cost representing the difference (difference) from the phoneme duration is used.

As the connection cost, a spectrum connection cost representing a spectrum difference (difference) at the connection boundary is used. Specifically, the fundamental frequency cost is

Calculate from Here, v _i is the phonetic environment of the speech unit u _i stored in the voice unit storage 21, f represents a function to extract the average fundamental frequency from phonetic environment v _i. Also, the long phoneme duration cost is

Calculate from Here, g represents the function to extract phoneme duration from the phonetic environment v _i. Spectral connection cost is the cepstrum distance between two speech segments:

Calculate from Here, h represents a function for taking out a cepstrum coefficient of a connection boundary of the speech unit u _i as a vector. Define the weighted sum of these subcost functions as the composite unit cost function:

Here, w _n represents the weight of the sub cost function. In the present embodiment, for the sake of simplicity, all w _n is set to "1". The above formula (4) is the synthesis unit cost of the speech unit when a speech unit is applied to a synthesis unit.

For each of a plurality of segments obtained by dividing the input phoneme sequence by synthesis unit, the result of calculating the synthesis unit cost from the above equation (4) is the sum of all segments is called the cost. A cost function for calculation is defined as shown in the following equation (5):

[3-2-2] Unit Selection Processing The multiple speech unit selection unit 24 uses the cost functions shown in the above formulas (1) to (5) in two steps per segment (that is, per synthesis unit). ) Select multiple speech segments.

  FIG. 6 is a flowchart for explaining the segment selection process.

  First, as a first-stage unit selection, in step S61, the voice having the smallest cost value calculated by the above equation (5) from the speech unit group stored in the speech unit storage unit 21 is used. Find a sequence of segments. A combination of speech units that minimizes the cost is called an optimal unit sequence. That is, each speech unit in the optimal speech unit sequence corresponds to each of a plurality of segments obtained by dividing the input phoneme sequence by synthesis unit, and is calculated from each speech unit in the optimal speech unit sequence. The cost value calculated from the synthesis unit cost and the equation (5) is smaller than any other speech unit sequence. Note that the search for the optimum unit sequence can be performed more efficiently by using dynamic programming (DP).

  Next, proceeding to step S62, in the second stage segment selection, a plurality of speech segments are selected per segment using the optimum segment sequence. Here, it is assumed that the number of segments is J, M speech units for obtaining average power information per segment, and N speech units for use in unit fusion are selected, and step S62 is performed. Details will be described.

[3-2-3] Method of selecting a plurality of speech segments per segment In steps S621 to S623, one of the J segments is set as a target segment. Steps S621 to S623 are repeated J times, and processing is performed so that J segments become the target segment once.

  First, in step S621, the speech unit of the optimal unit sequence is fixed to each segment other than the segment of interest. In this state, the speech units stored in the speech unit storage unit 21 are ranked with respect to the segment of interest according to the cost value of Equation (5), and the top M pieces for obtaining average power information A speech unit and the top N speech units to be used for unit fusion are selected.

  For example, as shown in FIG. 7, it is assumed that the input phoneme sequence is “ts · i · i · s · a ·. In this case, the synthesis unit corresponds to each of phonemes “ts”, “i”, “i”, “s”, “a”,..., And each of these phonemes corresponds to one segment. FIG. 7 shows a case where a segment corresponding to the third phoneme “i” in the input phoneme sequence is set as a target segment, and a plurality of speech segments are obtained for this target segment. For the segments other than the segment corresponding to the third phoneme “i”, the speech units 71a, 71b, 71d, 71e,.

In this state, among the speech elements stored in the speech element storage unit 21, for each speech element having the same phoneme name (phoneme symbol) as the phoneme “i” of the segment of interest, Equation (5) is obtained. To calculate the cost. However, when the cost is calculated for each speech unit, the value changes for the target cost of the target segment, the connection cost between the target segment and the previous segment, the target segment and the next segment. Since these are the connection costs with the segments, only these costs need be considered. That is,
(Procedure 1) Among the speech elements stored in the speech element storage unit 21, one of the speech elements having the same phoneme name (phoneme symbol) as the phoneme “i” of the segment of interest is selected as the speech element. and u _3. From the fundamental frequency f (v ₃ ) of the speech unit u ₃ and the target fundamental frequency f (t ₃ ), the fundamental frequency cost is calculated using Equation (1).

(Procedure 2) The phoneme duration cost is calculated from the phoneme duration g (v ₃ ) of the speech unit u ₃ and the target phoneme duration g (t ₃ ) using Equation (2). To do.

And (Step 3) cepstral coefficients of the speech unit _{u 3} h _{(u 3),} from a speech unit 51b cepstral coefficients _{(u 2)} h _{(u 2),} using equation (3), first spectrum Calculate the connection cost. Further, the cepstral coefficients of the speech unit _{u 3} h _{(u 3),} from the cepstral coefficients of the speech unit _{51d (u 4) h (u} 4), using equation (3), the second spectral concatenation cost Is calculated.

(Procedure 4) Calculate the weighted sum of the fundamental frequency cost, the phoneme duration time cost, and the first and second spectrum connection costs calculated by using each sub-cost function in (Procedure 1) to (Procedure 3). The cost of the speech unit u ₃ is calculated.

    (Procedure 5) Among the speech elements stored in the speech element storage unit 21, for each speech element having the same phoneme name (phoneme symbol) as the phoneme "i" of the segment of interest, the above (Procedure 1) to When the cost is calculated according to (Procedure 4), ranking is performed so that the speech unit having the smallest value has a higher rank (Step S621 in FIG. 6). For example, in FIG. 7, the speech unit 72a has the highest rank, and the speech unit 72e has the lowest rank. Then, the top M speech units from 72a to 72d for obtaining average power information are selected (step S622 in FIG. 6), and the top N speech units from 72a to 72c for fusing speech units are selected. A speech segment (N = <M) is selected (step S623 in FIG. 6).

  The above (Procedure 1) to (Procedure 5) are performed for each segment. As a result, M and N speech segments are obtained for each segment.

[3-3] Fusion speech unit creation unit 25
Next, the fusion speech unit creation unit 25 will be described.

  The fused speech element creating unit 25 fuses a plurality of speech elements selected by the multiple speech element selecting unit 24 to create a fused speech element.

[3-3-1] Process of Fusion Speech Unit Creation Unit 25 FIG.

First, in step S81, average power information of the selected M speech units is obtained. Average power information P _i for each voice

The average value P _ave of the power information P _i (1 = <i = <M) of each obtained piece

To obtain the average power information of M speech segments. In the equation, s _i (n) represents the audio signal of the i-th speech element, and T represents the number of samples.

  Next, in step S82, N speech segments are fused using a fusion method described later. N speech units selected by the multiple segment selection unit 24 are acquired from the speech unit storage unit 21, and the N speech units are merged to create a new speech unit (fused speech unit). Is generated.

Finally, in step S83, the power information of the fused speech unit is corrected to the average power information P _ave . The power information P _f of the fusion speech unit is obtained by the equation (6), and the ratio r for correcting the power information is calculated.

  Ask for. The power information is corrected by multiplying the unit r by the ratio r.

For simplicity, the power information P _f of the fusion speech unit may be an average value of the power information P _i (1 = <I = <N) of the N speech units.

[3-3-2] Power Information Correction FIG. 9 shows an example of power information correction. The table in FIG. 9 represents the power information P _i (1 = <i = <M) of the upper M speech units selected in the right unit of phoneme i when M = 15. . In this example, semitones are used as a synthesis unit. When N = 3, power information P _f = 2691671 of the fusion speech unit, M average power information P _ave = 1647084, and the ratio for correcting the power information is r = 0.78. The power information is corrected by multiplying r by the speech waveform of the fused speech unit.

  FIG. 10 shows an example of a waveform obtained by correcting the power information. FIG. 10 shows the phoneme i at the beginning of the sentence. FIG. 10A shows a case where the fusion speech unit is connected as it is without correction, and FIG. 10B shows a case where the power information correction according to the present invention is performed. The numbers on the horizontal axis represent pitch mark numbers. In FIG. 10 (a), the power information suddenly increases from the connection part pitch mark numbers 9 to 10 of the left half phoneme and the right half phoneme of the phoneme i. On the other hand, in FIG. 10B, the left semiphone is r = 1.28 and the right semiphone is r = 0.78, and it can be seen that the connection is smoothly connected. Note that the right semiphoneme corresponds to FIG.

[3-3-3] Speech Unit Fusion Method Next, the speech unit fusion method in step S82 will be described. This step performs different processing depending on whether the speech segment is a voiced sound or an unvoiced sound.

[3-3-3-1] Case of Voiced Sound First, the case of voiced sound will be described. In the case of voiced sound, a pitch waveform is extracted from the speech segment and fused at the level of the pitch waveform to create a new pitch waveform. A pitch waveform is a relatively short waveform that has a length up to several times the fundamental period of speech and does not have a fundamental period, and its spectrum represents the spectrum envelope of the speech signal. .

  The extraction method includes a method of simply cutting out with a fundamental period synchronization window, a method of performing inverse discrete Fourier transform on the power information spectrum envelope obtained by cepstrum analysis and PSE analysis, and a pitch waveform by the impulse response of the filter obtained by linear prediction analysis. There are various methods, such as a method for obtaining a pitch waveform that reduces distortion with respect to natural speech at the level of synthesized speech by a closed loop learning method.

  Here, a case where a pitch waveform is extracted using a method of cutting out with a basic period synchronization window will be described as an example with reference to the flowchart of FIG. A processing procedure in the case of generating one new speech unit by fusing the N speech units selected by the multiple speech unit selection unit 24 will be described.

  In step S111, marks (pitch marks) are added to the respective speech waveforms of the N speech units for each periodic interval. FIG. 12A shows a case in which pitch marks 122 are attached to the speech waveform 121 of one speech unit among the N speech units at every cycle interval.

  In step S112, as shown in FIG. 12B, windowing is performed with the pitch mark as a reference to cut out the pitch waveform. A Hanning window 123 is used as the window, and the window length is twice the basic period. Then, as shown in FIG. 12C, the windowed waveform 124 is cut out as a pitch waveform. The process shown in FIG. 12 (the process of step S112) is performed on each of the N speech segments. As a result, a sequence of pitch waveforms composed of a plurality of pitch waveforms is obtained for each of the N speech segments.

  Next, the process proceeds to step S113, and the pitches in the pitch waveform series of all N pitch waveforms are matched to the one having the largest number of pitch waveforms in the pitch waveform series of the N speech units of the segment. The pitch waveforms are duplicated so that the number of pitch waveforms is the same (for a series of pitch waveforms with a small number of pitch waveforms).

  FIG. 13 shows a series of pitch waveforms e1 to e3 cut out in step S112 from each of N (for example, three in this case) speech segments d1 to d3 of the segment. Since the number of pitch waveforms in the pitch waveform series e1 is 7, the number of pitch waveforms in the pitch waveform series e2 is 5, and the number of pitch waveforms in the pitch waveform series e3 is 6, the pitch waveform. Among the series e1 to e3, the series e1 has the largest number of pitch waveforms. Therefore, in accordance with the number of pitch waveforms in this series e1 (for example, the number of pitch waveforms here is 7), each of the other series e2 and e3 is one of the pitch waveforms in the series. Is copied and the number of pitch waveforms is set to seven. As a result, new pitch waveform series corresponding to the series e2 and e3 are e2 ′ and e3 ′, respectively.

  Next, the process proceeds to step S114. In this step, processing is performed for each pitch waveform. In step S114, the pitch waveforms corresponding to the N speech units of the segment are averaged for each position, and a new pitch waveform series is generated. The generated new pitch waveform sequence is used as a fused speech unit.

FIG. 14 shows pitch waveform series e1, e2 ′, e3 ′ obtained in step S113 from each of N (for example, three in this case) speech elements d1 to d3 of the segment. Since there are seven pitch waveforms in each series, in step S114, the first to seventh pitch waveforms are averaged with three speech segments, and a new pitch consisting of seven new pitch waveforms is obtained. A waveform series f1 is generated. That is, for example, the centroid of the first pitch waveform of the series e1, the first pitch waveform of the series e2 ′, and the first pitch waveform of the series e3 ′ is obtained, and is obtained as a new pitch waveform series f1. The first pitch waveform. The same applies to the second to seventh pitch waveforms of the new pitch waveform series f1. A series f1 of pitch waveforms is the above “fusion speech unit”. When obtaining the centroid, each pitch waveform may be weighted. In this case, the weight of e1 is w1, the weight of e2 is w2, the weight of e3 is w3, and the weighted average

  Thus, a new pitch waveform series f1 is obtained. In equation (9), the weight wi is assumed to be normalized.

  The pitch waveform fusion processing is not limited to averaging of pitch waveforms. For example, by using closed loop learning, an optimum pitch waveform sequence can be created at the level of the synthesized sound without extracting the pitch waveform of each speech unit. Closed-loop learning is a method of generating representative speech segments that reduce the distortion of natural speech at the level of synthesized speech that is actually synthesized by changing the fundamental frequency and prosodic duration. In closed-loop learning, a segment whose distortion is reduced at the level of the synthesized speech is generated. Therefore, a higher-quality speech unit is created than when a new speech unit is created by averaging pitch waveforms. (See Patent Document 2: Japanese Patent No. 3281281).

[3-3-3-2] In the case of unvoiced sound On the other hand, in the unit fusion processing step, in the case of an unvoiced sound segment, N speech elements of the segment selected by the multiple speech element selection unit 24 are used. Among them, the speech waveform of the speech unit having the first rank attached to each of the N speech units is used as it is.

[3-4] Fusion unit editing / connection unit 26
The fused segment editing / connecting unit 26 transforms the fused speech segments according to the input prosodic information and connects them to generate a speech waveform of synthesized speech. Since the fused speech unit is actually in the form of a pitch waveform, the basic speech and the phoneme duration length of the fused speech unit are indicated in the input prosodic information. The speech waveform can be generated by superimposing the pitch waveform so that the basic frequency and the target speech phoneme duration are the same.

  FIG. 15 is a diagram for explaining the processing of the fusion unit editing / connecting unit 26. In FIG. 15, the synthesized speech units obtained by the segment fusion unit for each synthesis unit of the phonemes “m”, “a”, “d”, and “o” are transformed and connected to each other and called “Mado”. The case where a speech waveform is generated is shown. As shown in FIG. 15, according to the target fundamental frequency and the target phoneme duration duration indicated in the input prosodic information, for each segment (synthesis unit), each pitch waveform in the fused speech segment is Change the basic frequency (change the pitch) or increase the number of pitch waveforms (change the time length). After that, synthesized speech is generated by connecting adjacent pitch waveforms within and between segments.

  As described above, in the present embodiment, in the multiple unit selection type speech synthesis, N speech units for merging speech units and M (N = <M) pieces for obtaining power information. By selecting the speech unit and correcting the power information of the fusion speech unit to the average power information of the M speech units, the discontinuity of connection is reduced and a natural synthesized speech is obtained.

[4] Modification Example [4-1] Modification Example 1
In the present embodiment, the power information of the fusion speech unit is corrected to the average power information of M speech units, but the power information of N speech units is added to the average power information of M speech units in advance. And the power information may be corrected by fusing the corrected N speech segments.

In this case, as shown in FIG. 16, the process of the fusion speech unit creating unit 25 obtains average power information of M speech units by using the equations (6) and (7) in step S161, and in step S162. In step S163, each of the N speech units is corrected so that its power information becomes P _ave, and the speech unit corrected in step S163 is merged to create a fused speech unit.

[4-2] Modification 2
In the present embodiment, the power information of the fusion speech unit is corrected to the average power information of the M speech units. However, the average power information of the M speech units and the average of the N speech units are corrected. The power information is obtained, a ratio for correcting the average power information of the N speech units to be the average power information of the M speech units is obtained, and this ratio is multiplied by each of the N speech units. Thus, the N speech units may be corrected, and the fused N speech units may be created by fusing the corrected N speech units.

In this case, as shown in FIG. 23, the process of the fusion speech unit creating unit 26 obtains average power information P _ave of M speech units by using the equations (6) and (7) in step S231, Similarly, in step S232, average power information P _f of N speech units is obtained, and in step S233, the ratio r is obtained from the average power information P _f and P _ave by equation (8). Thereafter, in step S234, each of the N speech units is corrected by multiplying the obtained ratio r, and in step S235, the corrected N speech units are merged to create a fused speech unit. To do.

[4-3] Modification 3
In the present embodiment, the power information is described as the mean square value represented by Expression (6). However, in the case of an average absolute amplitude value, instead of Expression (6).

And the average absolute amplitude ratio instead of equation (8)

Is used. As a result, the calculation of the square root is not necessary, and the calculation can be performed only by integer arithmetic.

[4-4] Modification 4
Further, in the step of correcting the power information of the fusion speech unit or the selected speech unit in step S83 of FIG. 8 and step S162 of FIG. 16, the correction obtained by formula (8) or formula (11) is performed. Correction may be made only when the ratio r is smaller than 1.0. This means that the correction is made only in the direction of decreasing the power information, and thus it is possible to avoid amplifying the noise component added to the speech segment.

[Second Embodiment]
Next, the fusion speech unit creation unit 25 according to the second embodiment will be described.

  FIG. 17 shows the processing of the fused speech unit creation unit 25 according to the second embodiment.

  In the second embodiment, the weight wi of the fused speech unit in the equation (9) is determined from the statistics of a plurality of M pieces of power information.

  In step S171 in FIG. 17, the average and variance of the power information of the M voices selected by the multiple voice element selection unit 24 are obtained.

In step S172, the likelihood of the power information of each of the N speech units used for unit fusion is obtained. Likelihood assumes a Gaussian distribution,

  In this way, the weight information increases as the power information of each of the N speech units used for unit fusion is closer to the average of the distribution obtained from the power information of the M speech units, and the weight increases as the power information deviates from the average. Get smaller. For this reason, it is possible to reduce the fusion weight of the segments in which the power information of each selected speech unit deviates from the average value in the segment, and it is possible to reduce deterioration in sound quality due to the fusion.

  Further, as an approximation of the above fusion weight, when the power information of each of the N speech units deviates from a section having a predetermined probability in the power information distribution of the M speech units. The weight can be set to 0 and the other segments can be merged as equal weights. FIG. 18 shows this process. In step S181, the average and standard deviation of the power information of the selected M speech units are obtained, and in step S182, a section in which the power information has a predetermined probability is obtained. For example, if the section is (μ−1.96σ <Pi <μ + 1.96σ), the probability that Pi is in that section is 95%.

  Next, in step S183, the speech segment outside the section described above is removed. It can be removed by setting the fusion weight wi when it is out of the section to zero.

In step S184, the speech units are merged to obtain a fused speech unit. When applied to the data of FIG. 9, the interval is (−273573 <Pi <3567739), and P ₃ = 4091979 is out of this range. For this reason, by merging the fusion weights with wi = 0.5, w2 = 0.5, and w3 = 0, it is possible to remove outliers. The determination of the interval is not limited to the above-described method, and a method based on a quartile range that is one of the statistics can be considered.

  For example, by sorting the power, the difference between the 3 / 4th power value (upper quartile) and the 1 / 4th power value (lower quartile) is called the quartile range. A value obtained by subtracting a constant multiple (1.5 times, etc.) from the power value of the lower quartile, and a range obtained by adding the constant multiple to the power value of the upper quartile is defined as a section. Things are outliers.

  According to the present embodiment, when the power information of the top N segments selected for a certain segment deviates, the fusion weight can be reduced or removed from the fusion. As a result, it is possible to prevent deterioration in sound quality due to the fusion of segments having different power information of synthesized speech, and natural synthesized speech can be obtained. Further, in combination with the first embodiment, the fusion weight can be determined in the second embodiment, and the power information can be corrected by the method of the first embodiment.

[Third Embodiment]
In the third embodiment, in the unit selection type speech synthesis, the power information of the selected optimum speech unit is corrected to the average power information of a plurality of units. The difference from the first and second embodiments is that the unit fusion processing is not included.

[1] Configuration of Speech Synthesizer 14 FIG. 18 shows a configuration example of the speech synthesizer 14 according to the second embodiment.

  The speech synthesis unit 14 includes a speech unit storage unit 191, a phoneme environment storage unit 192, a phoneme sequence / prosodic information input unit 193, a multiple speech unit selection unit 194, a speech unit creation unit 195, a speech unit editing / connection unit. 195, the voice waveform output unit 15.

[1] Speech segment storage unit 191
Similar to the first embodiment, the speech unit storage unit 191 stores speech units obtained by analyzing the database, and the phoneme environment information storage unit 192 stores the phoneme environment of each speech unit. Has been.

[2] Multiple speech element selection unit 193
The multiple speech segment selector 193 estimates the degree of distortion between the input prosody information and the prosodic information included in the phoneme environment of the speech segment for each synthesis unit of the input phoneme sequence, and minimizes the degree of distortion. A plurality of speech units and an optimal speech unit are selected from the speech units stored in the speech unit storage unit 191 so as to be converted into the same. Selection of a plurality of speech segments can be performed based on the cost function described above, as shown in FIG. The processing in FIG. 6 is different from the processing in FIG. 6 in that only the optimal speech unit is selected instead of the top N speech units. As a result, a plurality of M speech units corresponding to each segment of the phoneme symbol sequence of the input phoneme sequence and the optimum speech unit are selected.

[3] Speech segment creation unit 195
Next, the speech segment creation unit 195 will be described.

  The speech unit creation unit 195 corrects the power information of the optimum speech unit selected by the multiple speech unit selection unit 194 and creates a speech unit used for synthesis.

  FIG. 21 shows the processing of the speech segment creation unit 195.

First, in step S211, power information Pi (1 = <i = <M) of each of the selected M speech units is obtained, and the average power information P _ave is obtained. It can obtain | require from Formula (6) and Formula (7) similarly to 1st Embodiment. Then, to correct the power information _{P 1} of the optimum speech unit in step S212 to the M average power information _{P ave} obtained in step S211. Here, the ratio r for correcting the power information is

  Ask for. The power information is corrected by multiplying the optimal speech unit by this ratio r.

In the data of FIG. 9, M pieces of average power information P _ave = 1647084, power information P ₁ of the optimum unit P = 2859883, and the ratio r = 0.76. The power information is corrected by multiplying this r by the speech waveform of the optimum speech unit.

[4] Speech segment editing / connection unit 196
The speech segment editing / connection unit 196 generates speech waveforms of synthesized speech by transforming speech segments according to input prosodic information and connecting them. A pitch waveform is cut out from a speech unit so that the fundamental frequency and phoneme duration of the speech unit are the basic frequency of the target speech and the phoneme duration of the target speech indicated in the input prosodic information. An audio waveform can be generated by superimposing the pitch waveform.

  As described above, in the present embodiment, in the unit selection type speech synthesis, the selected speech unit is corrected to the average power information of a plurality of M speech units, so that the sense of stability is increased and the synthesized speech is synthesized. Improve the quality.

[5] Modified Example Similarly to the second embodiment, a section may be obtained from the power information of M speech units, and an optimal speech unit may be selected within the section.

  In this case, the speech segment creation unit 195 is as shown in FIG.

  In step S221, the average and standard deviation of the power information of the M speech units are obtained. In step S222, a section in which the power information has a predetermined probability is obtained.

In step S223, if the power information P1 of the _first speech unit is within the section, it is used, and if not, it is determined whether or not the power information P2 of the _second speech unit is within the section. A speech unit with the lowest cost is selected from among the power information included in the section. As a result, when the power information of the upper speech unit is greatly different, it is possible to select the optimum speech unit while excluding it as an outlier. The power information of the selected speech unit may be corrected to M pieces of average power information.

  The speech unit selected in this way can be edited / connected by the speech unit editing / connecting unit 196 to obtain a synthesized speech.

  Similar to the first embodiment, average absolute amplitude may be used instead of average power information.

  Further, since correction is performed only in the direction in which the power information is reduced as in the first embodiment, the correction ratio r is greater than 1.0 in the step of correcting the power information of the optimum speech unit in step S212 in FIG. You may correct | amend only when small. As a result, it is possible to avoid amplifying the noise component added to the optimum speech segment.

It is a block diagram which shows the structure of the speech synthesizer which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of a speech synthesizer. It is a figure which shows the example of a memory | storage of the speech unit of a speech unit storage part. It is a figure which shows the memory example of the phoneme environment of a phoneme environment storage part. It is a figure explaining the procedure for obtaining a speech unit from speech data. It is a flowchart for demonstrating the processing operation | movement of a several speech unit selection part. It is a figure explaining the procedure for calculating | requiring a several speech unit with respect to each of the some segment corresponding to an input phoneme series. It is a flowchart for demonstrating the processing operation | movement of a fusion speech unit creation part. It is a figure which shows the example of power information correction | amendment. It is a figure which shows the example of power information correction | amendment. It is a flowchart explaining the process of an element fusion step. It is a figure for demonstrating the process of an element fusion part. It is a figure for demonstrating the process of an element fusion part. It is a figure for demonstrating the process of an element fusion part. It is a figure explaining the processing operation of a segment edit and a connection part. It is a flowchart for demonstrating the processing operation | movement of a fusion speech unit creation part. It is a flowchart for demonstrating the processing operation of the fusion speech unit preparation part in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the processing operation of the fusion speech unit preparation part in 2nd Embodiment. It is a block diagram which shows the structural example of the speech synthesizer in the 3rd Embodiment of this invention. It is a flowchart for demonstrating the processing operation | movement of the several speech unit selection part in 3rd Embodiment. It is a flowchart which shows the process of the fusion speech unit preparation part in 3rd Embodiment. It is a flowchart which shows the process of the fusion speech unit preparation part in 3rd Embodiment. It is a flowchart for demonstrating the processing operation | movement of a fusion speech unit creation part.

Explanation of symbols

11 Text Input Unit 12 Language Processing Unit 13 Prosody Processing Unit 14 Speech Synthesis Unit 15 Speech Waveform Output Unit 21 Speech Segment Storage Unit 22 Phoneme Environment Storage Unit 23 Phoneme Sequence / Prosody Information Input Unit 24 Multiple Speech Unit Selection Unit 25 Fusion Speech Segment sequence creation unit 26 Fusion speech segment editing / connection unit

Claims (14)

In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
The selection means selects N and M (N <M) speech segments,
In the representative speech segment generation means,
An average value of power information is obtained from the selected M speech segments,
Creating a fused speech unit by fusing the selected N speech units;
The speech synthesizer characterized in that the representative speech unit is generated by correcting the power information of the fused speech unit to be an average value of the power information obtained from the M speech units. In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
The selection means selects N and M (N <M) speech segments,
In the representative speech segment generation means,
An average value of power information is obtained from the selected M speech segments,
The power information of each of the selected N speech units is corrected to be an average value of the power information,
The speech synthesizer characterized in that the representative speech unit is generated by fusing the corrected N speech units. In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
The selection means selects N and M (N <M) speech segments,
In the representative speech segment generation means,
An average value of power information of the selected M speech units is obtained,
An average value of power information of the selected N speech units is obtained,
Obtaining a correction value for correcting the average value of the power information of the N speech units to be an average value of the power information of the M speech units;
Correcting each of the N speech segments by applying the correction value;
The speech synthesizer characterized in that the representative speech unit is generated by fusing the corrected N speech units. In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
The selection means selects N and M (N <M) speech segments,
In the representative speech segment generation means,
A power information statistic is determined from the selected M speech segments;
Obtaining power information of each of the selected N speech segments;
Determining the weight of each of the N speech units based on the statistic of the obtained power information and the power information of the N speech units;
The speech synthesizer characterized in that the representative speech unit is generated by fusing N speech units based on the weights. In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
The selection means selects N and M (N <M) speech segments,
In the representative speech segment generation means,
A section where the distribution of the power information is a predetermined probability or more from a statistic of power information of the selected M speech units, or a section based on the quartile range of the power information,
Obtaining power information of each of the selected N speech segments;
If the power information of the N speech units is out of the section, exclude it from the speech unit to be selected,
The speech synthesizer characterized in that the representative speech unit is generated by fusing speech units within the section among the selected N speech units . In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
In the selection means, M speech units and an optimal speech unit with a low degree of distortion of the synthesized speech are selected,
In the representative speech segment generation means,
An average value of power information is obtained from the selected M speech segments,
The speech synthesizer, wherein the representative speech segment is generated by correcting power information of the optimal speech segment to be an average value of the power information. In a speech synthesizer that generates a synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units.
Storage means for storing a plurality of speech segments corresponding to the synthesis unit;
Selection means for selecting a plurality of speech units from the speech units stored in the storage unit based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. Representative speech segment generation means for generating a representative speech segment;
Speech waveform generation means for generating a speech waveform by connecting the generated representative speech segments;
With
In the selection means, M speech segments are selected,
In the representative speech segment generation means,
A section where the distribution of the power information is a predetermined probability or more from a statistic of power information of the selected M speech units, or a section based on the quartile range of the power information,
The representative speech unit is generated by selecting an optimal speech unit having a low degree of distortion of the synthesized speech from speech units having power information included in the power information section. Speech synthesizer. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, N and M (N <M) speech segments are selected,
In the representative speech segment generation function,
An average value of power information is obtained from the selected M speech segments,
Creating a fused speech unit by fusing the selected N speech units;
A speech synthesis program, wherein the representative speech unit is generated by correcting the power information of the fused speech unit to be an average value of power information obtained from the M speech units. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, N and M (N <M) speech segments are selected,
In the representative speech segment generation function,
An average value of power information is obtained from the selected M speech segments,
The power information of each of the selected N speech units is corrected to be an average value of the power information,
The speech synthesis program characterized by generating the representative speech unit by fusing the corrected N speech units. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, N and M (N <M) speech segments are selected,
In the representative speech segment generation function,
An average value of power information of the selected M speech units is obtained,
An average value of power information of the selected N speech units is obtained,
Obtaining a correction value for correcting the average value of the power information of the N speech units to be an average value of the power information of the M speech units;
Correcting each of the N speech segments by applying the correction value;
The speech synthesis program characterized by generating the representative speech unit by fusing the corrected N speech units. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, N and M (N <M) speech segments are selected,
In the representative speech segment generation function,
A power information statistic is determined from the selected M speech segments;
Obtaining power information of each of the selected N speech segments;
Determining the weight of each of the N speech units based on the statistic of the obtained power information and the power information of the N speech units;
The speech synthesis program characterized in that the representative speech unit is generated by fusing N speech units based on the weights. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, N and M (N <M) speech segments are selected,
In the representative speech segment generation function,
A section where the distribution of the power information is a predetermined probability or more from a statistic of power information of the selected M speech units, or a section based on the quartile range of the power information,
Obtaining power information of each of the selected N speech segments;
If the power information of the N speech units is out of the section, exclude it from the speech unit to be selected,
The speech synthesis program characterized by generating the representative speech unit by fusing speech units within the section among the selected N speech units . In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, M speech units and an optimal speech unit with a low degree of distortion of the synthesized speech are selected,
In the representative speech segment generation function,
An average value of power information is obtained from the selected M speech segments,
The speech synthesis program, wherein the representative speech segment is generated by correcting power information of the optimal speech segment to be an average value of the power information. In a speech synthesis program for generating synthesized speech by dividing a phoneme sequence obtained from input text into predetermined synthesis units, obtaining representative speech units for each synthesis unit, and connecting these representative speech units,
On the computer,
A storage function for storing a plurality of speech segments corresponding to the synthesis unit;
A selection function for selecting a plurality of speech units from the speech units stored in the storage function based on the degree of distortion of the synthesized speech for each synthesis unit of the phoneme sequence obtained from the input text; ,
A statistic of power information is obtained from the plurality of selected speech segments, and the power information is corrected based on the statistic of the power information so as to improve the quality of the synthesized speech, thereby corresponding to the synthesis unit. A representative speech segment generation function for generating a representative speech segment;
A speech waveform generation function for generating a speech waveform by connecting the generated representative speech segments;
Is to realize
In the selection function, M speech segments are selected,
In the representative speech segment generation function,
A section where the distribution of the power information is a predetermined probability or more from a statistic of power information of the selected M speech units, or a section based on the quartile range of the power information,
The representative speech unit is generated by selecting an optimal speech unit having a low degree of distortion of the synthesized speech from speech units having power information included in the power information section. Speech synthesis program.

Images