Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Works
Speaker Anonymization Datasets and Evaluation
Methodology
Results and Discussion
Conclusion
References
All Topics
Art
Music

Deep Learning-based F0 Synthesis for Speaker Anonymization

Profile image of Nils PetersNils Peters

2023, arXiv (Cornell University)

https://doi.org/10.48550/ARXIV.2306.16860
visibility

…

description

5 pages

link

1 file

Abstract

Voice conversion for speaker anonymization is an emerging concept for privacy protection. In a deep learning setting, this is achieved by extracting multiple features from speech, altering the speaker identity, and waveform synthesis. However, many existing systems do not modify fundamental frequency (F0) trajectories, which convey prosody information and can reveal speaker identity. Moreover, mismatch between F0 and other features can degrade speech quality and intelligibility. In this paper, we formally introduce a method that synthesizes F0 trajectories from other speech features and evaluate its reconstructional capabilities. Then we test our approach within a speaker anonymization framework, comparing it to a baseline and a state-of-the-art F0 modification that utilizes speaker information. The results show that our method improves both speaker anonymity, measured by the equal error rate, and utility, measured by the word error rate.

Related papers

Generating identities with mixture models for speaker anonymization
Ivan Martinovic

Computer Speech & Language, 2021

Speaker anonymization methods are a growing research area, due to the common use of voice interfaces coupled with growing privacy requirements. However, existing systems suffer from several issues, in particular a reduction in the entropy space of the newly created voices. This is problematic as it reduces the diversity of the produced anonymous voices, thus making distinguishing between anonymized voices more difficult, and limiting the number of anonymous voices that can be generated. In this work we propose a method for creating the new identity component for anonymized voices, termed an x-vector, which aims to better reflect the natural diversity of voices, in turn increasing the diversity of the voices of anonymized speakers. We combine this identity generation method with existing anonymization schemes, to produce an overall anonymization system, which we evaluate. Our results demonstrate that our scheme creates more diverse anonymized voices than the existing baseline method. Furthermore, our results show that the assumption of perfect de-coupling between identity and non-identity voice components used in existing speaker anonymization frameworks does not hold, highlighting a clear avenue for future work.

View PDFchevron_right
Reversible speaker de-identification using pre-trained transformation functions
Carmen Magariños

Computer Speech & Language, 2017

Speaker de-identification approaches must accomplish three main goals: universality, naturalness and reversibility. The main drawback of the traditional approach to speaker de-identification using voice conversion techniques is its lack of universality, since a parallel corpus between the input and target speakers is necessary to train the conversion parameters. It is possible to make use of a synthetic target to overcome this issue, but this harms the naturalness of the resulting de-identified speech. Hence, a technique is proposed in this paper in which a pool of pre-trained transformations between a set of speakers is used as follows: given a new user to de-identify, its most similar speaker in this set of speakers is chosen as the source speaker, and the speaker that is the most dissimilar to the source speaker is chosen as the target speaker. Speaker similarity is measured using the i-vector paradigm, which is usually employed as an objective measure of speaker de-identification performance, leading to a system with high de-identification accuracy. The transformation method is based on frequency warping and amplitude scaling, in order to obtain natural sounding speech while masking the identity of the speaker. In addition, compared to other voice conversion approaches, the proposed method is easily reversible. Experiments were conducted on Albayzin database, and performance was evaluated in terms of objective and subjective measures. These results showed a high success when de-identifying speech, as well as a great naturalness of the transformed voices. In addition, when making the transformation parameters available to a trusted holder, it is possible to invert the de-identification procedure, hence recovering the original speaker identity. The computational cost of the proposed approach is small, making it possible to produce de-identified speech in real-time with a high level of naturalness.

View PDFchevron_right
Adversarial Disentanglement of Speaker Representation for Attribute-Driven Privacy Preservation
mohammad mohammadamini

Interspeech 2021

In speech technologies, speaker's voice representation is used in many applications such as speech recognition, voice conversion, speech synthesis and, obviously, user authentication. Modern vocal representations of the speaker are based on neural embeddings. In addition to the targeted information, these representations usually contain sensitive information about the speaker, like the age, sex, physical state, education level or ethnicity. In order to allow the user to choose which information to protect, we introduce in this paper the concept of attributedriven privacy preservation in speaker voice representation. It allows a person to hide one or more personal aspects to a potential malicious interceptor and to the application provider. As a first solution to this concept, we propose to use an adversarial autoencoding method that disentangles in the voice representation a given speaker attribute thus allowing its concealment. We focus here on the sex attribute for an Automatic Speaker Verification (ASV) task. Experiments carried out using the VoxCeleb datasets have shown that the proposed method enables the concealment of this attribute while preserving ASV ability. 1 Throughout this paper, the term sex refers to the biological differences between female and male [20].

View PDFchevron_right
Voice conversion using deep neural networks
Hy Nguyen
View PDFchevron_right
Normalization Techniques for Hiding Speakers Identity
Urmila Shrawankar

In most security domain transmission of secret and confidential voice data requires that the identity of the speaker should not get revealed but at the same time data contents in speech signal should be interpreted. Speech and speaker normalization techniques help to solve this issue. This paper proposes speaker normalization technique that attempts to alter the characteristics of speaker’s voice so that it would be difficult for an eavesdropper to identify the speaker. Intended listener can retrieve the particular information. Evaluations of subjective and objective tests confirm the effectiveness of the proposed method and because of this method security enhances.

View PDFchevron_right
ASVspoof 2019: a large-scale public database of synthetic, converted and replayed speech
Yu Tsao

2020

Automatic speaker verification (ASV) is one of the most natural and convenient means of biometric person recognition. Unfortunately, just like all other biometric systems, ASV is vulnerable to spoofing, also referred to as "presentation attacks." These vulnerabilities are generally unacceptable and call for spoofing countermeasures or "presentation attack detection" systems. In addition to impersonation, ASV systems are vulnerable to replay, speech synthesis, and voice conversion attacks. The ASVspoof 2019 edition is the first to consider all three spoofing attack types within a single challenge. While they originate from the same source database and same underlying protocol, they are explored in two specific use case scenarios. Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques. Replay spoofing attacks wit...

View PDFchevron_right
Preserving Privacy in Speaker and Speech Characterisation
Dijana Petrovska-delacrétaz

Computer Speech & Language

Speech recordings are a rich source of personal, sensitive data that can be used to support a plethora of diverse applications, from health profiling to biometric recognition. It is therefore essential that speech recordings are adequately protected so that they cannot be misused. Such protection, in the form of privacy-preserving technologies, is required to ensure that: (i) the biometric profiles of a given individual (e.g., across different biometric service operators) are unlinkable; (ii) leaked, encrypted biometric information is irreversible, and that (iii) biometric references are renewable. Whereas many privacy-preserving technologies have been developed for other biometric characteristics, very few solutions have been proposed to protect privacy in the case of speech signals. Despite privacy preservation this is now being mandated by recent European and international data protection regulations. With the aim of fostering progress and collaboration between researchers in the speech, biometrics and applied cryptography communities, this survey article provides an introduction to the field, starting with a legal perspective on privacy preservation in the case of speech data. It then establishes the requirements for effective privacy preservation, reviews generic cryptography-based solutions, followed by specific techniques that are applicable to speaker characterisation (biometric applications) and speech characterisation (non-biometric applications). Glancing at non-biometrics, methods are presented to avoid function creep, preventing the exploitation of biometric information, e.g., to single out an identity in speech-assisted health care via I Recent advances in speaker and language recognition and characterisation.

View PDFchevron_right
Design of Voice Privacy System using Linear Prediction
Shrishti Singh

2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020

Speaker’s identity is the most crucial information exploited (implicitly) by an Automatic Speaker Verification (ASV) system. Numerous attacks can be obliterated simultaneously if privacy preservation is exercised for a speaker’s identity. The baseline of the Voice Privacy Challenge 2020 by INTERSPEECH uses the Linear Prediction (LP) model of speech, and McAdam’s coefficient for achieving speaker de-identification. The baseline approach focuses on altering only the pole angles using McAdam’s coefficient. However, from speech acoustics and digital resonator design, the radius of the poles is associated with various energy losses. The energy losses implicitly carry speaker-specific information during speech production. To that effect, the authors have brought fine-tuned changes in both pole angle and pole radius, resulting in 18.98% higher value of EER for Vctk-test-com dataset, and 5% lower WER for Libri-test dataset compared to the baseline. This means privacy-preservation is indeed ...

View PDFchevron_right
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Tomi Kinnunen

2020

Automatic speaker verification (ASV) is one of the most natural and convenient means of biometric person recognition. Unfortunately, just like all other biometric systems, ASV is vulnerable to spoofing, also referred to as "presentation attacks." These vulnerabilities are generally unacceptable and call for spoofing countermeasures or "presentation attack detection" systems. In addition to impersonation, ASV systems are vulnerable to replay, speech synthesis, and voice conversion attacks. The ASVspoof 2019 edition is the first to consider all three spoofing attack types within a single challenge. While they originate from the same source database and same underlying protocol, they are explored in two specific use case scenarios. Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques. Replay spoofing attacks wit...

View PDFchevron_right
Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World
ASIF SHA

Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021

Advances in deep learning have introduced a new wave of voice synthesis tools, capable of producing audio that sounds as if spoken by a target speaker. If successful, such tools in the wrong hands will enable a range of powerful attacks against both humans and software systems (aka machines). This paper documents efforts and findings from a comprehensive experimental study on the impact of deep-learning based speech synthesis attacks on both human listeners and machines such as speaker recognition and voicesignin systems. We find that both humans and machines can be reliably fooled by synthetic speech, and that existing defenses against synthesized speech fall short. These findings highlight the need to raise awareness and develop new protections against synthetic speech for both humans and machines. CCS CONCEPTS • Computing methodologies → Machine learning; • Security and privacy → Biometrics.

View PDFchevron_right

References (22)

  1. N. Tomashenko et al., "Introducing the VoicePrivacy initiative," in Proc. Interspeech Conf., 2020.
  2. N. Tomashenko et al. "2nd VoicePrivacy chal- lenge evaluation plan." (2022), [Online]. Available: https://arxiv.org/abs/2203.12468.
  3. H. Turner, G. Lovisotto, and I. Martinovic, "Speaker anonymization with distribution-preserving x-vector generation for the VoicePrivacy challenge 2020," in VoicePrivacy Challenge Submission, 2020.
  4. P. Champion, D. Jouvet, and A. Larcher, "A study of f0 modification for x-vector based speech pseudonymiza- tion across gender," in 2nd AAAI Workshop on Privacy- Preserving AI, 2021.
  5. U. E. Gaznepoglu and N. Peters, "Exploring the impor- tance of f0 trajectories for speaker anonymization using x-vectors and neural waveform models," in Workshop on MLSLP, 2021.
  6. C. O. Mawalim, K. Galajit, J. Karnjana, S. Kidani, and M. Unoki, "Speaker anonymization by modifying fundamental frequency and x-vector singular value," Computer Speech & Language, vol. 73, 2022.
  7. F. Fang et al., "Speaker anonymization using x-vector and neural waveform models," in Proc. 10th ISCA Speech Synthesis Workshop, 2019.
  8. N. Tomashenko et al., "The VoicePrivacy 2020 chal- lenge: Results and findings," Computer Speech & Lan- guage, vol. 74, 2022.
  9. L. Tavi, T. Kinnunen, and R. González Hautamäki, "Improving speaker de-identification with functional data analysis of f0 trajectories," Speech Communication, vol. 140, 2022.
  10. U. E. Gaznepoglu, A. Leschanowsky, and N. Pe- ters, "Voiceprivacy 2022 system description: Speaker anonymization with feature-matched F0 trajectories," in VoicePrivacy Challenge Submission, 2022.
  11. N. Tomashenko et al. "The VoicePrivacy 2022 challenge results." (2022), [Online]. Available: https://www.voiceprivacychallenge.org/results-2022/docs/VoicePriva
  12. S. A. Zahorian and H. Hu, "A spectral/temporal method for robust fundamental frequency tracking," The Journal Acoust. Soc. of America, vol. 123, no. 6, 2008.
  13. J. W. Kim, J. Salamon, P. Li, and J. P. Bello, "CREPE: A convolutional representation for pitch estimation," in Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2018.
  14. L. Ardaillon and A. Roebel, "Fully-convolutional net- work for pitch estimation of speech signals," in Proc. Interspeech Conf., 2019.
  15. D. N. Tran, U. Batricevic, and K. Koishida, "Robust pitch regression with voiced/unvoiced classification in nonstationary noise environments," in Proc. Interspeech Conf., 2020.
  16. R. Vaysse, C. Astésano, and J. Farinas, "Performance analysis of various fundamental frequency estimation algorithms in the context of pathological speech," The Journal Acoust. Soc. of America, vol. 152, no. 5, 2022.
  17. N. Tomashenko et al. "1st VoicePrivacy chal- lenge evaluation plan." (2020), [Online]. Available: https://arxiv.org/abs/2205.07123.
  18. P.-G. Noé et al., "Towards a unified assessment frame- work of speech pseudonymisation," Computer Speech & Language, vol. 72, 2022.
  19. P. Champion, D. Jouvet, and A. Larcher, "Speaker infor- mation modification in VoicePrivacy 2020 toolchain," in VoicePrivacy Challenge Submission, 2020.
  20. A. Paszke et al., "PyTorch: An imperative style, high- performance deep learning library," in Proc. of 33rd Int. Conf. on Neural Inf. Proc. Sys. (NeurIPS), 2019.
  21. V. Fomin, J. Anmol, S. Desroziers, J. Kriss, and A. Tejani, High-level library to help with training neural networks in PyTorch, 2020.
  22. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, "Optuna: A next-generation hyperparameter optimiza- tion framework," in Proc. ACM SIGKDD, 2019. B. Gfeller, C. Frank, D. Roblek, M. Sharifi, M. Tagliasacchi, and M. Velimirović, "SPICE: Self- supervised pitch estimation," IEEE/ACM TASLP, vol. 28, 2020.

Related papers

VoicePrivacy 2022 System Description: Speaker Anonymization with Feature-matched F0 Trajectories
Nils Peters

VoicePrivacy Challenge, 2022

We introduce a novel method to improve the performance of the VoicePrivacy Challenge 2022 baseline B1 variants. Among the known deficiencies of x-vector-based anonymization systems is the insufficient disentangling of the input features. In particular, the fundamental frequency (F0) trajectories, which are used for voice synthesis without any modifications. Especially in cross-gender conversion, this situation causes unnatural sounding voices, increases word error rates (WERs), and personal information leakage. Our submission overcomes this problem by synthesizing an F0 trajectory, which better harmonizes with the anonymized x-vector. We utilized a low-complexity deep neural network to estimate an appropriate F0 value per frame, using the linguistic content from the bottleneck features (BN) and the anonymized x-vector. Our approach results in a significantly improved anonymization system and increased naturalness of the synthesized voice. Consequently, our results suggest that F0 extraction is not required for voice anonymization.

View PDFchevron_right
Exploring the Importance of F0 Trajectories for Speaker Anonymization using X-vectors and Neural Waveform Models
Nils Peters

2021

Voice conversion for speaker anonymization is an emerging field in speech processing research. Many state-of-the-art approaches are based on the resynthesis of the phoneme posteriorgrams (PPG), the fundamental frequency (F0) of the input signal together with modified X-vectors. Our research focuses on the role of F0 for speaker anonymization, which is an understudied area. Utilizing the VoicePrivacy Challenge 2020 framework and its datasets we developed and evaluated eight low-complexity F0 modifications prior resynthesis. We found that modifying the F0 can improve speaker anonymization by as much as 8% with minor word-error rate degradation.

View PDFchevron_right
Self-Supervised Speech Representations Preserve Speech Characteristics while Anonymizing Voices
Abner Lemuel Ramirez Hernandez

ArXiv, 2022

Collecting speech data is an important step in training speech recognition systems and other speech-based machine learning models. However, the issue of privacy protection is an increasing concern that must be addressed. The current study investi-gates the use of voice conversion as a method for anonymizing voices. In particular, we train several voice conversion models using self-supervised speech representations including Wav2Vec2.0, Hubert and UniSpeech. Converted voices retain a low word error rate within 1% of the original voice. Equal error rate increases from 1.52% to 46.24% on the LibriSpeech test set and from 3.75% to 45.84% on speakers from the VCTK corpus which signifies degraded performance on speaker verification. Lastly, we conduct experiments on dysarthric speech data to show that speech features relevant to articulation, prosody, phonation and phonology can be extracted from anonymized voices for discriminating between healthy and pathological speech.

View PDFchevron_right
Speaker Anonymization with Distribution-Preserving X-Vector Generation for the VoicePrivacy Challenge 2020
Ivan Martinovic

ArXiv, 2020

In this paper, we present a Distribution-Preserving Voice Anonymization technique, as our submission to the VoicePrivacy Challenge 2020. We notice that the challenge baseline system generates fake X-vectors which are very similar to each other, significantly more so than those extracted from organic speakers. This difference arises from averaging many X-vectors from a pool of speakers in the anonymization processs, causing a loss of information. We propose a new method to generate fake X-vectors which overcomes these limitations by preserving the distributional properties of X-vectors and their intra-similarity. We use population data to learn the properties of the X-vector space, before fitting a generative model which we use to sample fake X-vectors. We show how this approach generates X-vectors that more closely follow the expected intra-similarity distribution of organic speaker X-vectors. Our method can be easily integrated with others as the anonymization component of the syst...

View PDFchevron_right
Voice Privacy Through x-Vector and CycleGAN-Based Anonymization
DIPESH KUMAR SINGH

Interspeech 2021, 2021

With the rise in usage of voice assistants and spoken language interfaces, important concerns regarding voice data privacy have been prompted. In an attempt to reduce the threat of attacks on voice data, in this paper, we propose a speaker anonymization system based on CycleGAN. This method modifies the speaker's gender and accent information from the original speech signal. The proposed method gives a more naturalsounding anonymized voice in addition to a de-identified speaker. We have chosen baseline-1 of The Voice Privacy Challenge-2020 as our baseline system. Training of Cycle-GAN, ASR, and ASV experiments are performed on the subset of Librispeech corpus. In this paper, the double anonymization technique is also explored in which the CycleGAN-based anonymization technique is adopted on top of the baseline system. Experimental results show that combining the proposed method with the x-vector and neural source-filter (NSF) model-based method (baseline system) gives up to 5.61% relative improvement in EER of original-anonymized, enroll-trial pairs. However, it gives up to 19.30% relative improvement in EER for anonymized-anonymized enroll-trial pairs. We observed that along with the good speaker de-identification, the anonymized utterances have adequate speech intelligibility and naturalness.

View PDFchevron_right

Related topics

  • Computer Science
  • Music and identity
  • Word error rate
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved