Academia.eduAcademia.edu

Figure 5 – uploaded by Jinsung Yoon

See full PDFdownloadDownload figure
Figure 5: Block diagram of the DPGAN benchmark. It uses the standard WGAN framework. To guarantee differential privacy of the generator (with Post-processing Theorem), noise is added to the gradient of the discriminator during training to create a differentially private discriminator.

Figure 5 Block diagram of the DPGAN benchmark. It uses the standard WGAN framework. To guarantee differential privacy of the generator (with Post-processing Theorem), noise is added to the gradient of the discriminator during training to create a differentially private discriminator.

Related Figures (16)

Table 1: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 107” )-differentially private).
Table 1: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 107” )-differentially private).
Table 2: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private) over 6 different datasets. GAN is (00, o0)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. In this subsection, we evaluate PATE-GAN and DPGAN in Setting B (trained on synthetic, tested on real) to understand whether or not the models are capturing the feature-label relationships well. Intuitively, if a synthetic dataset is such that a model trained on it performs well when performance is measured on real data, then the relationship between feature and label in the synthetic data is similar to that in the real data. In Table | we give the results for the Kaggle Credit dataset for all 12 predictive models. In Table 2, we give the performance on each dataset averaged across the 12 methods for each of the 6 datasets. A breakdown of the performance of each predictive model for each dataset can be found in the Appendix. Across all datasets, we see that PATE-GAN is capable of generating synthetic samples that better preserve the feature-label relationship (according to AUROC and AUPRC) than DPGAN.
Table 2: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private) over 6 different datasets. GAN is (00, o0)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. In this subsection, we evaluate PATE-GAN and DPGAN in Setting B (trained on synthetic, tested on real) to understand whether or not the models are capturing the feature-label relationships well. Intuitively, if a synthetic dataset is such that a model trained on it performs well when performance is measured on real data, then the relationship between feature and label in the synthetic data is similar to that in the real data. In Table | we give the results for the Kaggle Credit dataset for all 12 predictive models. In Table 2, we give the performance on each dataset averaged across the 12 methods for each of the 6 datasets. A breakdown of the performance of each predictive model for each dataset can be found in the Appendix. Across all datasets, we see that PATE-GAN is capable of generating synthetic samples that better preserve the feature-label relationship (according to AUROC and AUPRC) than DPGAN.
Figure 1: Average AUROC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various € (with 6 = 107°) (Setting B).
Figure 1: Average AUROC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various € (with 6 = 107°) (Setting B).
Table 3: Synthetic Ranking Probability of PATE-GAN and the benchmark when comparing Setting A and Setting C for various € (with 6 = 10~°) in terms of AUROC. The Synthetic Ranking Agreement of Original GAN is 0.9091, which is also attained by both PATE-GAN and DPGAN for e = 50.
Table 3: Synthetic Ranking Probability of PATE-GAN and the benchmark when comparing Setting A and Setting C for various € (with 6 = 10~°) in terms of AUROC. The Synthetic Ranking Agreement of Original GAN is 0.9091, which is also attained by both PATE-GAN and DPGAN for e = 50.
Table 4: Trade-off between the number of teachers and the performances (AUROC, AUPRC) The number of teachers is a hyper-parameter of PATE-GAN and we choose the number of teachers among {N/10, N/50, N/100, N/500, N/1000, N/5000, N/10000} where NV is the total number of samples. As we described in the previous section, there is a trade-off between number of teach- ers and the corresponding quality of the synthetic data. Table 4 quantitatively shows the trade-off between the number of teachers and the performance (in terms of both AUROC and AUPRC).
Table 4: Trade-off between the number of teachers and the performances (AUROC, AUPRC) The number of teachers is a hyper-parameter of PATE-GAN and we choose the number of teachers among {N/10, N/50, N/100, N/500, N/1000, N/5000, N/10000} where NV is the total number of samples. As we described in the previous section, there is a trade-off between number of teach- ers and the corresponding quality of the synthetic data. Table 4 quantitatively shows the trade-off between the number of teachers and the performance (in terms of both AUROC and AUPRC).
Table 5: No of samples, No of features, Average AUROC and AUPRC performance across 12 different predic- tive models trained and tested on the real data (Setting A) for the 6 datasets: Kaggle Credit, MAGGIC, UNOS, Kaggle Cervical Cancer, UCI ISOLET, UCI Epileptic Seizure Recognition. Table 5 summarises the 6 datasets we use and provides a baseline performance for a predictive model on each dataset - recall that Setting A refers to training and testing on the real data. The AUROC and AUPRC in this setting are upper bounds on the AUROC and AUPRC we could hope to achieve in Setting B.
Table 5: No of samples, No of features, Average AUROC and AUPRC performance across 12 different predic- tive models trained and tested on the real data (Setting A) for the 6 datasets: Kaggle Credit, MAGGIC, UNOS, Kaggle Cervical Cancer, UCI ISOLET, UCI Epileptic Seizure Recognition. Table 5 summarises the 6 datasets we use and provides a baseline performance for a predictive model on each dataset - recall that Setting A refers to training and testing on the real data. The AUROC and AUPRC in this setting are upper bounds on the AUROC and AUPRC we could hope to achieve in Setting B.
Figure 2: Average AUPRC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various € (with 6 = 107°) (Setting B). VARYING THE PRIVACY CONSTRAINT (€) IN TERMS OF AUPRC
Figure 2: Average AUPRC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various € (with 6 = 107°) (Setting B). VARYING THE PRIVACY CONSTRAINT (€) IN TERMS OF AUPRC
Table 6: Agreed ranking probability of PATE-GAN and the benchmark to order the features by variable impor- tance in terms of absolute Pearson correlation coefficient We compare the ranking of variables by their importance (according to absolute Pearson correlation coefficient with the label) on the original dataset and on both synthetic datasets. We report the results using agreed ranking probability. As can be seen in Table 6, PATE-GAN achieves consistently better agreed ranking probability across all values of tested € (with 6 = 10~°).
Table 6: Agreed ranking probability of PATE-GAN and the benchmark to order the features by variable impor- tance in terms of absolute Pearson correlation coefficient We compare the ranking of variables by their importance (according to absolute Pearson correlation coefficient with the label) on the original dataset and on both synthetic datasets. We report the results using agreed ranking probability. As can be seen in Table 6, PATE-GAN achieves consistently better agreed ranking probability across all values of tested € (with 6 = 10~°).
Table 7: Average AUROC and AUPRC performance of 12 different predictive models trained on the synthetic datasets for ¢ = 1,10 with 6 = 10~° - Setting B using UCI ISOLET dataset. GAN is (00, oo)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. HIGH-DIMENSIONAL RESULTS: UCI ISOLET AND UCI EPILEPTIC SEIZURE RECOGNITIO}
Table 7: Average AUROC and AUPRC performance of 12 different predictive models trained on the synthetic datasets for ¢ = 1,10 with 6 = 10~° - Setting B using UCI ISOLET dataset. GAN is (00, oo)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. HIGH-DIMENSIONAL RESULTS: UCI ISOLET AND UCI EPILEPTIC SEIZURE RECOGNITIO}
Table 8: Average AUROC and AUPRC performance across 12 different predictive models trained on the syn- thetic datasets with various « = 1,10 with 6 = 10~° - Setting B using UCI Epileptic Seizure Recognition dataset. GAN is (00, 00)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.
Table 8: Average AUROC and AUPRC performance across 12 different predictive models trained on the syn- thetic datasets with various « = 1,10 with 6 = 10~° - Setting B using UCI Epileptic Seizure Recognition dataset. GAN is (00, 00)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.
Table 9: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (co, 0o)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.
Table 9: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (co, 0o)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.
Table 10: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (00, 0o)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. UNOS HEART WAIT DATASET RESULT
Table 10: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (00, 0o)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. UNOS HEART WAIT DATASET RESULT
Table 11: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (co, co)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. KAGGLE CERVICAL CANCER DATASET RESULT
Table 11: Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10~°)-differentially private). GAN is (co, co)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN. KAGGLE CERVICAL CANCER DATASET RESULT
Figure 3: Block diagram of the training procedure for the teacher-discriminator during a single generato1 iteration. Teacher-discriminators are trained to minimize the classification loss when classifying samples as real samples or generated samples. During this step only the parameters of the teachers are updates (and not the generator). The two figures below indicate the iterative training procedure carried out by PATE-GAN; the figure: correspond to a single generator update.
Figure 3: Block diagram of the training procedure for the teacher-discriminator during a single generato1 iteration. Teacher-discriminators are trained to minimize the classification loss when classifying samples as real samples or generated samples. During this step only the parameters of the teachers are updates (and not the generator). The two figures below indicate the iterative training procedure carried out by PATE-GAN; the figure: correspond to a single generator update.
Figure 4: Block diagram of the training procedure for the student-discriminator and the generator. The student- discriminator is trained using noisy teacher-labelled generated samples (the noise provides the DP guarantees). The student is trained to minimize classification loss on this noisily labelled dataset, while the generator is trained to maximize the student loss. Note that the teachers are not updated during this step, only the student and the generator.
Figure 4: Block diagram of the training procedure for the student-discriminator and the generator. The student- discriminator is trained using noisy teacher-labelled generated samples (the noise provides the DP guarantees). The student is trained to minimize classification loss on this noisily labelled dataset, while the generator is trained to maximize the student loss. Note that the teachers are not updated during this step, only the student and the generator.
Related topics:
Computer ScienceDifferential Privacy
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved