Academia.eduAcademia.edu

Figure 2 – uploaded by Giovanni Cherubin

See full PDFdownloadDownload figure
Figure 2: Comparison between direct and indirect approxi- mations. We show the difference between the nonconformity scores of full CP and their approximation as a function of the training set size, averaged across 100 test points. These results support that the direct method is never worse than the indirect (Condition 1), and that both approximate full CP increasingly better (Theorem 2). The standard deviation for the direct approach (blue) is negligible.

Figure 2 Comparison between direct and indirect approxi- mations. We show the difference between the nonconformity scores of full CP and their approximation as a function of the training set size, averaged across 100 test points. These results support that the direct method is never worse than the indirect (Condition 1), and that both approximate full CP increasingly better (Theorem 2). The standard deviation for the direct approach (blue) is negligible.

Related Figures (7)

Figure |: Prediction sets generated by CP methods (€ = 0.2) for CIFAR-10 examples. Our method (ACP) yields prediction sets that (1) contain the true label and (2) are the smallest. ACP approximates well full CP in large training sets, such as CIFAR-10, inheriting its statistical power.
Figure |: Prediction sets generated by CP methods (€ = 0.2) for CIFAR-10 examples. Our method (ACP) yields prediction sets that (1) contain the true label and (2) are the smallest. ACP approximates well full CP in large training sets, such as CIFAR-10, inheriting its statistical power.
Figure 3: Difference in nonconformity scores for different numbers of features (3a) and for various regularization strengths (3b). The differences in scores are averaged across 100 test points. We report the difference between p-values (3c) and error rate (3d).
Figure 3: Difference in nonconformity scores for different numbers of features (3a) and for various regularization strengths (3b). The differences in scores are averaged across 100 test points. We report the difference between p-values (3c) and error rate (3d).
Table 1: Efficiency AUC (¢ € [0,0.2]). The table indicates that differences in the AUC are statistically significant as compared to ACP (D) (*) and ACP (O) (7). Note a randomized version of full CP, called smooth CP, ensures the error rate is exactly € (instead of “‘at most €”).
Table 1: Efficiency AUC (¢ € [0,0.2]). The table indicates that differences in the AUC are statistically significant as compared to ACP (D) (*) and ACP (O) (7). Note a randomized version of full CP, called smooth CP, ensures the error rate is exactly € (instead of “‘at most €”).
Figure 4: Prediction set for a fixed € (4a), and prediction set size w.r.t. ¢ for ACP (4b) and comparing methods (4c). For each method, ¥ is the highest ¢ for which the prediction set includes the true label; higher is better. We also show the AUC in the interval € € [0, 0.2]; lower is better.
Figure 4: Prediction set for a fixed € (4a), and prediction set size w.r.t. ¢ for ACP (4b) and comparing methods (4c). For each method, ¥ is the highest ¢ for which the prediction set includes the true label; higher is better. We also show the AUC in the interval € € [0, 0.2]; lower is better.
Figure 5: Average prediction set size as a function of the significance level ¢ in MNIST (Sa) and CIFAR-10 (5b) for MLPc. Smaller prediction set size indicates better efficiency. Validity. As a way of interpreting why ACP performed better than the other methods, we measure their empirical
Figure 5: Average prediction set size as a function of the significance level ¢ in MNIST (Sa) and CIFAR-10 (5b) for MLPc. Smaller prediction set size indicates better efficiency. Validity. As a way of interpreting why ACP performed better than the other methods, we measure their empirical
Table 3: Fuzziness of ACP and SCP on MNIST and CIFAR- 10. A smaller fuzziness corresponds to a higher efficiency. The table indicates that differences are statistically significant as compared to ACP (D) (*) and ACP (O) (7).
Table 3: Fuzziness of ACP and SCP on MNIST and CIFAR- 10. A smaller fuzziness corresponds to a higher efficiency. The table indicates that differences are statistically significant as compared to ACP (D) (*) and ACP (O) (7).
Table 2: Difference between the expected error (€ = 0.2) and the empirical error rate on CIFAR-10. A positive value indicates a conservative prediction set; a negative value is a validity violation, which may be due to statistical fluctuation.
Table 2: Difference between the expected error (€ = 0.2) and the empirical error rate on CIFAR-10. A positive value indicates a conservative prediction set; a negative value is a validity violation, which may be due to statistical fluctuation.
Related topics:
Computer Science
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved