Academia.eduAcademia.edu

Figure 3 – uploaded by ANURAG RANJAN

See full PDFdownloadDownload figure
Figure 3: Our method for structured sparsity using a linear subspace (LCS+L+IN) and a point sub- space (LCS+P+IN) compared to Universal Slimming (US) (Yu & Huang, 2019), Network Slimming (NS) (Yu et al., 2018), and Learning Efficient Convolutions (LEC) (Liu et al., 2017). LEC does not provide an open-source implementation of their method for ResNet18, so we omit it. We do not allow fine-tuning or recalibration.

Figure 3 Our method for structured sparsity using a linear subspace (LCS+L+IN) and a point sub- space (LCS+P+IN) compared to Universal Slimming (US) (Yu & Huang, 2019), Network Slimming (NS) (Yu et al., 2018), and Learning Efficient Convolutions (LEC) (Liu et al., 2017). LEC does not provide an open-source implementation of their method for ResNet18, so we omit it. We do not allow fine-tuning or recalibration.

Related Figures (10)

Figure 1: (a) Depiction of our method for learning a linear subspace of networks w* parameterized by @ € [a1, 2]. When compressing with compression function f and compression level , we ob- tain a spectrum of networks which demonstrate an efficiency-accuracy trade-off. (b) Our algorithm.
Figure 1: (a) Depiction of our method for learning a linear subspace of networks w* parameterized by @ € [a1, 2]. When compressing with compression function f and compression level , we ob- tain a spectrum of networks which demonstrate an efficiency-accuracy trade-off. (b) Our algorithm.
Figure 2: Analysis of observed batch-wise means 4 and stored BatchNorm means ys during testing for models trained with TopK unstructured sparsity. The models are trained with different target sparsities and evaluated with various inference-time sparsities. (a)-(b): The distribution of | — f2| across all layers. (c)-(d): The average value of |sz — | for individual layers. (e)-(f): The correlation between the average of |4z — | and test set error. Note that in (b) and (d), sparsities of 0 and 0.493 produce near-identical results, thus those curves are overlapping.
Figure 2: Analysis of observed batch-wise means 4 and stored BatchNorm means ys during testing for models trained with TopK unstructured sparsity. The models are trained with different target sparsities and evaluated with various inference-time sparsities. (a)-(b): The distribution of | — f2| across all layers. (c)-(d): The average value of |sz — | for individual layers. (e)-(f): The correlation between the average of |4z — | and test set error. Note that in (b) and (d), sparsities of 0 and 0.493 produce near-identical results, thus those curves are overlapping.
Figure 4: Our method for unstructured sparsity using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular TopK target. The TopK target refers to the fraction of weights that remain unpruned during training.
Figure 4: Our method for unstructured sparsity using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular TopK target. The TopK target refers to the fraction of weights that remain unpruned during training.
Figure 5: Our method for quantization using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular bit width target.
Figure 5: Our method for quantization using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular bit width target.
Figure 6: Standard evaluation of a linear subspace with network f(w* (a) ,y(a)) (Learned line), and evaluation when evaluating with reversed compression levels, f(w*(a, (1 — «)) (Reversed line).
Figure 6: Standard evaluation of a linear subspace with network f(w* (a) ,y(a)) (Learned line), and evaluation when evaluating with reversed compression levels, f(w*(a, (1 — «)) (Reversed line).
Figure 7: Analysis of the mean absolute difference between observed batch-wise means f2 and stored BatchNorm means yp during testing for cPreResNet models trained with NS (Yu et al., 2018) or US (Yu & Huang, 2019). (a)-(b): The distribution of |41 — ,2| across all layers. (c)-(d): The average value of |ts — ,4| for each individual BatchNorm layer. (e)-(f): The correlation between the average of | — {| and test set error.
Figure 7: Analysis of the mean absolute difference between observed batch-wise means f2 and stored BatchNorm means yp during testing for cPreResNet models trained with NS (Yu et al., 2018) or US (Yu & Huang, 2019). (a)-(b): The distribution of |41 — ,2| across all layers. (c)-(d): The average value of |ts — ,4| for each individual BatchNorm layer. (e)-(f): The correlation between the average of | — {| and test set error.
Figure 8: Analysis of the mean absolute difference between observed batch-wise means f2 and stored BatchNorm means jz during testing for cPreResNet models trained with different quantization bit widths. (a)-(b): The distribution of |42 — f4| across all layers. (c)-(d): The average value of |ps — | for each individual BatchNorm layer. (e)-(f): The correlation between the average of |4 — f4| and test set error.
Figure 8: Analysis of the mean absolute difference between observed batch-wise means f2 and stored BatchNorm means jz during testing for cPreResNet models trained with different quantization bit widths. (a)-(b): The distribution of |42 — f4| across all layers. (c)-(d): The average value of |ps — | for each individual BatchNorm layer. (e)-(f): The correlation between the average of |4 — f4| and test set error.
Figure 9: Our method for structured sparsity using a linear subspace (LCS+L+IN) and a point sub- space (LCS+P+IN), compared to Universal Slimming (US) (Yu & Huang, 2019), Network Slimming (NS) (Yu et al., 2018), and Learning Efficient Convolutions (LEC) (Liu et al., 2017).
Figure 9: Our method for structured sparsity using a linear subspace (LCS+L+IN) and a point sub- space (LCS+P+IN), compared to Universal Slimming (US) (Yu & Huang, 2019), Network Slimming (NS) (Yu et al., 2018), and Learning Efficient Convolutions (LEC) (Liu et al., 2017).
Figure 10: Our method for quantization using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular bit width target.
Figure 10: Our method for quantization using a linear subspace (LCS+L+GN) and a point subspace (LCS+P+GN) compared to networks trained for a particular bit width target.
Related topics:
Computer Science
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved