Academia.eduAcademia.edu

Figure 6 – uploaded by Raymond Spiteri

See full PDFdownloadDownload figure
Figure 6. Purity as a function of dimension, parameterized by profile length for 4-gram represen- tation when LSI / ICA dimension reduction is used (top/bottom) on a typical dataset (RD-256). Best N-gram Parameters Overall, we are interested in selecting a good profile length and reduced dimensionality for 3-gram and 4-gram representations. The general result is that for each of the 3-gram and 4-gram representation, and for a mid-range profile length (around 2000), ICA is the method of choice. But choosing between the 3-gram and 4-gram representations seems to depend on the data set. Typical results are shown in Fig. 6.

Figure 6 Purity as a function of dimension, parameterized by profile length for 4-gram represen- tation when LSI / ICA dimension reduction is used (top/bottom) on a typical dataset (RD-256). Best N-gram Parameters Overall, we are interested in selecting a good profile length and reduced dimensionality for 3-gram and 4-gram representations. The general result is that for each of the 3-gram and 4-gram representation, and for a mid-range profile length (around 2000), ICA is the method of choice. But choosing between the 3-gram and 4-gram representations seems to depend on the data set. Typical results are shown in Fig. 6.

Related Figures (5)

Figure 1. Comparing dimension reduction techniques, ICA, LSI, and DF on the word rep- resentation of a typical dataset (RD-256). The results show that for all datasets, clustering quality using ICA is better than using LSI in the whole range of dimensionalities investigated. For low dimensionalities, es- pecially lowerthan 50, for all datasets, the DF based method has the worst performance among the dimension reduction methods used. The best performance of DF is comparable to the best performance of LSI and ICA, but at much higher dimensionalities.
Figure 1. Comparing dimension reduction techniques, ICA, LSI, and DF on the word rep- resentation of a typical dataset (RD-256). The results show that for all datasets, clustering quality using ICA is better than using LSI in the whole range of dimensionalities investigated. For low dimensionalities, es- pecially lowerthan 50, for all datasets, the DF based method has the worst performance among the dimension reduction methods used. The best performance of DF is comparable to the best performance of LSI and ICA, but at much higher dimensionalities.
Figure 2. Comparing dimension reduction techniques, ICA, LSI and DF, on the term rep- resentation of a typical dataset (RD-256).
Figure 2. Comparing dimension reduction techniques, ICA, LSI and DF, on the term rep- resentation of a typical dataset (RD-256).
ple, in datasets RD256 and RD512, further increasing the profile length beyond 2000 makes clustering quality worse. The optimum profile length is different for each dataset, but overall we do not need to go further than profile length of 4000 to get the best clustering result. Profile length equal to 2000 is still one of the best profile lengths for 3-gram when we use LSI as the dimension reduction method.
ple, in datasets RD256 and RD512, further increasing the profile length beyond 2000 makes clustering quality worse. The optimum profile length is different for each dataset, but overall we do not need to go further than profile length of 4000 to get the best clustering result. Profile length equal to 2000 is still one of the best profile lengths for 3-gram when we use LSI as the dimension reduction method.
Figure 3. Purity as a function of dimension, parameterized by profile length for 3-gram representation when ICA dimension reduc- tion is used on a typical dataset (RD-256).
Figure 3. Purity as a function of dimension, parameterized by profile length for 3-gram representation when ICA dimension reduc- tion is used on a typical dataset (RD-256).
Figure 5. Purity as a function of dimension, parameterized by profile length for 3-gram representation when DF based dimension re- duction is used on a typical dataset (RD-256).
Figure 5. Purity as a function of dimension, parameterized by profile length for 3-gram representation when DF based dimension re- duction is used on a typical dataset (RD-256).
Related topics:
Independent Component AnalysisDimension ReductionText ClusteringLatent Semantic IndexingDocument representation
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved