580 California St., Suite 400
San Francisco, CA, 94104
Figure 1 – uploaded by Alejandro Moreo
Table 1 Performance evaluation of different sets of embeddings on 17 intrinsic-task benchmarks, grouped ac- cording to task (2nd row) and evaluation measure (3rd row). Interestingly, the results obtained with our method (BE-sorted) are, by and large, very similar to the ones obtained on the original corpus (Garbled(0%)), and almost always superior to those obtained by Garbled(5%), thus confirming our initial hypothesis. When character sorting is not used, performance seems to deteriorate as the fraction of garbled word occurrences increases. The results also clearly indicate that Full-sorted fares worse than BE-sorted, thus bringing empirical support to the intuition according to which, for computer-based distributional semantic models, as well as for humans, the first and the last letters should remain in place in order to achieve comparable performance. We are currently investigating this aspect in greater detail. Interestingly, the results obtained with our method (BE-sorted) are, by and large, very similar Table 1