Academia.eduAcademia.edu

Figure 3 – uploaded by Francesco Perono Cacciafoco

See full PDFdownloadDownload figure
Table 4. Hyperparameters and their tuned values for experiments.

Table 4 Hyperparameters and their tuned values for experiments.

Related Figures (171)

Table 1. Summary of related work. As a result, this paper proposes a transformer-based BERT model that leverages self-attention mechanisms, which have demonstrated remarkable efficacy in the context of machine learning and deep learning. The proposed model addresses the problems mentioned in the literature review. They have the ability to comprehend the correlation between consecutive items that are widely separated. The transformers achieved an exceptional performance. Additionally, the performance of the proposed method was
Table 1. Summary of related work. As a result, this paper proposes a transformer-based BERT model that leverages self-attention mechanisms, which have demonstrated remarkable efficacy in the context of machine learning and deep learning. The proposed model addresses the problems mentioned in the literature review. They have the ability to comprehend the correlation between consecutive items that are widely separated. The transformers achieved an exceptional performance. Additionally, the performance of the proposed method was
Figure 1. The workflow diagram of the proposed approach for sentiment classification. The proposed methodology’s workflow is depicted in Figure 1, illustrating the steps involved. Firstly, unstructured tweets related to ChatGPT are collected from Twitter using the Twitter Tweepy API. These tweets undergo several preprocessing steps to ensure cleanliness and remove noise. Lexicon-based techniques are then utilized to assign labels of oositive, negative, or neutral to the tweets. Feature extraction is performed using the Bag of Words (BoW) technique on the labeled dataset. The data is subsequently split into an 30/20 ratio for training and testing purposes. Following model training, evaluation metrics such as accuracy, precision, recall, and the F1 score are employed to analyze the model’s oerformance. Each component of the proposed methodology for sentiment classification is discussed in greater detail in the subsequent sections.
Figure 1. The workflow diagram of the proposed approach for sentiment classification. The proposed methodology’s workflow is depicted in Figure 1, illustrating the steps involved. Firstly, unstructured tweets related to ChatGPT are collected from Twitter using the Twitter Tweepy API. These tweets undergo several preprocessing steps to ensure cleanliness and remove noise. Lexicon-based techniques are then utilized to assign labels of oositive, negative, or neutral to the tweets. Feature extraction is performed using the Bag of Words (BoW) technique on the labeled dataset. The data is subsequently split into an 30/20 ratio for training and testing purposes. Following model training, evaluation metrics such as accuracy, precision, recall, and the F1 score are employed to analyze the model’s oerformance. Each component of the proposed methodology for sentiment classification is discussed in greater detail in the subsequent sections.
Table 2. Dataset statistics after splitting. The most important step in natural language processing (NLP) is the pre-processing ge. It enables us to remove any unnecessary information from our data so that we can oceed to the following processing stage. The Natural Language Toolkit (NLTK), which vides modules, is an open-source Python toolkit that can be used to perform operations ch as tokenization, stemming, classification, etc. The first step in preprocessing is to nvert all textual data to lowercase. Conversion is an essential step in sentiment classifica- n, as the machine considers “ChatGPT” and “chatgpt” as individual words. The dataset ntains text in upper, lower, and sentence case, which the model takes separately, which ects the classification performance as well and makes the data more complex if we do not nvert it all into lowercase. The second step is to remove numbers from the text because 2y do not provide meaningful information and are useless in the decision-making process. e removal of numerical data enhances the quality of the data [44]. The third step is to nove punctuation such as [?,@,#,/,&,%] to increase the quality of the dataset and the rformance of the models. The fourth step is to remove HTML and URL tags that also ovide no important information. The URLs in the text data are meaningless because 2y expand the dataset and require extra computation. It has no impact on the machine ning performance. The fifth step is to remove stopwords like ‘an’, ‘the’, ‘are’, ‘was’, s’, ‘they’, etc., from the tweets during preprocessing. The model’s accuracy improves, d the training process is faster, with only relevant information [44]. Additionally, the re- oval of stopwords allows for a more thorough analysis, which is advantageous for a 1ited dataset [45]. The last step is to perform stemming and lemmatization. The effective- ss of machine learning is slightly influenced by the stemming and lemmatization steps. ter performing all important preprocessing steps, the sample tweets are presented in ble 3.
Table 2. Dataset statistics after splitting. The most important step in natural language processing (NLP) is the pre-processing ge. It enables us to remove any unnecessary information from our data so that we can oceed to the following processing stage. The Natural Language Toolkit (NLTK), which vides modules, is an open-source Python toolkit that can be used to perform operations ch as tokenization, stemming, classification, etc. The first step in preprocessing is to nvert all textual data to lowercase. Conversion is an essential step in sentiment classifica- n, as the machine considers “ChatGPT” and “chatgpt” as individual words. The dataset ntains text in upper, lower, and sentence case, which the model takes separately, which ects the classification performance as well and makes the data more complex if we do not nvert it all into lowercase. The second step is to remove numbers from the text because 2y do not provide meaningful information and are useless in the decision-making process. e removal of numerical data enhances the quality of the data [44]. The third step is to nove punctuation such as [?,@,#,/,&,%] to increase the quality of the dataset and the rformance of the models. The fourth step is to remove HTML and URL tags that also ovide no important information. The URLs in the text data are meaningless because 2y expand the dataset and require extra computation. It has no impact on the machine ning performance. The fifth step is to remove stopwords like ‘an’, ‘the’, ‘are’, ‘was’, s’, ‘they’, etc., from the tweets during preprocessing. The model’s accuracy improves, d the training process is faster, with only relevant information [44]. Additionally, the re- oval of stopwords allows for a more thorough analysis, which is advantageous for a 1ited dataset [45]. The last step is to perform stemming and lemmatization. The effective- ss of machine learning is slightly influenced by the stemming and lemmatization steps. ter performing all important preprocessing steps, the sample tweets are presented in ble 3.
Table 4. Cont.
Table 4. Cont.
Figure 2. The architecture for the proposed sentiment classification. Robustly optimized BERT pretraining (RoBERTa) [72] is a transformer-based model used for various NLP tasks. It was developed in 2019. RoBERTa is a modification of the BERT model to overcome the limitations of the BERT model. RoBERTa is trained on 160 billion words, whereas BERT is trained on only 3.3 billion words. RoBERTa is trained on large data sets, is fast to train, and may use large batch sizes. ROBERTa uses a dynamic masking approach, and BERT uses a static approach.
Figure 2. The architecture for the proposed sentiment classification. Robustly optimized BERT pretraining (RoBERTa) [72] is a transformer-based model used for various NLP tasks. It was developed in 2019. RoBERTa is a modification of the BERT model to overcome the limitations of the BERT model. RoBERTa is trained on 160 billion words, whereas BERT is trained on only 3.3 billion words. RoBERTa is trained on large data sets, is fast to train, and may use large batch sizes. ROBERTa uses a dynamic masking approach, and BERT uses a static approach.
Table 5. Results of machine learning models using VADER and TextBlob techniques.
Table 5. Results of machine learning models using VADER and TextBlob techniques.
Figure 3. Performance of models using the TextBlob and VADER techniques. The X-axis presents the machine learning models that we utilized in this study, and the Y-axis presents the accuracy score. Figure 3. Performance of models using the TextBlob and VADER techniques. The X-axis presents the Table 5 also shows the results of various models using the VADER technique. Us- ing a VADER lexicon-based technique, SVM performs best with an accuracy of 90.72%. The models SGD and GBM both achieved an 89% accuracy score. The model that performs worse, in this case, is KNN, with a 54.38% accuracy. This model also performs poorly on the TextBlob technique. The only model in machine learning that performs with the highest accuracy is SVM with the linear kernel. The accuracy score of various machine learning models using TextBlob and Vader are compared in Figure 3.
Figure 3. Performance of models using the TextBlob and VADER techniques. The X-axis presents the machine learning models that we utilized in this study, and the Y-axis presents the accuracy score. Figure 3. Performance of models using the TextBlob and VADER techniques. The X-axis presents the Table 5 also shows the results of various models using the VADER technique. Us- ing a VADER lexicon-based technique, SVM performs best with an accuracy of 90.72%. The models SGD and GBM both achieved an 89% accuracy score. The model that performs worse, in this case, is KNN, with a 54.38% accuracy. This model also performs poorly on the TextBlob technique. The only model in machine learning that performs with the highest accuracy is SVM with the linear kernel. The accuracy score of various machine learning models using TextBlob and Vader are compared in Figure 3.
Table 6. Results of deep learning models using the TextBlob technique.
Table 6. Results of deep learning models using the TextBlob technique.
Table 7. Results of deep learning models using the VADER technique.
Table 7. Results of deep learning models using the VADER technique.
Table 8. Performance of transformer-based models using the TextBlob technique. Table 9. Performance of transformer-based models using the VADER technique.
Table 8. Performance of transformer-based models using the TextBlob technique. Table 9. Performance of transformer-based models using the VADER technique.
Table 10 shows the correct and wrong predictions by deep learning and BERT models ising the TextBlob. Results are given only for the TextBlob technique, as the models perform vell using the TextBlob technique. Out of 4000 predictions, the RNN made 3614 correct oredictions and 386 wrong predictions. The LSTM made 3718 correct predictions while 282 predictions are wrong. The BiLSTM has 3725 correct and 275 wrong predictions. The GRU shows 3693 correct predictions, compared to 307 wrong ones. Out of 4160 pre- dictions, the XLNet made 3576 correct and 584 wrong predictions. On the other hand, he RoBERTa made 3897 correct and 263 wrong predictions. The BERT made 4015 correct oredictions whereas 146 predictions are wrong. The results demonstrate that the BERT nodel performed better than the machine learning and deep learning models. Only with 2835 correct and 1165 wrong predictions, the only CNN model performed poorly.
Table 10 shows the correct and wrong predictions by deep learning and BERT models ising the TextBlob. Results are given only for the TextBlob technique, as the models perform vell using the TextBlob technique. Out of 4000 predictions, the RNN made 3614 correct oredictions and 386 wrong predictions. The LSTM made 3718 correct predictions while 282 predictions are wrong. The BiLSTM has 3725 correct and 275 wrong predictions. The GRU shows 3693 correct predictions, compared to 307 wrong ones. Out of 4160 pre- dictions, the XLNet made 3576 correct and 584 wrong predictions. On the other hand, he RoBERTa made 3897 correct and 263 wrong predictions. The BERT made 4015 correct oredictions whereas 146 predictions are wrong. The results demonstrate that the BERT nodel performed better than the machine learning and deep learning models. Only with 2835 correct and 1165 wrong predictions, the only CNN model performed poorly.
Table 10. Correct and wrong predictions by various models using the TextBlob technique.
Table 10. Correct and wrong predictions by various models using the TextBlob technique.
Table 10. Cont. 4.4. Results of K-Fold Cross-Validation
Table 10. Cont. 4.4. Results of K-Fold Cross-Validation
Table 11. K-fold cross-Validation results using TextBlob and VADER approaches.
Table 11. K-fold cross-Validation results using TextBlob and VADER approaches.
Figure 4. Comparison of LDA-based and BERT-based topic modeling techniques through word clouds: (a) Visualization of tweets using LDA topic modeling, and (b) Visualization of tweets using BERTopic modeling.
Figure 4. Comparison of LDA-based and BERT-based topic modeling techniques through word clouds: (a) Visualization of tweets using LDA topic modeling, and (b) Visualization of tweets using BERTopic modeling.
Figure 5. Most Prominent Topics extracted from ChatGPT Tweets using BERTopic. Figure 5 depicts the most prominent topics extracted by BERTopic. First, we load the BERT model and associated tokenizers. The tweet data are then preprocessed to extract the embeddings for the BERT model. Then, for dimension reduction or clustering, we used k-means clustering and the principal component analysis (PCA). The BERT model was used to extract the most prominent topics, which were then displayed in a scatter plot.
Figure 5. Most Prominent Topics extracted from ChatGPT Tweets using BERTopic. Figure 5 depicts the most prominent topics extracted by BERTopic. First, we load the BERT model and associated tokenizers. The tweet data are then preprocessed to extract the embeddings for the BERT model. Then, for dimension reduction or clustering, we used k-means clustering and the principal component analysis (PCA). The BERT model was used to extract the most prominent topics, which were then displayed in a scatter plot.
Figure 6. Words extracted from top ten topics with their frequency using the LDA model.
Figure 6. Words extracted from top ten topics with their frequency using the LDA model.
Figure 7. Visualization of highly discussed positive topics.
Figure 7. Visualization of highly discussed positive topics.
Figure 8. Visualization of highly discussed negative topics.
Figure 8. Visualization of highly discussed negative topics.
Table 12. Statistical test comparison with the proposed model.
Table 12. Statistical test comparison with the proposed model.
Table 13. Comparison of proposed approach with state-of-the-art existing studies. 4.8. Validation of Proposed Approach on Additional Dataset
Table 13. Comparison of proposed approach with state-of-the-art existing studies. 4.8. Validation of Proposed Approach on Additional Dataset
Table 14. Experimental results on the SemEvel2013 dataset. 4.9. Statistical Significance Test
Table 14. Experimental results on the SemEvel2013 dataset. 4.9. Statistical Significance Test
Figure 9. Sentiment ratio in extracted data. In this study, we observed that the majority of sentiment towards chatGPT was positive, indicating a generally favorable perception of the tool. This aligns with the notion that chatGPT has gained significant attention and popularity on various online platforms. The positive sentiment towards chatGPT can be attributed to its advanced language generation capabilities and its ability to engage in human-like conversations. Figure 9 shows the sentiment ratio for chatGPT.
Figure 9. Sentiment ratio in extracted data. In this study, we observed that the majority of sentiment towards chatGPT was positive, indicating a generally favorable perception of the tool. This aligns with the notion that chatGPT has gained significant attention and popularity on various online platforms. The positive sentiment towards chatGPT can be attributed to its advanced language generation capabilities and its ability to engage in human-like conversations. Figure 9 shows the sentiment ratio for chatGPT.
Table 15. Statistical significance t-test. The t-test can be interpreted as if the output p-value is greater than the alpha value (0.05), it indicates that the H, is accepted and there is no statistical significance. Moreover, if the p-value is less than the alpha value, it indicates that Ho is rejected and H, is accepted which means that there is statistical significance between the compared results. We perform a t-test on results using Textblob and compare all models’ performances. In all scenarios, the proposed approach rejects the Hy and accepted the Hz, which means that the proposed approach is statistically significant in comparison with other approaches.
Table 15. Statistical significance t-test. The t-test can be interpreted as if the output p-value is greater than the alpha value (0.05), it indicates that the H, is accepted and there is no statistical significance. Moreover, if the p-value is less than the alpha value, it indicates that Ho is rejected and H, is accepted which means that there is statistical significance between the compared results. We perform a t-test on results using Textblob and compare all models’ performances. In all scenarios, the proposed approach rejects the Hy and accepted the Hz, which means that the proposed approach is statistically significant in comparison with other approaches.
Figure 10. SentimentViz output for chatGPT sentiment. Additionally, we conducted an analysis using an external sentiment analysis tool called SentimentViz [78]. This tool allowed us to visualize people’s perceptions of ChatGPT based on their data. The sentiment analysis results obtained from SentimentViz comple- mented and validated the findings of the proposed approach. Figure 10 presents visual representations of the sentiment expressed by individuals regarding ChatGPT. This visu- alization provides further support for the positive sentiment observed in our study and reinforces the credibility of our results.
Figure 10. SentimentViz output for chatGPT sentiment. Additionally, we conducted an analysis using an external sentiment analysis tool called SentimentViz [78]. This tool allowed us to visualize people’s perceptions of ChatGPT based on their data. The sentiment analysis results obtained from SentimentViz comple- mented and validated the findings of the proposed approach. Figure 10 presents visual representations of the sentiment expressed by individuals regarding ChatGPT. This visu- alization provides further support for the positive sentiment observed in our study and reinforces the credibility of our results.
Figure 1. Architecture of SPARSAR with main pipeline organized into three levels. processed at first at a syntactic and semantic level and grammatical functions are evaluated Then, the poem is translated into a phonetic form, preserving its visual structure and its subdivision into verses and stanzas. Phonetically translated words are associated with mean duration values taking into account position in the word and stress. At the end of the analysis of the poem, the system can measure the following parameters: mean verse length in terms of msec. and in number of feet. The latter is derived by a verse representation into metrical structure. Another important component of the analysis of rhythm is constituted by the algorithm that measures and evaluates rhyme schemes at the stanza level and then the overall rhyming structure at the poem level. In addition, the system has access to a restricted list of typical pragmatically marked phrases and expressions that are used to convey specific discourse function and speech acts, and need specialized intonational contours.
Figure 1. Architecture of SPARSAR with main pipeline organized into three levels. processed at first at a syntactic and semantic level and grammatical functions are evaluated Then, the poem is translated into a phonetic form, preserving its visual structure and its subdivision into verses and stanzas. Phonetically translated words are associated with mean duration values taking into account position in the word and stress. At the end of the analysis of the poem, the system can measure the following parameters: mean verse length in terms of msec. and in number of feet. The latter is derived by a verse representation into metrical structure. Another important component of the analysis of rhythm is constituted by the algorithm that measures and evaluates rhyme schemes at the stanza level and then the overall rhyming structure at the poem level. In addition, the system has access to a restricted list of typical pragmatically marked phrases and expressions that are used to convey specific discourse function and speech acts, and need specialized intonational contours.
Table 1. Distribution of sounds of end-of-line rhyming words divided into four phonological classes.
Table 1. Distribution of sounds of end-of-line rhyming words divided into four phonological classes.
Table 2. Subdivision of the sonnets by number of classes. There is one sonnet with only one class and it is sonnet 146; then, there are 13 sonnets with 2 classes of sounds: 8, 9, 64, 71, 79, 81, 87, 90, 92, 96, 124, and 149. These sonnets contain rhyming pairs with low and middle sounds, except for three sonnets: sonnet 71 which contains high-back and middle sounds; sonnet 9 which contains high-front and low sounds; and sonnet 96 containing high-front and middle sounds. The themes developed in these sonnets fit perfectly into the rhyming sound class chosen. Let us consider sonnet VIII which is all devoted to music and string instruments which require more than one string to produce their sound, thus suggesting the need to find a companion and get married. Consider the line “the true concord of well tunéd sounds,” where hints to the need that sounds should be “well” tuned. Sonnet 81 celebrates the poet and his verse which shall survive when death will come. Sonnet 92 is in fact pessimistic in the possibility that love will last “for the term of life” and no betrayal will ensue. As to sonnet 146, it is a mixture of two seemingly different themes: a criticism of extravagant display or rich clothing of wealth by writers of the time, or perhaps his mistress and trying to convince her to change her ways for eternal salvation. Some critics regard this as the most profoundly religious or meditative sonnet. But, the feeling of the lover renouncing something brings back his mistress and the feeling of being powerless against her chastity, so that religious life becomes a desirable aim. In this sense, death can also be depicted as desirable. It is important to notice the overall strategy of choice of sound in relation to meaning,
Table 2. Subdivision of the sonnets by number of classes. There is one sonnet with only one class and it is sonnet 146; then, there are 13 sonnets with 2 classes of sounds: 8, 9, 64, 71, 79, 81, 87, 90, 92, 96, 124, and 149. These sonnets contain rhyming pairs with low and middle sounds, except for three sonnets: sonnet 71 which contains high-back and middle sounds; sonnet 9 which contains high-front and low sounds; and sonnet 96 containing high-front and middle sounds. The themes developed in these sonnets fit perfectly into the rhyming sound class chosen. Let us consider sonnet VIII which is all devoted to music and string instruments which require more than one string to produce their sound, thus suggesting the need to find a companion and get married. Consider the line “the true concord of well tunéd sounds,” where hints to the need that sounds should be “well” tuned. Sonnet 81 celebrates the poet and his verse which shall survive when death will come. Sonnet 92 is in fact pessimistic in the possibility that love will last “for the term of life” and no betrayal will ensue. As to sonnet 146, it is a mixture of two seemingly different themes: a criticism of extravagant display or rich clothing of wealth by writers of the time, or perhaps his mistress and trying to convince her to change her ways for eternal salvation. Some critics regard this as the most profoundly religious or meditative sonnet. But, the feeling of the lover renouncing something brings back his mistress and the feeling of being powerless against her chastity, so that religious life becomes a desirable aim. In this sense, death can also be depicted as desirable. It is important to notice the overall strategy of choice of sound in relation to meaning,
Table 3. Total count for vowel, final consonants and sonorant sounds organized into classes for all Shakespearean sonnets. Eventually, we come up with 61 more frequent heads with occurrences up to four and a total of 778 repeated vowel and consonant line-ending sounds. We now consider the remaining 288 rhyming pairs organized into “head” and “dependent”, i.e., the preceding end of the line’s rhyming word and the one in the corresponding alternate /adjacent end of line.
Table 3. Total count for vowel, final consonants and sonorant sounds organized into classes for all Shakespearean sonnets. Eventually, we come up with 61 more frequent heads with occurrences up to four and a total of 778 repeated vowel and consonant line-ending sounds. We now consider the remaining 288 rhyming pairs organized into “head” and “dependent”, i.e., the preceding end of the line’s rhyming word and the one in the corresponding alternate /adjacent end of line.
Table 4. (a) Distribution of stressed rhyming vowels in five phases. (b) Weighted values of the distribution of stressed rhyming vowels in five Phases.
Table 4. (a) Distribution of stressed rhyming vowels in five phases. (b) Weighted values of the distribution of stressed rhyming vowels in five Phases.
Table 5. (a) Distribution of stressed diphthongs in the sonnets divided in 5 phases. (b) Weighted valued of the distribution of stressed diphthongs in the sonnets in 5 phases. Both Phases 1 and 4 show a decrease of middle vs. low diphthongs, while the remain- ing three phases behave in the opposite manner: more middle than low diphthongs. The total distribution indicates Phase 3 as the highest number of diphthongs and Phase 4 as the lowest, just the opposite of the previous distribution. General totals show a distribution of middle vs. low diphthongs which is strongly in favour of middle ones. This is just the opposite of what we found in previous counts, and in part then compensates with the lack of high diphthongs.
Table 5. (a) Distribution of stressed diphthongs in the sonnets divided in 5 phases. (b) Weighted valued of the distribution of stressed diphthongs in the sonnets in 5 phases. Both Phases 1 and 4 show a decrease of middle vs. low diphthongs, while the remain- ing three phases behave in the opposite manner: more middle than low diphthongs. The total distribution indicates Phase 3 as the highest number of diphthongs and Phase 4 as the lowest, just the opposite of the previous distribution. General totals show a distribution of middle vs. low diphthongs which is strongly in favour of middle ones. This is just the opposite of what we found in previous counts, and in part then compensates with the lack of high diphthongs.
Table 6. Sound image of the sonnets. 3.2. Rhyming and Rhythm: The Sonnets and Poetic Devices 3.2.1. Contractions vs. Rhyme Schemes
Table 6. Sound image of the sonnets. 3.2. Rhyming and Rhythm: The Sonnets and Poetic Devices 3.2.1. Contractions vs. Rhyme Schemes
Table 7. Number of rhyme violations x five phases. We call these (pseudo) rhyming violations because current reciters available on Youtube do not dare use the old pronunciation required and produce a rhyming vio- lation by using Modern English pronunciation. One of these reciters is the famous actor John Gilgoud, who when reading Sonnet 66, correctly pronounces DESERT with its original meaning, but then in Sonnet 116 produces three violations when rhyming pairs required transformations that were clearly mandatory in Early Modern English, and they are | love | to be pronounced with the vowel of |remove! in lines 2/4, |come! to be pronounced with the vowel of |doom| in lines 10/12, and | loved! to be pronounced with the vowel of | proved | in the couplet. How do we know that these words should be pronounced in that manner and not in the opposite way—say | remove]! as |love!, |doom! as |come! and | proved! as | loved |, as is being asserted by Ben Crystal son of David? There are three criteria that determine the way in which words should rhyme: the first one is the rhyming constraints which were so stringent at the time owing to the fact that poetry was only recited and not read on books. Okay, then, there are rhyming constraints but how do they work, in which direction? The direction is determined by two factors: the first one is determined by universal phonological principles, as for instance the one the governs phonological variations of vowel sounds—in the vowel shift of verbs or nouns due to morphological changes—which systematically changed “low” and “mid” features into “high” features and not vice versa [32]. The other factor is simply lexical: i.e., not all words will be subject to a transformation in that period. As a result, some words had double pronunciation. This was extensively documented in books and articles published at the time and written by famous poets like Ben Jonson and a great number of grammarians of the XVI and XVII century. All this information is made available by the famous historical phonologist Wilhelm Vietor of the XIX century in a book published at first in 1889 (2 (we use 1909 Vol 2. edition that can be freely visualized at: https:/ /books.google.it/books?id=rh™EQAWAAQBAJ&printsec= frontcover&hl=it&source=gbs_ge_summary_ré&cad=0#v=onepageé&q&f=false accessed on 6 July 2023), by the title “A Shakespeare Phonology” which we have adopted as our refer- ence. Variants are then lexically determined. Some words involved in the transformation are listed below using ARPAbet as the phonetic alphabet in the excerpt taken from the lexicon. As can be easily noticed, variants are related also to stress position, but also to consonant sounds. T exvicon 1
Table 7. Number of rhyme violations x five phases. We call these (pseudo) rhyming violations because current reciters available on Youtube do not dare use the old pronunciation required and produce a rhyming vio- lation by using Modern English pronunciation. One of these reciters is the famous actor John Gilgoud, who when reading Sonnet 66, correctly pronounces DESERT with its original meaning, but then in Sonnet 116 produces three violations when rhyming pairs required transformations that were clearly mandatory in Early Modern English, and they are | love | to be pronounced with the vowel of |remove! in lines 2/4, |come! to be pronounced with the vowel of |doom| in lines 10/12, and | loved! to be pronounced with the vowel of | proved | in the couplet. How do we know that these words should be pronounced in that manner and not in the opposite way—say | remove]! as |love!, |doom! as |come! and | proved! as | loved |, as is being asserted by Ben Crystal son of David? There are three criteria that determine the way in which words should rhyme: the first one is the rhyming constraints which were so stringent at the time owing to the fact that poetry was only recited and not read on books. Okay, then, there are rhyming constraints but how do they work, in which direction? The direction is determined by two factors: the first one is determined by universal phonological principles, as for instance the one the governs phonological variations of vowel sounds—in the vowel shift of verbs or nouns due to morphological changes—which systematically changed “low” and “mid” features into “high” features and not vice versa [32]. The other factor is simply lexical: i.e., not all words will be subject to a transformation in that period. As a result, some words had double pronunciation. This was extensively documented in books and articles published at the time and written by famous poets like Ben Jonson and a great number of grammarians of the XVI and XVII century. All this information is made available by the famous historical phonologist Wilhelm Vietor of the XIX century in a book published at first in 1889 (2 (we use 1909 Vol 2. edition that can be freely visualized at: https:/ /books.google.it/books?id=rh™EQAWAAQBAJ&printsec= frontcover&hl=it&source=gbs_ge_summary_ré&cad=0#v=onepageé&q&f=false accessed on 6 July 2023), by the title “A Shakespeare Phonology” which we have adopted as our refer- ence. Variants are then lexically determined. Some words involved in the transformation are listed below using ARPAbet as the phonetic alphabet in the excerpt taken from the lexicon. As can be easily noticed, variants are related also to stress position, but also to consonant sounds. T exvicon 1
Table 8. Rhyme repetition rates in three Elizabethan poets. Table 9. Rhyme repetition word class-frequency distribution for Shakespeare’s sonnets.
Table 8. Rhyme repetition rates in three Elizabethan poets. Table 9. Rhyme repetition word class-frequency distribution for Shakespeare’s sonnets.
Now, let us consider the distribution of rhyming words into the corpus of the sonnets. As to general frequency data, the Sonnets contain a number of tokens equal to 18,283 with 3085 types, so-called Vocabulary Richness that is used to measure the ability of a writer to use different words in a corpus, corresponds to 16.87%, a high value for that time when compared with other poets. Also, the number of Hapax and Rare Words (indicating the union of Hapax, Dis and TrisLegomena) corresponds to average values for other poets, respectively to 56%, the first type, and 79%, the second one. If we look at similar data for
Now, let us consider the distribution of rhyming words into the corpus of the sonnets. As to general frequency data, the Sonnets contain a number of tokens equal to 18,283 with 3085 types, so-called Vocabulary Richness that is used to measure the ability of a writer to use different words in a corpus, corresponds to 16.87%, a high value for that time when compared with other poets. Also, the number of Hapax and Rare Words (indicating the union of Hapax, Dis and TrisLegomena) corresponds to average values for other poets, respectively to 56%, the first type, and 79%, the second one. If we look at similar data for
Figure 2. The eleven most positively marked sonnets: 7, 24, 43, 47, 52, 76, 85, 87, 128, 136, 154. As will appear clearly from the charts below, all the data show a contrasting behaviour which will be attested by correlation values. Where sentiment values increase, the cor- responding values for vowels and consonants decrease. To allow better perusing of the trends we split the sonnets into separate tables according to whether their sentiment values are positive or negative. The first chart contains the eleven sonnets which received the highest positive sentiment values. All the charts are drawn from the tables of data derived from the analysis files in xml format, which will be made available as supplementary data (please see Figure 2).
Figure 2. The eleven most positively marked sonnets: 7, 24, 43, 47, 52, 76, 85, 87, 128, 136, 154. As will appear clearly from the charts below, all the data show a contrasting behaviour which will be attested by correlation values. Where sentiment values increase, the cor- responding values for vowels and consonants decrease. To allow better perusing of the trends we split the sonnets into separate tables according to whether their sentiment values are positive or negative. The first chart contains the eleven sonnets which received the highest positive sentiment values. All the charts are drawn from the tables of data derived from the analysis files in xml format, which will be made available as supplementary data (please see Figure 2).
Table 10. Quantitative data for six appraisal classes for sonnets with highest contrast. We report for each word frequency type in column 1—there is only one head word thee) with frequency 28—the corresponding number of tokens in Table 9, followed by he sum of tokens, the incremental sum and the corresponding percentage with respect o total corpus. As can be noticed from the last column, where incremental percent of hyme-pair words corpus coverage is reported, the total of rare words, i.e., type rhyme-pair vith frequency of occurrence lower than 4, is 62.59%, a fairly low value if compared to he measure evaluated on simple type/token ratios. If we look at most important English oets, as documented in a previous paper , we can see that the average value for Rare Nords is 77.88%. However, we are here dealing with rhyming words and the comparison nay not be so relevant.
Table 10. Quantitative data for six appraisal classes for sonnets with highest contrast. We report for each word frequency type in column 1—there is only one head word thee) with frequency 28—the corresponding number of tokens in Table 9, followed by he sum of tokens, the incremental sum and the corresponding percentage with respect o total corpus. As can be noticed from the last column, where incremental percent of hyme-pair words corpus coverage is reported, the total of rare words, i.e., type rhyme-pair vith frequency of occurrence lower than 4, is 62.59%, a fairly low value if compared to he measure evaluated on simple type/token ratios. If we look at most important English oets, as documented in a previous paper , we can see that the average value for Rare Nords is 77.88%. However, we are here dealing with rhyming words and the comparison nay not be so relevant.
Figure 3. Chart of the 16 borderline sonnets positively marked for sentiment. In this chart we added the ratio for Abstract/Concrete, which shows a peak for son- net 73. As the chart clearly shows, the line for Sentiment borders 1, as to the remaining variables, Vowels is the one oscillating most after Abstract. Voiced and Consonants are fairly always aligned apart from sonnet 33 and 102. In both sonnets, the number of “Ob- struents” (|b,d,p,t,k,g |) is very low and real consonants are substituted by “Continuants” (|s,sh,th,f,v,h|) both voiced and unvoiced. In the following analysis, for this reason, I will only consider Voicing as the relevant variable for consonants and this will show better agreement in the overall data. Now, we show charts for all negatively marked sonnets using only three variables, starting from Figure 4 below.
Figure 3. Chart of the 16 borderline sonnets positively marked for sentiment. In this chart we added the ratio for Abstract/Concrete, which shows a peak for son- net 73. As the chart clearly shows, the line for Sentiment borders 1, as to the remaining variables, Vowels is the one oscillating most after Abstract. Voiced and Consonants are fairly always aligned apart from sonnet 33 and 102. In both sonnets, the number of “Ob- struents” (|b,d,p,t,k,g |) is very low and real consonants are substituted by “Continuants” (|s,sh,th,f,v,h|) both voiced and unvoiced. In the following analysis, for this reason, I will only consider Voicing as the relevant variable for consonants and this will show better agreement in the overall data. Now, we show charts for all negatively marked sonnets using only three variables, starting from Figure 4 below.
Figure 4. Chart of the 42 negatively marked sonnets: 3, 8, 9, 19, 28, 30, 34, 35, 50, 55, 57, 58, 60, 62, 63, 65, 66, 71, 86, 89, 92, 103, 107, 112, 116, 120, 121, 124, 126, 127, 129, 132, 133, 134, 138, 139, 140, 143, 146, 148, 149.
Figure 4. Chart of the 42 negatively marked sonnets: 3, 8, 9, 19, 28, 30, 34, 35, 50, 55, 57, 58, 60, 62, 63, 65, 66, 71, 86, 89, 92, 103, 107, 112, 116, 120, 121, 124, 126, 127, 129, 132, 133, 134, 138, 139, 140, 143, 146, 148, 149.
Figure 5. The eleven most positively marked sonnets show the same slightly positive correlatio for Vowels—Voicing but very strong negative correlation between Vowels-Sentiment and slight! negative for Voicing /Sentiment at —0.11423482—colours in this case have no meaning. Figure 5. The eleven most positively marked sonnets show the same slightly positive correlatior Correlation between Vowel and Sentiment is positive but very weak; correlation be- tween the Voicing parameter and Sentiment is again negative and very weak at —0.0065037. Thus, results for the 42 sonnets negatively marked by sentiment show that we have negative correlation between vowels and voicing, and vowels and sentiment, but positive correlation between voicing and sentiment. So, it is just the opposite of what we obtain with positively marked sonnets. And finally, in Figure 5. we show the eleven most positively marked sonnets show the same contrasting results.
Figure 5. The eleven most positively marked sonnets show the same slightly positive correlatio for Vowels—Voicing but very strong negative correlation between Vowels-Sentiment and slight! negative for Voicing /Sentiment at —0.11423482—colours in this case have no meaning. Figure 5. The eleven most positively marked sonnets show the same slightly positive correlatior Correlation between Vowel and Sentiment is positive but very weak; correlation be- tween the Voicing parameter and Sentiment is again negative and very weak at —0.0065037. Thus, results for the 42 sonnets negatively marked by sentiment show that we have negative correlation between vowels and voicing, and vowels and sentiment, but positive correlation between voicing and sentiment. So, it is just the opposite of what we obtain with positively marked sonnets. And finally, in Figure 5. we show the eleven most positively marked sonnets show the same contrasting results.
Figure 6. The 44 sonnets classified with Sarcasm with the highest level of Judgements—colours in this case have no meaning.
Figure 6. The 44 sonnets classified with Sarcasm with the highest level of Judgements—colours in this case have no meaning.
Table 11. Quantitative data for six appraisal classes for sonnets with lowest contrast.
Table 11. Quantitative data for six appraisal classes for sonnets with lowest contrast.
Figure 7. The 50 sonnets classified with Irony, with a lower level of Judgement Negative but higher Affect Negative.
Figure 7. The 50 sonnets classified with Irony, with a lower level of Judgement Negative but higher Affect Negative.
Figure 8. The 60 Sonnets classified by critics as neutral.
Figure 8. The 60 Sonnets classified by critics as neutral.
Figure 9. Distribution of 89 sonnets manually classified by ATF with no contrast. 23a — RAP Agee PH
Figure 9. Distribution of 89 sonnets manually classified by ATF with no contrast. 23a — RAP Agee PH
Table 12. Quantitative data for six appraisal classes for sonnets with no contrast. a a I I I a The experiment with ATF classes matching critics’ evaluation has been fairly successful, but how do these classes gauge with the Sound—-Sense harmony? In order to check this, we transferred the data related to vowels and consonants and matched them with ratios of the three main ATF categories: Appreciation Positive/Negative, Affect Positive/Negative, and Judgement Positive/Negative. As in previous computation, all data below 1 will be interpreted as a case of superior Negative Polarity and the opposite when data are above 1. To allow a better view of the overall data, we split them into sonnets with contrast to the first group that we show in Figure 9, and sonnets with no contrast to the second group, that we show in Figure 10. This time, however, we used our classification and abandoned the critics’ one.
Table 12. Quantitative data for six appraisal classes for sonnets with no contrast. a a I I I a The experiment with ATF classes matching critics’ evaluation has been fairly successful, but how do these classes gauge with the Sound—-Sense harmony? In order to check this, we transferred the data related to vowels and consonants and matched them with ratios of the three main ATF categories: Appreciation Positive/Negative, Affect Positive/Negative, and Judgement Positive/Negative. As in previous computation, all data below 1 will be interpreted as a case of superior Negative Polarity and the opposite when data are above 1. To allow a better view of the overall data, we split them into sonnets with contrast to the first group that we show in Figure 9, and sonnets with no contrast to the second group, that we show in Figure 10. This time, however, we used our classification and abandoned the critics’ one.
Figure 10. Distribution of 65 sonnets classified as Judgements with contrast and their sound data. All correlation measures with Judgements are negative:
Figure 10. Distribution of 65 sonnets classified as Judgements with contrast and their sound data. All correlation measures with Judgements are negative:
Figure 11. Distribution of 65 sonnets classified by ATF as Affect with contrast and their sound data. In Figure 11 below we use again sound data and the second parameter Affect:
Figure 11. Distribution of 65 sonnets classified by ATF as Affect with contrast and their sound data. In Figure 11 below we use again sound data and the second parameter Affect:
Figure 12. Distribution of 65 sonnets classified by ATF as Appraisal with contrast and their sound data.
Figure 12. Distribution of 65 sonnets classified by ATF as Appraisal with contrast and their sound data.
Comparing the “contrast” criterion with the sentiment-based classification is not pos- sible; however, the “contrast” group of sonnets is included in majority by the “negatively” marked sonnets, with the exception of 16 sonnets which are the following ones: Now the only positive correlations are the ones shown by Affect with Vowels and with Appraisal; the remaining correlations are all negative. The subdivision operated now using our manual classification with ATF seems more consistent than the one made before using the critics’ evaluation. As a first comment, these data confirm our previous evaluation made on the basis of sentiment analysis, i.e., the sonnets are mainly disharmonic due to Shakespeare’s intention to produce ironic effects on the audience. Here below is the list of the 89 sonnets classified by our manual ATF labeling as having no contrast:
Comparing the “contrast” criterion with the sentiment-based classification is not pos- sible; however, the “contrast” group of sonnets is included in majority by the “negatively” marked sonnets, with the exception of 16 sonnets which are the following ones: Now the only positive correlations are the ones shown by Affect with Vowels and with Appraisal; the remaining correlations are all negative. The subdivision operated now using our manual classification with ATF seems more consistent than the one made before using the critics’ evaluation. As a first comment, these data confirm our previous evaluation made on the basis of sentiment analysis, i.e., the sonnets are mainly disharmonic due to Shakespeare’s intention to produce ironic effects on the audience. Here below is the list of the 89 sonnets classified by our manual ATF labeling as having no contrast:
Figure 13. Poems considered as deviants evaluated for their degree of sense /sound harmony. WNT OER TAFE MANA EM SETI BE GSS | MEET ences ava | SE REOTOERY ier em I, In addition to the evaluation of positive /negative values, we consider the two parameters we already computed related to Metrical Length and Rhyming Scheme that we add together and use for its 10% added value to compensate for poetic relevant features. On the basis of poetic devices analyzed by SPARSAR, a list of 14 poems is considered as deviant, and they are the following: A Sunrise, The Gunner, The Explorer’s Wife, For My Grandfather, Idyll, Middle Harbour, Politician, To a Poet, The Captain of the Oberon, Palace of Dreams, The Room, Vancouver by Rail, Henry Lawson, and Achilles and the Woman. In Figure 13 we show the first map of sense-sound evaluation where the split of the “deviants” poems appears clearly:
Figure 13. Poems considered as deviants evaluated for their degree of sense /sound harmony. WNT OER TAFE MANA EM SETI BE GSS | MEET ences ava | SE REOTOERY ier em I, In addition to the evaluation of positive /negative values, we consider the two parameters we already computed related to Metrical Length and Rhyming Scheme that we add together and use for its 10% added value to compensate for poetic relevant features. On the basis of poetic devices analyzed by SPARSAR, a list of 14 poems is considered as deviant, and they are the following: A Sunrise, The Gunner, The Explorer’s Wife, For My Grandfather, Idyll, Middle Harbour, Politician, To a Poet, The Captain of the Oberon, Palace of Dreams, The Room, Vancouver by Rail, Henry Lawson, and Achilles and the Woman. In Figure 13 we show the first map of sense-sound evaluation where the split of the “deviants” poems appears clearly:
Figure 14. Sixteen poems from different periods of Webb’s poetic production computed for their Sense/Sound Harmony.
Figure 14. Sixteen poems from different periods of Webb’s poetic production computed for their Sense/Sound Harmony.
Figure 15. Sixteen poems taken mainly from late poetic production computed for their sense /sound harmony. In Figure 16, I will now show a bigger picture containing 50 poems, where we can see again the great majority of them being positioned on the left hand side. The positive side is enriched by “Moonlight” from Early Poems, and “Song of the Brain” from Socrates, and the middle disharmonic list now counts 16 poems.
Figure 15. Sixteen poems taken mainly from late poetic production computed for their sense /sound harmony. In Figure 16, I will now show a bigger picture containing 50 poems, where we can see again the great majority of them being positioned on the left hand side. The positive side is enriched by “Moonlight” from Early Poems, and “Song of the Brain” from Socrates, and the middle disharmonic list now counts 16 poems.
Figure 16. Fifty poems computed by sense/sound harmony.
Figure 16. Fifty poems computed by sense/sound harmony.
Table 1. “Shallow” annotation output by UDPipe. Sentence: “It gives us the basis for several deductions” (Doyle, The Hound of the Baskervilles, 1901). Table 2. “Deep” annotation output by UDPipe. Sentence: “It gives us the basis for several deductions (Doyle, The Hound of the Baskervilles, 1901).
Table 1. “Shallow” annotation output by UDPipe. Sentence: “It gives us the basis for several deductions” (Doyle, The Hound of the Baskervilles, 1901). Table 2. “Deep” annotation output by UDPipe. Sentence: “It gives us the basis for several deductions (Doyle, The Hound of the Baskervilles, 1901).
Table 3. “Shallow” annotation by UDPipe. Sentence: “There, however, stood only a single bowl” (Spyri, Heidi, 1880).
Table 3. “Shallow” annotation by UDPipe. Sentence: “There, however, stood only a single bowl” (Spyri, Heidi, 1880).
Table 4. “Deep” annotation by UDPipe. Sentence: “There, however, stood only a single bowl” (Spyri, Heidi, 1880). For each token, the analysis gives the form as it appears in the text and its lemma. This information is not used in the method described here since our goal is to examine the discriminative power of morphosyntactic features. In addition, as noted above, general vo- cabulary may be largely dependent on genre or subject matter and may confound analysis. It is worth noting that the elimination of word forms and lemmas from consideration simpli- fies preprocessing and, to some degree, compensates for the time required to extract input features from the parsed text. Minimal clean-up of the .txt file is required; chapter titles and the like can be left in the document without affecting the results of the classification.
Table 4. “Deep” annotation by UDPipe. Sentence: “There, however, stood only a single bowl” (Spyri, Heidi, 1880). For each token, the analysis gives the form as it appears in the text and its lemma. This information is not used in the method described here since our goal is to examine the discriminative power of morphosyntactic features. In addition, as noted above, general vo- cabulary may be largely dependent on genre or subject matter and may confound analysis. It is worth noting that the elimination of word forms and lemmas from consideration simpli- fies preprocessing and, to some degree, compensates for the time required to extract input features from the parsed text. Minimal clean-up of the .txt file is required; chapter titles and the like can be left in the document without affecting the results of the classification.
Figure 1. Universal Dependency tree for “It gives us the basis for several deductions”. structure of an entire sentence, as visualized in a dependency tree such as the one showr below (Figure 1). The syntactic “path” from the sentence root to each “leaf” token is given by the ‘combination of head id and dependency relationship. The syntactic function of each worc s clearly and specifically defined by these two values. For example, the word basis is the bj of the word gives. obj is the UD label for what is traditionally called the “direct object of a verb (a list of syntax labels along with examples can be found on the UD website ittps: / /universaldependencies.org/en/dep/index.html, accessed 1 January 2024). The vord several is labeled as amod of deductions. amod indicates an adjectival modifier. The worc leductions itself is an nmod of basis. In UD annotation, nmod means “nominal modifier”, < 1oun or noun phrase directly dependent on and specifying another noun (or noun phrase ‘or example, the prepositional phrase in “toys for children”.
Figure 1. Universal Dependency tree for “It gives us the basis for several deductions”. structure of an entire sentence, as visualized in a dependency tree such as the one showr below (Figure 1). The syntactic “path” from the sentence root to each “leaf” token is given by the ‘combination of head id and dependency relationship. The syntactic function of each worc s clearly and specifically defined by these two values. For example, the word basis is the bj of the word gives. obj is the UD label for what is traditionally called the “direct object of a verb (a list of syntax labels along with examples can be found on the UD website ittps: / /universaldependencies.org/en/dep/index.html, accessed 1 January 2024). The vord several is labeled as amod of deductions. amod indicates an adjectival modifier. The worc leductions itself is an nmod of basis. In UD annotation, nmod means “nominal modifier”, < 1oun or noun phrase directly dependent on and specifying another noun (or noun phrase ‘or example, the prepositional phrase in “toys for children”.
Table 5. Number of input features by number of type-value components in each feature have selected among them based on frequency alone. For each combination length, only those type—value pairs which occur in approximately 5% of the tokens in the corpus have been included as input variables for classification. The process of populating feature type: with their values is computationally slow for combinations of more than two elements so we have used a smaller sample corpus for each language. Thus, the 5% cut-off i: an approximation. A separate set of variables has been identified in this way for eact language. Because UDPipe produces different types of morphological annotation, anc because syntactic annotation, although it largely consists of the same relationship labels has different frequency distributions in various languages, the same selection procedure with the same 5% cut-off results in a different quantity of features for each language. Detail: are given in Table 5.
Table 5. Number of input features by number of type-value components in each feature have selected among them based on frequency alone. For each combination length, only those type—value pairs which occur in approximately 5% of the tokens in the corpus have been included as input variables for classification. The process of populating feature type: with their values is computationally slow for combinations of more than two elements so we have used a smaller sample corpus for each language. Thus, the 5% cut-off i: an approximation. A separate set of variables has been identified in this way for eact language. Because UDPipe produces different types of morphological annotation, anc because syntactic annotation, although it largely consists of the same relationship labels has different frequency distributions in various languages, the same selection procedure with the same 5% cut-off results in a different quantity of features for each language. Detail: are given in Table 5.
Table 6. Results of classification by individual novel (45 classes). Clearly, the works in each corpus are sharply distinguishable at the morphosyntactic level. Unfortunately, there is little published research to which these results may usefully be compared. Generally, recent stylometric research has a quasi-forensic tendency, focused on the ability to “prove” authorship of particular texts. In such cases, there is no reason to examine the discriminability of the individual works of an author. In contrast, our interest is in the descriptive value of stylometric measures as applied to works as well as authors Our assumption in this study is that input features that both discriminate texts clearly and are understandable in terms of traditional stylistics may serve as the basis of valuable stylometric descriptions. Our results indicate that discriminability is high even with the relatively small 500-word samples; this success can be taken as an indication that a good deal of stylistic information is in fact conveyed by the features that we have proposed We will examine some of the most important of these distinguishing features in the next section. It is worth mentioning here that the same procedure (albeit with different input features for each corpus) works quite well for each language tested. In fact, it is apparent from the 500-word samples that the morphosyntactic signal is somewhat stronger the more morphologically complex the language is. This complexity is reflected in the number of features as reported in Table 5: a sharper distinction seems to exist between works in Polish which has 1137 total input features, compared to between works in English (653 total features).
Table 6. Results of classification by individual novel (45 classes). Clearly, the works in each corpus are sharply distinguishable at the morphosyntactic level. Unfortunately, there is little published research to which these results may usefully be compared. Generally, recent stylometric research has a quasi-forensic tendency, focused on the ability to “prove” authorship of particular texts. In such cases, there is no reason to examine the discriminability of the individual works of an author. In contrast, our interest is in the descriptive value of stylometric measures as applied to works as well as authors Our assumption in this study is that input features that both discriminate texts clearly and are understandable in terms of traditional stylistics may serve as the basis of valuable stylometric descriptions. Our results indicate that discriminability is high even with the relatively small 500-word samples; this success can be taken as an indication that a good deal of stylistic information is in fact conveyed by the features that we have proposed We will examine some of the most important of these distinguishing features in the next section. It is worth mentioning here that the same procedure (albeit with different input features for each corpus) works quite well for each language tested. In fact, it is apparent from the 500-word samples that the morphosyntactic signal is somewhat stronger the more morphologically complex the language is. This complexity is reflected in the number of features as reported in Table 5: a sharper distinction seems to exist between works in Polish which has 1137 total input features, compared to between works in English (653 total features).
Table 7. Results of leave-one-out classification by author (15 classes). The sharp decrease in classification accuracy is striking. Presumably, an explanation is to be found in the greatly increased difficulty of the problem. The results of the most closely comparable previous studies point to the same conclusion. Maciej Eder has published three important studies on authorship attribution [19,26,27] in which the corpora are similar to our own. The accuracy of Eder’s experiments is consistent with our results. For example, Eder (2010) classifies samples of various sizes drawn from 63 English novels; for samples of around 1000 words, accuracy falls between 40% and 50%. A more precise comparison is unfortunately not possible. All three of Eder’s works present their results in graphs rather than tables. Thus, only rough estimates for the accuracy of a given sample size are possible. Most of Eder’s data are based on the most frequent words. For a corpus of 66 German novels, samples ranging from 500 to 2000 words seem to yield accuracy scores from 30% to 60%. Evidently, the low accuracy of our authorship attribution tests (as compared to novel- by-novel classification) is not anomalous. Furthermore, it does not seem likely that the combination of input features and classifier that was quite good at identifying individual novels would become uninformative about the authorship of those same works. The field
Table 7. Results of leave-one-out classification by author (15 classes). The sharp decrease in classification accuracy is striking. Presumably, an explanation is to be found in the greatly increased difficulty of the problem. The results of the most closely comparable previous studies point to the same conclusion. Maciej Eder has published three important studies on authorship attribution [19,26,27] in which the corpora are similar to our own. The accuracy of Eder’s experiments is consistent with our results. For example, Eder (2010) classifies samples of various sizes drawn from 63 English novels; for samples of around 1000 words, accuracy falls between 40% and 50%. A more precise comparison is unfortunately not possible. All three of Eder’s works present their results in graphs rather than tables. Thus, only rough estimates for the accuracy of a given sample size are possible. Most of Eder’s data are based on the most frequent words. For a corpus of 66 German novels, samples ranging from 500 to 2000 words seem to yield accuracy scores from 30% to 60%. Evidently, the low accuracy of our authorship attribution tests (as compared to novel- by-novel classification) is not anomalous. Furthermore, it does not seem likely that the combination of input features and classifier that was quite good at identifying individual novels would become uninformative about the authorship of those same works. The field
Table 8. Selection of input features “preferred” in Oliver Twist. A few examples will help to illustrate the phenomena underlying these values. The first feature is grammatically transparent. This sentence from Oliver Twist has two examples: “They talked of hope and comfort”. The two bold-faced nouns are annotated with feature #1; obviously singular, they are preceded by their dependency parent, talked. Although this dependency—a noun upon a verb—is the most frequent structure annotated with feature
Table 8. Selection of input features “preferred” in Oliver Twist. A few examples will help to illustrate the phenomena underlying these values. The first feature is grammatically transparent. This sentence from Oliver Twist has two examples: “They talked of hope and comfort”. The two bold-faced nouns are annotated with feature #1; obviously singular, they are preceded by their dependency parent, talked. Although this dependency—a noun upon a verb—is the most frequent structure annotated with feature
Table 9. Input features “avoided” in Oliver Twist. Feature #1B represents dependencies of the infinitive form of the verb. The English infinitive is morphologically the same as the dictionary lemma. It primarily occurs in one of two configurations. An infinitive can be “introduced” by the particle to, as in the following examples: “I have come out myself to take him there”; and “... the parish would like him to learn a right pleasant trade ...”. It is apparent that to plus the infinitive has a wide range of syntactic functions. In the first example, to take expresses the purpose for which the action of the main verb was undertaken. The infinitive phrase can be deleted from the sentence without making it ungrammatical. In contrast, the syntax (and semantics) of like in In addition to input features that are strongly “preferred” in Oliver Twist, there are others that are sharply “avoided”. We will look only at three of the most important, as given in Table 9.
Table 9. Input features “avoided” in Oliver Twist. Feature #1B represents dependencies of the infinitive form of the verb. The English infinitive is morphologically the same as the dictionary lemma. It primarily occurs in one of two configurations. An infinitive can be “introduced” by the particle to, as in the following examples: “I have come out myself to take him there”; and “... the parish would like him to learn a right pleasant trade ...”. It is apparent that to plus the infinitive has a wide range of syntactic functions. In the first example, to take expresses the purpose for which the action of the main verb was undertaken. The infinitive phrase can be deleted from the sentence without making it ungrammatical. In contrast, the syntax (and semantics) of like in In addition to input features that are strongly “preferred” in Oliver Twist, there are others that are sharply “avoided”. We will look only at three of the most important, as given in Table 9.
Table 10. Summary of standard deviations of input features for selected authors.
Table 10. Summary of standard deviations of input features for selected authors.
Table 11. Selected input features preferred or avoided by the class “Dickens”.
Table 11. Selected input features preferred or avoided by the class “Dickens”.
Table 12. Selected input features where frequency variability weakens the “Dickens” signal. BH = Bleak House, GE = Great Expectations and OT = Oliver Twist.
Table 12. Selected input features where frequency variability weakens the “Dickens” signal. BH = Bleak House, GE = Great Expectations and OT = Oliver Twist.
Figure 1. This work’s overview. Six methods are used to machine-generate papers, which are then mixed with human-written ones to create our benchmark dataset. Seven models are then tested as baselines to identify the authorship of a given output. Figure 1. This work’s overview. Six methods are used to machine-generate papers, which are then
Figure 1. This work’s overview. Six methods are used to machine-generate papers, which are then mixed with human-written ones to create our benchmark dataset. Seven models are then tested as baselines to identify the authorship of a given output. Figure 1. This work’s overview. Six methods are used to machine-generate papers, which are then
Table 1. Data sources included in our dataset and their respective sizes. 3.1. Real Papers Collection
Table 1. Data sources included in our dataset and their respective sizes. 3.1. Real Papers Collection
3.2. Fake Papers Generation AUS This combination of models, ranging from CFG to state-of-the-art LLMs, aims to generate a diverse set of artificially generated scientific papers. Concrete examples of generated papers can be found in Appendix A.
3.2. Fake Papers Generation AUS This combination of models, ranging from CFG to state-of-the-art LLMs, aims to generate a diverse set of artificially generated scientific papers. Concrete examples of generated papers can be found in Appendix A.
Figure 3. Our co-created test dataset TEST-CC contains 4000 papers with varying shares of real and ChatGPT-paraphrased sections. The co-created component of our dataset mimics papers written by humans and models concurrently, a combination that is likely to appear in practice. That means texts originally written by either a human or an LLM and subsequently extended, paraphrased, or otherwise adjusted by the other. To create such papers at scale, we take a set of 4000 real papers from our TEST dataset (see Table 2) and paraphrase them with ChatGPT [8]. To stay within ChatGPT’s context length limits, we paraphrase each paper section—i.e., abstract, introduction, and conclusion—in a separate prompt. We then construct co-created papers with varying shares of human and machine input by combining original and paraphrased sections as shown in Figure 3.
Figure 3. Our co-created test dataset TEST-CC contains 4000 papers with varying shares of real and ChatGPT-paraphrased sections. The co-created component of our dataset mimics papers written by humans and models concurrently, a combination that is likely to appear in practice. That means texts originally written by either a human or an LLM and subsequently extended, paraphrased, or otherwise adjusted by the other. To create such papers at scale, we take a set of 4000 real papers from our TEST dataset (see Table 2) and paraphrase them with ChatGPT [8]. To stay within ChatGPT’s context length limits, we paraphrase each paper section—i.e., abstract, introduction, and conclusion—in a separate prompt. We then construct co-created papers with varying shares of human and machine input by combining original and paraphrased sections as shown in Figure 3.
Table 2. Overview of the datasets used to train and evaluate the classifiers. Each column represents the number of papers used per source. Concerning real papers, unless indicated, we use samples extracted with parsing 1 (see Section 3.1).
Table 2. Overview of the datasets used to train and evaluate the classifiers. Each column represents the number of papers used per source. Concerning real papers, unless indicated, we use samples extracted with parsing 1 (see Section 3.1).
Figure 4. LLMFE follows a four-step process: (1) Generate features suitable for distinguishing real and fake papers using the LLM based on multiple pairs of one real and one fake paper each. (2) Remove duplicate features through hierarchical clustering on embeddings of the feature descriptions. (3) Score scientific papers along the remaining features using the LLM. (4) Finally, train a Random Forest Classifier to predict the real or fake label based on the feature scores.
Figure 4. LLMFE follows a four-step process: (1) Generate features suitable for distinguishing real and fake papers using the LLM based on multiple pairs of one real and one fake paper each. (2) Remove duplicate features through hierarchical clustering on embeddings of the feature descriptions. (3) Score scientific papers along the remaining features using the LLM. (4) Finally, train a Random Forest Classifier to predict the real or fake label based on the feature scores.
Table 3. Experiment results reported with accuracy metric. Out-of-domain experiments, i.e., evalu- ation on unseen generators, are highlighted in blue. Highest values per test set are highlighted in bold. (*) ChatGPT-IO and LLMFE accuracies have been evaluated on randomly sampled subsets of 100 scientific papers per test set due to API limits.
Table 3. Experiment results reported with accuracy metric. Out-of-domain experiments, i.e., evalu- ation on unseen generators, are highlighted in blue. Highest values per test set are highlighted in bold. (*) ChatGPT-IO and LLMFE accuracies have been evaluated on randomly sampled subsets of 100 scientific papers per test set due to API limits.
Figure 6 shows the distribution of Flesch-Kincaid Grade Level [69] and Gunning Fog [70] readability metrics [71] for papers from the different generators and real papers. Flesch-Kincaid measures the technical difficulty of the papers, while Gunning Fog mea- sures the readability of the papers. The comparison confirms our observation that our machine-generated papers are representative of real papers with a slight increase in writing sophistication from SCIgen and GPT-2 to ChatGPT and GPT-3 generators, with Galactica being the median.
Figure 6 shows the distribution of Flesch-Kincaid Grade Level [69] and Gunning Fog [70] readability metrics [71] for papers from the different generators and real papers. Flesch-Kincaid measures the technical difficulty of the papers, while Gunning Fog mea- sures the readability of the papers. The comparison confirms our observation that our machine-generated papers are representative of real papers with a slight increase in writing sophistication from SCIgen and GPT-2 to ChatGPT and GPT-3 generators, with Galactica being the median.
Figure 7. The generators exhibit different tendencies for the length of the generated fake scientific papers. (a) shows the length distribution of generated abstracts, (b) shows the same for introductions, and (c) shows conclusion lengths.
Figure 7. The generators exhibit different tendencies for the length of the generated fake scientific papers. (a) shows the length distribution of generated abstracts, (b) shows the same for introductions, and (c) shows conclusion lengths.
Figure 8. Paraphrasing sections with ChatGPT has a tendency to result in sections shorter than the original. The reduction in section length is most visible for the longer introduction and conclusion sections. For an analysis of lengths of generated fake scientific papers, see Figure 7 in the appendix.
Figure 8. Paraphrasing sections with ChatGPT has a tendency to result in sections shorter than the original. The reduction in section length is most visible for the longer introduction and conclusion sections. For an analysis of lengths of generated fake scientific papers, see Figure 7 in the appendix.
Figure Al. Example 1 of prompting ChatGPT to produce sections of a scientific paper given the paper title.
Figure Al. Example 1 of prompting ChatGPT to produce sections of a scientific paper given the paper title.
Figure A2. Example 2 of prompting ChatGPT to produce sections of a scientific paper given the paper title.
Figure A2. Example 2 of prompting ChatGPT to produce sections of a scientific paper given the paper title.
Table A1. Hyperparameters used to generate each paper section in the Galactica model. Each row corresponds to a decoding of a section based on the previous input sections. Here we used parameters of the MODEL.GENERATE function provided by Huggingface [74].
Table A1. Hyperparameters used to generate each paper section in the Galactica model. Each row corresponds to a decoding of a section based on the previous input sections. Here we used parameters of the MODEL.GENERATE function provided by Huggingface [74].
Table A2. Experiment results for the different bag-of-words classifiers reported with accuracy metric. Out-of-domain experiments are highlighted in blue. The highest values per test set are highlighted in bold. Appendix B.2. GPT-3
Table A2. Experiment results for the different bag-of-words classifiers reported with accuracy metric. Out-of-domain experiments are highlighted in blue. The highest values per test set are highlighted in bold. Appendix B.2. GPT-3
Figure A3. Extract from the hierarchical clustering dendrogram learned during the feature consolida- tion step of LLMFE. The full dendrogram lists all 884 features. The distance threshold was chosen so that 83 clusters were created from the 884 features. Figure A3. Extract from the hierarchical clustering dendrogram learned during the feature consolida-
Figure A3. Extract from the hierarchical clustering dendrogram learned during the feature consolida- tion step of LLMFE. The full dendrogram lists all 884 features. The distance threshold was chosen so that 83 clusters were created from the 884 features. Figure A3. Extract from the hierarchical clustering dendrogram learned during the feature consolida-
Table A3. Experiment results for different ChatGPT prompting styles reported with accuracy metric. Out-of-domain experiments are highlighted in blue. Highest values per test set are highlighted in bold. (*) ChatGPT accuracies have been evaluated on randomly sampled subsets of 100 scientific papers per test set and prompting style due to API limits. Table A3. Experiment results for different ChatGPT prompting styles reported with accuracy metric.
Table A3. Experiment results for different ChatGPT prompting styles reported with accuracy metric. Out-of-domain experiments are highlighted in blue. Highest values per test set are highlighted in bold. (*) ChatGPT accuracies have been evaluated on randomly sampled subsets of 100 scientific papers per test set and prompting style due to API limits. Table A3. Experiment results for different ChatGPT prompting styles reported with accuracy metric.
Figure A5. Explainability insights from our Logistic Regression (LR) and Random Forest (RF)
Figure A5. Explainability insights from our Logistic Regression (LR) and Random Forest (RF)
Appendix C.2. RoBERTa
Appendix C.2. RoBERTa
Figure A12. RoBERTa: Example of SHAP explanation on a ChatGPT generated abstract correctly classified.
Figure A12. RoBERTa: Example of SHAP explanation on a ChatGPT generated abstract correctly classified.
Figure A15. RoBERTa: Example of LIME explanation on a GPT-2 generated abstract correctly classified. Figure A14. RoBERTa: Example of LIME explanation on a SClgen generated abstract correctly classified.
Figure A15. RoBERTa: Example of LIME explanation on a GPT-2 generated abstract correctly classified. Figure A14. RoBERTa: Example of LIME explanation on a SClgen generated abstract correctly classified.
Figure A18. Galactica: Example of SHAP explanation on a real paper correctly classified. Appendix C.3. Galactica
Figure A18. Galactica: Example of SHAP explanation on a real paper correctly classified. Appendix C.3. Galactica
Figure A19. Galactica: Example of SHAP explanation on a misclassified real paper.
Figure A19. Galactica: Example of SHAP explanation on a misclassified real paper.
Figure A20. Galactica: Example of SHAP explanation on a Galactica generated paper correctly classified.
Figure A20. Galactica: Example of SHAP explanation on a Galactica generated paper correctly classified.
Figure A21. Galactica: Example of SHAP explanation on a misclassified Galactica generated paper.
Figure A21. Galactica: Example of SHAP explanation on a misclassified Galactica generated paper.
_ _ de Oo _ YY - ne YY _ For a clear and systematic picture aiming to aid the reader with understanding this Paper, a concept map illustrating a summary of the main topics and their relations discussed in our work is shown in Figure 1. Figure 1. Concept map of the main topics and their relations discussed in the paper.
_ _ de Oo _ YY - ne YY _ For a clear and systematic picture aiming to aid the reader with understanding this Paper, a concept map illustrating a summary of the main topics and their relations discussed in our work is shown in Figure 1. Figure 1. Concept map of the main topics and their relations discussed in the paper.
Table 1. Overview of the linguistic markers for depression extracted in the selected papers. Source: Own work. Table 2. Overview of the linguistic markers for dementia extracted in the selected papers. Source: Own work.
Table 1. Overview of the linguistic markers for depression extracted in the selected papers. Source: Own work. Table 2. Overview of the linguistic markers for dementia extracted in the selected papers. Source: Own work.
4.3. Relevant Datasets
4.3. Relevant Datasets
Table 3. Overview of the linguistic markers for hallucinations from people extracted in the selected papers. Source: Own work. Table 4. Overview of the linguistic markers for artificial hallucinations extracted in the selected papers. Source: Own work.
Table 3. Overview of the linguistic markers for hallucinations from people extracted in the selected papers. Source: Own work. Table 4. Overview of the linguistic markers for artificial hallucinations extracted in the selected papers. Source: Own work.
4.3.1. Depression
4.3.1. Depression
Table 1. Features extracted from Profiling—UD. There have been increasingly large collections of data compiled across the internet. With advancements in technologies, these datasets are annotated and automatically anal- ysed for multiple purposes [22]. However, linguistic profiling of texts is usually carried out for multiple different projects with a variety of end goals in mind. Language verification, author identification and verification, and text classification are a few to highlight here. Our focus is to identify specific linguistic features of a given text that influence the text classification into genres and specific subgenres. A brief review of the studies which have focused on linguistic profiling of fictional and non-fictional texts points to the study by [11], where they tried to estimate the readability of Italian fictional prose based on the linguistic profiling of the texts. Even though their study shows promising results, from a fictional prose point of view the dataset considered in the study is devoid of the fictional texts or does not cover most of the subgenres of the fictional type. Therefore, it is very important to conduct studies that consider multiple fictional subgenres that are popularly noted in the literature and compare their linguistic composition with the non-fictional text type. In the study by [11], the four major categories considered were literature further divided into children and adult literature, journalism (newspaper), educational writing (educational materials for primary school and high school) and scientific prose. When we look at the datasets which are utilized across literature for the task of classification or readability or
Table 1. Features extracted from Profiling—UD. There have been increasingly large collections of data compiled across the internet. With advancements in technologies, these datasets are annotated and automatically anal- ysed for multiple purposes [22]. However, linguistic profiling of texts is usually carried out for multiple different projects with a variety of end goals in mind. Language verification, author identification and verification, and text classification are a few to highlight here. Our focus is to identify specific linguistic features of a given text that influence the text classification into genres and specific subgenres. A brief review of the studies which have focused on linguistic profiling of fictional and non-fictional texts points to the study by [11], where they tried to estimate the readability of Italian fictional prose based on the linguistic profiling of the texts. Even though their study shows promising results, from a fictional prose point of view the dataset considered in the study is devoid of the fictional texts or does not cover most of the subgenres of the fictional type. Therefore, it is very important to conduct studies that consider multiple fictional subgenres that are popularly noted in the literature and compare their linguistic composition with the non-fictional text type. In the study by [11], the four major categories considered were literature further divided into children and adult literature, journalism (newspaper), educational writing (educational materials for primary school and high school) and scientific prose. When we look at the datasets which are utilized across literature for the task of classification or readability or
Table 2. Summary of the dataset of the study. Hence, we built a dataset which consists of both fictional and non-fictional texts with a special focus on carrying out a detailed linguistic analysis. Table 2 highlights the number of text samples (shown in brackets) considered in each subgenre grouped across fictional and non-fictional genres. The selected texts were divided into chapters, and it was made sure that the overall size of each of the texts would be around 100-2000 words. Preprocessing of the selected text was carried out to remove licensing information, unnecessary spaces and punctuation.
Table 2. Summary of the dataset of the study. Hence, we built a dataset which consists of both fictional and non-fictional texts with a special focus on carrying out a detailed linguistic analysis. Table 2 highlights the number of text samples (shown in brackets) considered in each subgenre grouped across fictional and non-fictional genres. The selected texts were divided into chapters, and it was made sure that the overall size of each of the texts would be around 100-2000 words. Preprocessing of the selected text was carried out to remove licensing information, unnecessary spaces and punctuation.
Table 3. Summary of the raw textual features across genres. Table 4. Summary of the lexical variety features across genres.
Table 3. Summary of the raw textual features across genres. Table 4. Summary of the lexical variety features across genres.
Similarly, we looked at the parts of speech distribution in the various subgenres. Table 3 highlights the individual values of the distribution of parts of speech across various subgenres. When the values are compared across fiction and non-fictional texts, it can be noted that fictional texts have a lower number of adjectives but a higher number of adverbs, adpositions, pronouns and punctuation when compared to non-fictional texts. Whereas non-fictional texts have two times higher values of auxiliary verbs and nouns with slightly elevated values in numbers compared to fictional texts. No significant differences were noted in the values of coordinating and subordinating conjunctions, determiners, interjections, symbols and pronouns across fictional and non-fictional texts. Overall, the lexical density of fictional and non-fictional texts remained the same. Table 5 highlights the parts of speech distribution across all subgenres.
Similarly, we looked at the parts of speech distribution in the various subgenres. Table 3 highlights the individual values of the distribution of parts of speech across various subgenres. When the values are compared across fiction and non-fictional texts, it can be noted that fictional texts have a lower number of adjectives but a higher number of adverbs, adpositions, pronouns and punctuation when compared to non-fictional texts. Whereas non-fictional texts have two times higher values of auxiliary verbs and nouns with slightly elevated values in numbers compared to fictional texts. No significant differences were noted in the values of coordinating and subordinating conjunctions, determiners, interjections, symbols and pronouns across fictional and non-fictional texts. Overall, the lexical density of fictional and non-fictional texts remained the same. Table 5 highlights the parts of speech distribution across all subgenres.
When we carefully examine the parts of speech distribution across non-fictional texts, we can note that instructional texts had significantly fewer adjectives, adverbs and auxiliary verbs when compared to other non-fictional texts. Also, the concentration of proper nouns in instructional texts is statistically higher than in any other text. Persuasive texts have a statistically significant fewer number of nouns, punctuation and adpositions, but higher values in pronouns and verbs overall. Based on the values of determiners, particle structure, subordinate conjunctions and interjections, we can group non-fictional texts into two groups: discussion—persuasive and explanatory-—instructional. No significant differences were noted in the lexical density across all the subgenres. Therefore, it can be noted that open class and closed class words are equally important in the classification of texts into fictional and non-fictional genres.
When we carefully examine the parts of speech distribution across non-fictional texts, we can note that instructional texts had significantly fewer adjectives, adverbs and auxiliary verbs when compared to other non-fictional texts. Also, the concentration of proper nouns in instructional texts is statistically higher than in any other text. Persuasive texts have a statistically significant fewer number of nouns, punctuation and adpositions, but higher values in pronouns and verbs overall. Based on the values of determiners, particle structure, subordinate conjunctions and interjections, we can group non-fictional texts into two groups: discussion—persuasive and explanatory-—instructional. No significant differences were noted in the lexical density across all the subgenres. Therefore, it can be noted that open class and closed class words are equally important in the classification of texts into fictional and non-fictional genres.
Table 6. Feature details after dimensionality reduction. 4.2. Constructing RF Models
Table 6. Feature details after dimensionality reduction. 4.2. Constructing RF Models
Figure 2. Variable importance plot of the RF genre model. NOTE: The x-axis shows the permutation relevance (mean decrease in accuracy) of each feature; the y-axis lists the features of the genre model.
Figure 2. Variable importance plot of the RF genre model. NOTE: The x-axis shows the permutation relevance (mean decrease in accuracy) of each feature; the y-axis lists the features of the genre model.
Figure 3. Variable importance plot of the RF sub-genre model. NOTE: The x-axis shows the per- mutation relevance (mean decrease in accuracy) of each feature; the y-axis lists the features of the subgenre model.
Figure 3. Variable importance plot of the RF sub-genre model. NOTE: The x-axis shows the per- mutation relevance (mean decrease in accuracy) of each feature; the y-axis lists the features of the subgenre model.
Figure 4. SFS optimal features of each feature set.
Figure 4. SFS optimal features of each feature set.
Table 7. Subgenre selection of top features. Table 8. Genre selection of top features.
Table 7. Subgenre selection of top features. Table 8. Genre selection of top features.
Table 9. Accuracy of the model with feature selection.
Table 9. Accuracy of the model with feature selection.
precise meaning of a polysemous word. Notably, the 1980s witnessed significant progress in WSD research, facilitated by the availability of extensive lexical resources and corpora. Ultimately, WSD entails the task of identifying the accurate sense of a word within its specific contextual framework [3]. WSD is not considered a final objective; instead, it is recognized as an intermediary task with relevance to various applications within the field of NLP. Figure 1 presents the WSD conceptual diagram. In machine translation, WSD is an important step because a number of words in every language have a different translation according to the context of their usage [3-6]. It is an important issue to be considered during language translation. WSD assumes a crucial role in ensuring precise text analysis across a wide range of applications [7,8]. For example, an intelligence-gathering system could distinguish between references to illicit drugs and medicinal drugs through the application of WSD. Research works such as named entity recognition and bioinformatics research can also use WSD. In the realm of information retrieval (IR), the primary concern lies in determining the accurate sense of a polysemous word within a given query before initiating the search for its corresponding answer [9,10]. Enhancing the efficiency and effectiveness of an IR system entails the resolution of ambiguity within a query. Similarly, in sentiment analysis, the elimination of ambiguity is crucial for determining the correct sentiment tags (e.g., negative or positive) associated with a sentence [11,12]. In question-answering (QA) systems, WSD assumes a significant role in identifying the appropriate types of answers that correspond to a given question type [13,14]. Furthermore, WSD is necessary to accurately assign the appropriate part of speech tagging (POS) to a word, as its POS can vary depending on the contextual usage [15,16]. ei gag oes 1 ie eww # ae egy aw a ae 1. =e a = 1
precise meaning of a polysemous word. Notably, the 1980s witnessed significant progress in WSD research, facilitated by the availability of extensive lexical resources and corpora. Ultimately, WSD entails the task of identifying the accurate sense of a word within its specific contextual framework [3]. WSD is not considered a final objective; instead, it is recognized as an intermediary task with relevance to various applications within the field of NLP. Figure 1 presents the WSD conceptual diagram. In machine translation, WSD is an important step because a number of words in every language have a different translation according to the context of their usage [3-6]. It is an important issue to be considered during language translation. WSD assumes a crucial role in ensuring precise text analysis across a wide range of applications [7,8]. For example, an intelligence-gathering system could distinguish between references to illicit drugs and medicinal drugs through the application of WSD. Research works such as named entity recognition and bioinformatics research can also use WSD. In the realm of information retrieval (IR), the primary concern lies in determining the accurate sense of a polysemous word within a given query before initiating the search for its corresponding answer [9,10]. Enhancing the efficiency and effectiveness of an IR system entails the resolution of ambiguity within a query. Similarly, in sentiment analysis, the elimination of ambiguity is crucial for determining the correct sentiment tags (e.g., negative or positive) associated with a sentence [11,12]. In question-answering (QA) systems, WSD assumes a significant role in identifying the appropriate types of answers that correspond to a given question type [13,14]. Furthermore, WSD is necessary to accurately assign the appropriate part of speech tagging (POS) to a word, as its POS can vary depending on the contextual usage [15,16]. ei gag oes 1 ie eww # ae egy aw a ae 1. =e a = 1
Various approaches and methods used for WSD are classified into two categories, including knowledge-based approaches and ML (Machine Learning) based approaches. In knowledge-driven approaches, external lexical resources such as Wordnet, dictionary, and thesauri are required to perform WSD, and in ML-based techniques, classifiers are trained to carry out the WSD task on sense-annotated corpora. Figure 2 presents the different WSD approaches, and the explanation for each category can be explained further.
Various approaches and methods used for WSD are classified into two categories, including knowledge-based approaches and ML (Machine Learning) based approaches. In knowledge-driven approaches, external lexical resources such as Wordnet, dictionary, and thesauri are required to perform WSD, and in ML-based techniques, classifiers are trained to carry out the WSD task on sense-annotated corpora. Figure 2 presents the different WSD approaches, and the explanation for each category can be explained further.
Figure 3. Decision Tree Example. A decision tree is a classification method that repeatedly divides the training dataset ind organizes the classification rules in a tree-like structure [26,27]. Every interior node »f the decision tree represents a test performed on an attribute value, and the branches epresent the outcomes of the test. The word sense is determined when a leaf node is eached. An illustration of a decision tree for WSD is depicted in Figure 3. In this example, he sense of the polysemous word “bank” that is active is a noun within the sentence, “I vill be at the bank of the Narmada River in the afternoon.” The tree has been constructed ind traversed to ultimately select the sense “bank/RIVER.” A null value in a leaf node ndicates that there is no sense selection present for that particular attribute value.
Figure 3. Decision Tree Example. A decision tree is a classification method that repeatedly divides the training dataset ind organizes the classification rules in a tree-like structure [26,27]. Every interior node »f the decision tree represents a test performed on an attribute value, and the branches epresent the outcomes of the test. The word sense is determined when a leaf node is eached. An illustration of a decision tree for WSD is depicted in Figure 3. In this example, he sense of the polysemous word “bank” that is active is a noun within the sentence, “I vill be at the bank of the Narmada River in the afternoon.” The tree has been constructed ind traversed to ultimately select the sense “bank/RIVER.” A null value in a leaf node ndicates that there is no sense selection present for that particular attribute value.
Figure 4. Illustrating SVM Classification. An SVM [32] serves the purpose of both classification and regression tasks. This approach is rooted in the concept of identifying a hyperplane that can effectively isolate positive examples from negative ones with the highest possible margin. The edge/margin represents the interspace between the hyperplane and the nearest examples for positive and negative, which are referred to as support vectors. In Figure 4, circle and square represent two different classes, the bold line represents the hyperplane that isolates the two classes while the dashed lines indicate the support vectors closest to positive and negative example. These support vectors play an important role in constructing an SVM classifier. The vectors have an impact on the position and the orientation of the hyperplane, and by removing or adding support vectors, adjustments can be made to the position of the hyperplane. In Figure 4,
Figure 4. Illustrating SVM Classification. An SVM [32] serves the purpose of both classification and regression tasks. This approach is rooted in the concept of identifying a hyperplane that can effectively isolate positive examples from negative ones with the highest possible margin. The edge/margin represents the interspace between the hyperplane and the nearest examples for positive and negative, which are referred to as support vectors. In Figure 4, circle and square represent two different classes, the bold line represents the hyperplane that isolates the two classes while the dashed lines indicate the support vectors closest to positive and negative example. These support vectors play an important role in constructing an SVM classifier. The vectors have an impact on the position and the orientation of the hyperplane, and by removing or adding support vectors, adjustments can be made to the position of the hyperplane. In Figure 4,
Figure 5. Ensemble Methods: Combining the Strengths of Multiple Models. In order to enhance the accuracy of disambiguation, it is common to employ a combi- nation of different classifiers. This combination strategy is called ensemble methods, which combine algorithms of different nature or with different characteristics [37]. Ensemble methods are more powerful than single-supervised techniques as they can overcome the weakness of a single approach. Strategies such as majority voting, the AdaBoost system of Freund and Schapire [38], rank-based combination, and probability mixture can be utilized to combine the different classifiers to improve accuracy. Figure 5 presents the simple approach of the ensemble WSD approach. 2.2.2. Unsupervised Techniques Unsupervised techniques do not make use of sense annotated datasets or external knowledge sources. Instead, they operate under the assumption that senses with similar meanings occur in similar contexts. These techniques aim to determine senses from the text by clustering the word occurrences based on some measure of contextual similarity. This task is known as word sense induction or discrimination. Unsupervised techniques offer significant potential in overcoming the bottleneck of knowledge acquisition, as they do not require manual efforts. Here are some approaches that are used for unsupervised WSD.
Figure 5. Ensemble Methods: Combining the Strengths of Multiple Models. In order to enhance the accuracy of disambiguation, it is common to employ a combi- nation of different classifiers. This combination strategy is called ensemble methods, which combine algorithms of different nature or with different characteristics [37]. Ensemble methods are more powerful than single-supervised techniques as they can overcome the weakness of a single approach. Strategies such as majority voting, the AdaBoost system of Freund and Schapire [38], rank-based combination, and probability mixture can be utilized to combine the different classifiers to improve accuracy. Figure 5 presents the simple approach of the ensemble WSD approach. 2.2.2. Unsupervised Techniques Unsupervised techniques do not make use of sense annotated datasets or external knowledge sources. Instead, they operate under the assumption that senses with similar meanings occur in similar contexts. These techniques aim to determine senses from the text by clustering the word occurrences based on some measure of contextual similarity. This task is known as word sense induction or discrimination. Unsupervised techniques offer significant potential in overcoming the bottleneck of knowledge acquisition, as they do not require manual efforts. Here are some approaches that are used for unsupervised WSD.
Table 1. Comparative Analysis of Knowledge-Based, Supervised, Unsupervised, and Semi- Supervised Techniques.
Table 1. Comparative Analysis of Knowledge-Based, Supervised, Unsupervised, and Semi- Supervised Techniques.
3. WSD Execution Process WSD is the task of determining an ambiguous word’s suitable sense based on context. WSD has seen a variety of methods. The majority of methods are based on different statistical methods. A few methods use corpora that have been sense-tagged, while others use unsupervised learning. The flowchart in Figure 6 shows the steps that are performed for WSD.
3. WSD Execution Process WSD is the task of determining an ambiguous word’s suitable sense based on context. WSD has seen a variety of methods. The majority of methods are based on different statistical methods. A few methods use corpora that have been sense-tagged, while others use unsupervised learning. The flowchart in Figure 6 shows the steps that are performed for WSD.
Figure 6. Flowchart of WSD Execution Process.
Figure 6. Flowchart of WSD Execution Process.
Unsupervised techniques are cost-effective, and they use unlabeled data. Thus, they can be used for languages that lack sense-tagged datasets. However, they may struggle with sense overlapping and lack deep semantic interpretation, leading to less precise disam- biguation compared to supervised methods. Data sparsity can also limit their effectiveness, requiring substantial data for satisfactory performance. Evaluating their performance can be challenging without a definitive gold standard for comparison. Combining unsu- pervised techniques with supervised or knowledge-based approaches can address their limitations and enhance overall WSD performance. me 2 oe ee eT OT TE ED eh ic eee el 2 Bw he ee kt beet See Ue. biguation compared to supervised methods. Data sparsity can also limit their effectiveness,
Unsupervised techniques are cost-effective, and they use unlabeled data. Thus, they can be used for languages that lack sense-tagged datasets. However, they may struggle with sense overlapping and lack deep semantic interpretation, leading to less precise disam- biguation compared to supervised methods. Data sparsity can also limit their effectiveness, requiring substantial data for satisfactory performance. Evaluating their performance can be challenging without a definitive gold standard for comparison. Combining unsu- pervised techniques with supervised or knowledge-based approaches can address their limitations and enhance overall WSD performance. me 2 oe ee eT OT TE ED eh ic eee el 2 Bw he ee kt beet See Ue. biguation compared to supervised methods. Data sparsity can also limit their effectiveness,
Table 3. Data Sources available for Hindi WSD.
Table 3. Data Sources available for Hindi WSD.
Figure 1. A structure generated by the S3 system in the third iteration. The F, X, and Y symbols are mapped to the draw forward action, symbols [ and | traditionally represent the save and return to the position actions, and the characters + and — command the cursor to turn by an angle of +27.5°.
Figure 1. A structure generated by the S3 system in the third iteration. The F, X, and Y symbols are mapped to the draw forward action, symbols [ and | traditionally represent the save and return to the position actions, and the characters + and — command the cursor to turn by an angle of +27.5°.
Figure 2. General Outline of the Grammar Inference Algorithm. 2.6. Image Parsing The process of parsing the image into an input sequence is divided into three steps. First, all of the straight lines are detected in the image. Then, all of these lines are connected, building a model of the structure in the image so in the last step we can generate a sequence that accurately describes this model.
Figure 2. General Outline of the Grammar Inference Algorithm. 2.6. Image Parsing The process of parsing the image into an input sequence is divided into three steps. First, all of the straight lines are detected in the image. Then, all of these lines are connected, building a model of the structure in the image so in the last step we can generate a sequence that accurately describes this model.
Figure 3. A few line points cast into virtual space with granularity = 2.
Figure 3. A few line points cast into virtual space with granularity = 2.
Figure 4. General outline of a single iteration of the second phase of the fitness function. nee 2 ae ed vhich gives us a set of vectors [xo Xp wee Xn & Al , where x, and x, is the occurrence ‘ount of terminal symbol in the nth rule and the axiom, respectively. Because the number of ‘ombinations can be large, we can take the simplest ten for the best results. Now we know 1ow many symbols to insert but not where. To avoid exploring all of the possibilities, since he rules’ successors must appear in the target sequence, we can reduce the search space by only using the appropriate subsequences in the target sequence, which is done in step 3. ‘rom the found subsequences, in step 4, the algorithm generates a population for the GA. since the axiom does not appear in the sequence, we only know the number of symbols to »e added but not their positions; therefore, the symbols are randomly inserted. In the last step, a GA finds a system that can recreate the target sequence using the generated initial opulation.
Figure 4. General outline of a single iteration of the second phase of the fitness function. nee 2 ae ed vhich gives us a set of vectors [xo Xp wee Xn & Al , where x, and x, is the occurrence ‘ount of terminal symbol in the nth rule and the axiom, respectively. Because the number of ‘ombinations can be large, we can take the simplest ten for the best results. Now we know 1ow many symbols to insert but not where. To avoid exploring all of the possibilities, since he rules’ successors must appear in the target sequence, we can reduce the search space by only using the appropriate subsequences in the target sequence, which is done in step 3. ‘rom the found subsequences, in step 4, the algorithm generates a population for the GA. since the axiom does not appear in the sequence, we only know the number of symbols to »e added but not their positions; therefore, the symbols are randomly inserted. In the last step, a GA finds a system that can recreate the target sequence using the generated initial opulation.
1 Standard deviation. Table 1. Results of comparison with the algorithm from [14]. 3.2. More Complex Systems
1 Standard deviation. Table 1. Results of comparison with the algorithm from [14]. 3.2. More Complex Systems
Figure 5. Runtime distribution for single rule system. Figure 6. Runtime distribution for two rule system.
Figure 5. Runtime distribution for single rule system. Figure 6. Runtime distribution for two rule system.
To test the replicability of our results, runtime distribution for the GA has been tested on systems with one and two rewriting rules. Test examples were taken from [14]. As seen in Figure 5, for a single rule system, most algorithm executions ran for a similar amount of time, around 100 ms, with very few stragglers that ran for more than 600 ms. For a system with two rules (Figure 6), we can notice similar behavior. However, here, we can see that a significant amount of runs finished quickly, meaning the initial population already contained a candidate with very high fitness. This lets us conclude that the algorithm has a low tendency to get stuck in areas of search space containing candidates with low fitness.
To test the replicability of our results, runtime distribution for the GA has been tested on systems with one and two rewriting rules. Test examples were taken from [14]. As seen in Figure 5, for a single rule system, most algorithm executions ran for a similar amount of time, around 100 ms, with very few stragglers that ran for more than 600 ms. For a system with two rules (Figure 6), we can notice similar behavior. However, here, we can see that a significant amount of runs finished quickly, meaning the initial population already contained a candidate with very high fitness. This lets us conclude that the algorithm has a low tendency to get stuck in areas of search space containing candidates with low fitness.
Figure 8. Fitness progression of our algorithm. Figure 9. Fitness progression of the algorithm from [12].
Figure 8. Fitness progression of our algorithm. Figure 9. Fitness progression of the algorithm from [12].
Figure 7. Hits heatmap of our algorithm.
Figure 7. Hits heatmap of our algorithm.
Figure 10. Hits histograms of the algorithm from [12].
Figure 10. Hits histograms of the algorithm from [12].
Table 1. Elementary features with their corresponding weights. In Revesz [8], the weight of all features is 1. However, in this study, a different set of weights for each feature is used. The weight of each feature is the inverse of its frequency of occurrence across all symbols in Linear A. In other words, a feature that exists in most symbols will have a lower weight compared to a feature that only exists in some. This means that sharing a rarely occurring feature is given more importance than sharing a commonly occurring one.
Table 1. Elementary features with their corresponding weights. In Revesz [8], the weight of all features is 1. However, in this study, a different set of weights for each feature is used. The weight of each feature is the inverse of its frequency of occurrence across all symbols in Linear A. In other words, a feature that exists in most symbols will have a lower weight compared to a feature that only exists in some. This means that sharing a rarely occurring feature is given more importance than sharing a commonly occurring one.
Table 2. Feature-based similarity scores for a sub-set of symbol pairs.
Table 2. Feature-based similarity scores for a sub-set of symbol pairs.
Table 3. Python program results for Ancient Egyptian. Table 4. Python program results for Luwian.
Table 3. Python program results for Ancient Egyptian. Table 4. Python program results for Luwian.
Table 5. Python program results for Hittite. Table 6. Python program results for Proto-Celtic.
Table 5. Python program results for Hittite. Table 6. Python program results for Proto-Celtic.
Table 7. Python program results for Uralic.
Table 7. Python program results for Uralic.
6. Discussion anguages, suggesting that the result is coincidental rather than indicative of concrete links. The limited number of matches could be due to the phonetic values used for the comparison. The feature-based similarity measure, with the parameters utilized in this paper, was only successful in producing 43 matches for comparing Linear A with other anguages. In contrast, since Linear A and B potentially share 92 similar signs, naturally the phonetic grid based on Linear B includes more signs. There are several reasons for the derived phonetic grid being small. Firstly, it could simply indicate a lack of concrete inks between the scripts. Secondly, while the feature-based similarity measure allows for an analysis of different writing systems, it is not without its limitations. The method depends highly on the elementary feature set, and since we only had a few features, it is plausible to assume that certain important features may have been missed during the analysis. Additionally, a small feature set also increases the probability of finding multiple matches for any symbol with the same similarity scores, and breaking the tie becomes a challenging decision. In Revesz [8], for instance, the tie is broken by choosing the symbol that is earlier in the standard ordering of symbols.
6. Discussion anguages, suggesting that the result is coincidental rather than indicative of concrete links. The limited number of matches could be due to the phonetic values used for the comparison. The feature-based similarity measure, with the parameters utilized in this paper, was only successful in producing 43 matches for comparing Linear A with other anguages. In contrast, since Linear A and B potentially share 92 similar signs, naturally the phonetic grid based on Linear B includes more signs. There are several reasons for the derived phonetic grid being small. Firstly, it could simply indicate a lack of concrete inks between the scripts. Secondly, while the feature-based similarity measure allows for an analysis of different writing systems, it is not without its limitations. The method depends highly on the elementary feature set, and since we only had a few features, it is plausible to assume that certain important features may have been missed during the analysis. Additionally, a small feature set also increases the probability of finding multiple matches for any symbol with the same similarity scores, and breaking the tie becomes a challenging decision. In Revesz [8], for instance, the tie is broken by choosing the symbol that is earlier in the standard ordering of symbols.
_ Below we show an enhanced image of the first sign from the right in the third row, our drawing of it, and the Old Hungarian n sign written with a mirror symmetry and an 4 sign:
_ Below we show an enhanced image of the first sign from the right in the third row, our drawing of it, and the Old Hungarian n sign written with a mirror symmetry and an 4 sign:
Here we need to be careful to ignore the engravings that depict part of the back and the belly of a deer. The lines to be ignored are shown in black in our drawing. The seventh sign from the right in the third row is an Old Hungarian ft sign:
Here we need to be careful to ignore the engravings that depict part of the back and the belly of a deer. The lines to be ignored are shown in black in our drawing. The seventh sign from the right in the third row is an Old Hungarian ft sign:
In the enhanced photo, the Old Hungarian 1 is clearly visible. In addition, there are two parallel lines that belong to the head of one of the engraved deers. These lines do not belong to the Old Hungarian inscription and should be ignored. Unfortunately,
In the enhanced photo, the Old Hungarian 1 is clearly visible. In addition, there are two parallel lines that belong to the head of one of the engraved deers. These lines do not belong to the Old Hungarian inscription and should be ignored. Unfortunately,
Table 1. The Altai inscription and its row-by-row transliterations into Old Hungarian and Latin.
Table 1. The Altai inscription and its row-by-row transliterations into Old Hungarian and Latin.
ae Se eee In the fourth line, there are additional missing details in SartkoZauly’s drawing. The first sign from the left is missing its top half, the diamond sign misses on side, and in the second word, which is written with smaller signs, the third sign from the right misses a small horizontal crossing line segment. These can also be verified by a careful observation of the original photo of the Altai inscription. In addition, the following ligature was also overlooked: 3.2. Transliteration and Translation of the Altai Inscription
ae Se eee In the fourth line, there are additional missing details in SartkoZauly’s drawing. The first sign from the left is missing its top half, the diamond sign misses on side, and in the second word, which is written with smaller signs, the third sign from the right misses a small horizontal crossing line segment. These can also be verified by a careful observation of the original photo of the Altai inscription. In addition, the following ligature was also overlooked: 3.2. Transliteration and Translation of the Altai Inscription
Table 1. Cont. There are certain peculiarities in the Latin transliteration that we made in order to ob- tain meaningful words. In particular, we believe that the scribe was not using the standard Old Hungarian signs but mixed up some of the similar looking signs. In particular, the scribe mixed up the Old Hungarian letters for r and z, which are the following, respectively:
Table 1. Cont. There are certain peculiarities in the Latin transliteration that we made in order to ob- tain meaningful words. In particular, we believe that the scribe was not using the standard Old Hungarian signs but mixed up some of the similar looking signs. In particular, the scribe mixed up the Old Hungarian letters for r and z, which are the following, respectively:
Figure 2. An alternative drawing of inscription. Here the red lines are those that seem extra to the letters that are apparently needed for a meaningful reading of the inscription.
Figure 2. An alternative drawing of inscription. Here the red lines are those that seem extra to the letters that are apparently needed for a meaningful reading of the inscription.
Table 1. The Old Hungarian Runic script with its Hungarian transliteration.
Table 1. The Old Hungarian Runic script with its Hungarian transliteration.
3. Data Sources and Data Curation KarZaubaj SartkoZauly’s drawing had some minor inaccuracies. He included a photo- graph in his work. A new drawing based on that photo is shown in Figure 1. The drawing shows that some parts of the inscription are unclear because of the drawings of the deer and some cracks in the rock.
3. Data Sources and Data Curation KarZaubaj SartkoZauly’s drawing had some minor inaccuracies. He included a photo- graph in his work. A new drawing based on that photo is shown in Figure 1. The drawing shows that some parts of the inscription are unclear because of the drawings of the deer and some cracks in the rock.
Figure 1. The author’s redrawing of the inscription based on the photograph in SartkoZauly [4].
Figure 1. The author’s redrawing of the inscription based on the photograph in SartkoZauly [4].
Figure 2. An enhanced drawing of the inscription with red highlighting of those elements that undisputedly belong to the inscription. The six sign groups are also labeled (a-f).
Figure 2. An enhanced drawing of the inscription with red highlighting of those elements that undisputedly belong to the inscription. The six sign groups are also labeled (a-f).
Figure 3. Two interpretations of sign group (d) in the middle of the photograph. The first interpretation of sign group (d) leads to the following sign sequence:
Figure 3. Two interpretations of sign group (d) in the middle of the photograph. The first interpretation of sign group (d) leads to the following sign sequence:
Table 2. The Altai Mountain inscription with incorrect signs highlighted in brown. It is apparent to Hungarian language speakers that some words do not make sense, although they are close to common Hungarian words. For example, in sign group (f), the intended name PETER can be easily recognized instead of the nonsense string PETEZ. This suggests that the scribe made a spelling mistake. In particular, the scribe wrote the Old Hungarian Z sign instead of the Old Hungarian R sign. Since Old Hungarian inscriptions are written from right to left, we first convert the sign groups into a left-to-right order as shown in Table 2. Next, we also attempted a transliteration to find the meaning of the words.
Table 2. The Altai Mountain inscription with incorrect signs highlighted in brown. It is apparent to Hungarian language speakers that some words do not make sense, although they are close to common Hungarian words. For example, in sign group (f), the intended name PETER can be easily recognized instead of the nonsense string PETEZ. This suggests that the scribe made a spelling mistake. In particular, the scribe wrote the Old Hungarian Z sign instead of the Old Hungarian R sign. Since Old Hungarian inscriptions are written from right to left, we first convert the sign groups into a left-to-right order as shown in Table 2. Next, we also attempted a transliteration to find the meaning of the words.
Table 3. The Altai Mountain inscription after replacing incorrect signs with intended ones. The mix-up of the above pairs of Old Hungarian signs is a natural consequence of their similar look. Nevertheless, it is possible to ask why exactly these signs are mixed up in the inscription. To answer that question, we can apply a mathematically based approach to sign similarities. This approach was developed in an earlier paper that compared the Minoan Linear A, the Carian, and the Old Hungarian script [7]. The approach starts by identifying which sign has which of the following thirteen features:
Table 3. The Altai Mountain inscription after replacing incorrect signs with intended ones. The mix-up of the above pairs of Old Hungarian signs is a natural consequence of their similar look. Nevertheless, it is possible to ask why exactly these signs are mixed up in the inscription. To answer that question, we can apply a mathematically based approach to sign similarities. This approach was developed in an earlier paper that compared the Minoan Linear A, the Carian, and the Old Hungarian script [7]. The approach starts by identifying which sign has which of the following thirteen features:
Figure 4. A feature analysis of the Old Hungarian Runic signs: 1 indicates that the sign in the row contains the feature in the column; —1 indicates that it does not contain the feature. This analysis uses the Altai Mountain version of the Z sign.
Figure 4. A feature analysis of the Old Hungarian Runic signs: 1 indicates that the sign in the row contains the feature in the column; —1 indicates that it does not contain the feature. This analysis uses the Altai Mountain version of the Z sign.
Among the above, the G-L pair has a similarity of 12, the R-Z and the Z-CS pairs have similarities 11 and 13, respectively, and the D-I pair has a similarity of 12. Hence, these frequently mixed up pairs also have high similarity scores according to the similarity matrix in Figure 5. Hence, the strong agreement between the mathematical model and the teacher’s experience shows that the G-L and R-Z pair mix-ups in the Old Hungarian Runic inscription in Figure 1 were likely due to an accident. 0 _ At my request, Klara Friedrich, a prominent researcher and teacher of the Old Hun- garian Runic script, verified that, in her decades of experience, it is common to mix up the following letters:
Among the above, the G-L pair has a similarity of 12, the R-Z and the Z-CS pairs have similarities 11 and 13, respectively, and the D-I pair has a similarity of 12. Hence, these frequently mixed up pairs also have high similarity scores according to the similarity matrix in Figure 5. Hence, the strong agreement between the mathematical model and the teacher’s experience shows that the G-L and R-Z pair mix-ups in the Old Hungarian Runic inscription in Figure 1 were likely due to an accident. 0 _ At my request, Klara Friedrich, a prominent researcher and teacher of the Old Hun- garian Runic script, verified that, in her decades of experience, it is common to mix up the following letters:
Figure 6. The Dulo clan’s tamga (a), Kayi tamga (b), and Peter Kun’s tamga (c). Picture credits: Wikipedia https: / /en.wikipedia.org /wiki/Dulo (accessed on 16 May 2022) and https: / /en.wikipedia org/wiki/Tamga (accessed on 16 May 2022).
Figure 6. The Dulo clan’s tamga (a), Kayi tamga (b), and Peter Kun’s tamga (c). Picture credits: Wikipedia https: / /en.wikipedia.org /wiki/Dulo (accessed on 16 May 2022) and https: / /en.wikipedia org/wiki/Tamga (accessed on 16 May 2022).
Figure 7. Dr. Peter Kun’s email that verifies that he wrote the inscription in June 2000. This original email contains some minor misspellings. For example, the names of ethnic groups are written in lowercase letters, which is the common way of writing ethnic names in Hungarian. Figure 7. Dr. Peter Kun’s email that verifies that he wrote the inscription in June 2000. This original
Figure 7. Dr. Peter Kun’s email that verifies that he wrote the inscription in June 2000. This original email contains some minor misspellings. For example, the names of ethnic groups are written in lowercase letters, which is the common way of writing ethnic names in Hungarian. Figure 7. Dr. Peter Kun’s email that verifies that he wrote the inscription in June 2000. This original
Figure 8. Confusability of Peter Kun’s tamga (left) and Old Hungarian signs (right).
Figure 8. Confusability of Peter Kun’s tamga (left) and Old Hungarian signs (right).
Figure 9. A valid decipherment needs to get three things correct: signs, syntax, and semantics. The above Venn diagram places four proposals for sign group (d) on the basis of correctness according to these three criteria.
Figure 9. A valid decipherment needs to get three things correct: signs, syntax, and semantics. The above Venn diagram places four proposals for sign group (d) on the basis of correctness according to these three criteria.
Related topics:
Information ScienceDiachronic Linguistics (Or Historical Linguistics)LexicologyHistorical LinguisticsComputational LinguisticsApplied CryptographyCryptographyAegean ScriptsDecipherment of Unknown LanguagesComputational Linguistics: Machine Translation
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved