The Impacts of Pre-processing Steps on Kurdish Sorani Text Classification A thesis Submitted to the Council of the College of Science at the University of Sulaimani in partial fulfillment of the requirements for the Degree of Master of Science in Computer By Arazo Mohamed Mostafa B.Sc. Computer (2007), University of Kirkuk Supervised By Dr. Tarik Ahmed Rashid Professor

May, 2017

Gulan, 2717

‫‪‬‬ ‫َاي أَيُّ َها النَّاسا إِ َّاّن َخلَ ْقنَاكم ِمن ذَ َكرا َوأنثَى‬ ‫وب َوقَبَائِ َال لِتَ َع َارفوا اإِ َّان أَ ْكَرَمك ْام‬ ‫َو َج َع ْلنَاك ْام شع ًا‬ ‫اّللَ َعلِيما َخبِيا‬ ‫اّلل أَتْ َقاك ْام إِ َّان َّا‬ ‫ند َِّا‬ ‫ِع َا‬ ‫]سورة الحجرات‪[13 :‬‬

‫‪‬‬

Dedications I dedicate this work to:My dear parents. My lovely brothers. My lovely and only sister, Asreen. My faithful friends, the companions of the long road.

Arazo

Acknowledgments First of all, I want to express my best thanks to Allah for all His Graces in granting patience and faith, thank You God for giving me enough strength and health to complete my thesis successfully. I would like to thank my family, especially my parents and in the foreground my Mother who never ceased to support and for patience with me during the study period. I would like to express my special thanks and appreciation to my supervisor, Prof. Dr. Tarik A. Rashid for his guidance, encouragement, and support from the initial to the final stages of this study which enabled me to develop an understanding of the subject. Finally, my deepest thanks go to my friends in the labor for their constant encouragement and support during my study.

Arazo

Abstract Due to the large volume of text documents uploaded on the Internet daily. The quantity of Kurdish documents which can be obtained via the web increases drastically with each passing day. Considering news appearances, specifically, documents identified with categories, for example, health, politics, and sport appear to be in the wrong category or archives might be positioned in a nonspecific category called others. Therefore, the text classification or categorization (which is addressed in general as is the task of learning under the supervisor) is needed. The usefulness of text classification is manifested in different fields. Text categorization is a process of automatically assigning a fixed set of documents into class labels depending on their data contents.

Even though there are the considerable number of studies conducted on text classification in other languages such as English, Chinese, Spanish, and Arabic, nonetheless, the quantity of studies conducted in Kurdish is extremely restricted due to the absence of openness, and convenience of datasets.

In this thesis, a novel pre-processing method (Normalization, Stemming, Stop words filtering and removing non-Kurdish texts and symbols) is proposed. This is an active approach to increase accuracy in Kurdish Sorani text classification. Additionally, a new dataset named KDC-4007 is created. KDC-4007 is a well-documented dataset and its document arrangements are appropriate with well-known text mining tools.

I

Three widely used classifiers namely, Support Vector Machine (SVM), Decision Tree (C4.5) and Nieve Bayes (NB) within the field of text classification and feature weighting TF×IDF are evaluated on KDC-4007. In this thesis, six experiments have been performed to determine the impact of the pre-processing step method on each classifier. The experimental results indicate that the best accuracy value is obtained through the SVM classifier, followed by the NB classifier, and then the DT (C4.5) classifier in all evaluations.

KDC-4007 dataset is publicly available and the experimental results of this study can be used in the comparative experiments by other researchers. Experiments are performed in the Java programming language using the classes of WEKA Data Mining tool.

II

CONTENTS Abstract……………………………………………………………………

I

Contents…………………………………………………………………..

III

List of Tables….……………………………..………….………………..

VII

List of Figure……….………………………………….………………...

IX

List of Abbreviations....…………………………………………………... XI

Chapter One: General Introduction 1.1 Introduction……………………………………………………..…….

1

1.2 Literature Survey……………………………………………………... 4 1.3 The Problem Statement ………………………………………………. 10 1.4 The Aims of the Thesis ……………………………………….…….... 11 1.5 Thesis Contributions…………………………….……………………. 11 1.6 Thesis Layout…………………….………………….……………….. 13

Chapter Two: Text Classification Basics 2.1 Introduction…………………………………...……………………..

14

2.2 Text Mining……………………………………...………………….. 14 2.3 Text Classification…………………...……………………………… 16 2.4 Applications of Text Classification…………………………………. 18 2.4.1 Hierarchical Classification of Web pages ….………………..……. 18 2.4.2 Word Sense Disambiguation ……………………………………... 19 2.4.3 Email Classification and Spam Filtering………………...………... 19 2.4.4 Automatic indexing for Boolean IR systems ……………………... 19 2.5 Text Pre-processing Steps …..…………………………………..…... 20 III

2.5.1 Tokenization ……….………..………………….……..………….. 20 2.5.2 Stop Word Filtering ……………………………………………..... 20 2.5.3 Stemming ……….………………….………..……………………

21

2.6 Text Representation ………………….……………………………... 22 2.7 Term Weighting …………………………………………………….. 23 2.7.1 Boolean or binary weighting ……………………………………... 23 2.7.2 Term Frequency TF………..…………………………….………... 24 2.7.3 Term Frequency Inverse Document Frequency TF.IDF..……..….. 24 2.8 Machine Learning Classifiers for Text Classification task…..….…... 25 2.8.1 Naive Bayes classifier (NB)………………………………………. 25 2.8.2 Decision Trees Classifier (DT)……………………………………. 27 2.8.3 Support Vector Machine (SVM)………………………………….. 29 2.9 Evaluation and Performance Measures……………………………... 32 2.9.1 Hold out…………………………………..………..……....…..….. 32 2.9.2 K-fold Cross validation ……………………………….…….…….

33

2.9.3 Error rate ……………………………………………………….…

34

2.9.4 F1-measure…………………………………..………………….… 34 2.9.5 Confusion matrix………………………..…………………..….…

36

2.10 General Kurdish Formulation……………………………………… 37

Chapter Three: Proposed Approach for Kurdish Sorani Text Classification 3.1 Introduction………………………………………………………….

39

3.2 System Structure…………………………………….………………. 39 3.2.1 Data Collection …………………..…………………………..…… 41 3.2.1.1 Sample I…………………………………………………….…… 42

IV

3.2.1.2 Sample II…………………………………….…………………..

42

3.2.1.3 Sample III………………………………….……………………. 42 3.2.2 Data Pre-processing ………………………..….…….……………. 43 3.2.2.1 Tokenization…………….……………….……............................ 44 3.2.2.2 Kurdish Normalization………………..…………..………..…… 45 3.2.2.3 Kurdish Stemming- Steps ……………………….….…..……..... 46 3.2.2.4 List of Kurdish Stop Words ………...…………………..………. 49 3.2.2.5 Evaluation of the Stemmer Algorithm………………….……….. 52 3.2.3 Experimentation of Methodology…………………………………. 56 3.2.4 Data Representation and Term weighting ………………………... 57 3.2.5 Machine Lrearing Classifiers……………………………………… 58 3.2.6 Evaluation Metrics…………………..…………………………….. 59 3.3 Implementation and Practical Work……………………….……….. 59

Chapter Four: Test Results Evaluation 4.1 Introduction………………………………...………………………..

62

4.2 Analysis of Sample Results for Kurdish Sorani Text..……….……... 62 4.2.1 Experimental Results for Sample I……………..…….…………… 62 4.2.2 Experimental Results for Sample II..….……………..……..……..

64

4.2.3 Experimental Results for Sample III.………….……....………….. 65 4.2.3.1 The Results on SVM..………………………..………..………… 66 4.2.3.2 The Results on DT..…………………………………...………… 69 4.2.3.3 The Results on NB……………………….....…………………… 72 4.2.4 Results and Discussion Sample III ………………………………. 75 4.2.5 Analytical Discussions……………………….....………………... 79

V

Chapter Five: Conclusions and Future Works 5.1 Conclusions………………………………..………………………...

81

5.2 Suggestions for Future Work………………………..………………. 83

Publication………………………………………………………………..

84

Appendix …………………………………………………………..…….

85

References ………………………………………………………………..

89

VI

List of Tables

Table

Page

Table Name

No.

No.

2.1

Set of Training Example [59]……...……….…………...

27

2.2

Describing Confusion Matrix………..…………………

36

2.3

Example of Kurdish Affixes ...…………….…………..

38

3.1

Number of Classes and Documents in Selected Sources..

43

3.2

Prefixes and Suffixes Removed by Kurdish Stemmingsteps……………………………………………………..

3.3

Example of Stop Word Affixes………………………..

3.4

The

Dataset

Versions

of

Experimentation

46 47

of

Methodology…………………………………………….

57

4.1

Stop Words Removed from Document Test on Sample I

63

4.2

Results of Paice’s Evaluation Method using Sample II…

65

4.3

Accuracies of SVM of the Dataset Using Fold=10…….

66

4.4

Accuracies

of

SVM

of

the

Dataset

Using

percentage70%............................................................ 4.5

Experimental Results of Recall, Precision and F1 for SVM fold=10……………………………………………

4.6

67

Experimental Results of Average Recall, Precision and F1

for

SVM

Classifier

on

Six

Tests

Using

Percentage70%................................................................. 4.7

66

68

Accuracies of DT (C4.5) on the Six versions of the Dataset Using 10-Fold……………………………..…...

VII

69

4.8

Accuracies of DT (C4.5) on the Six Versions of the Dataset Using Percentage 70% for Training and 30% for Testing……………………………………….…………

4.9

70

Experimental Results of Average Recall, Precision and F1-measure 1 for DT (C4.5) Classifier on Six Tests Using 10-Fold Cross Validation……………….………

4.10

71

Experimental Results of Average Recall, Precision and F1-measure for DT (C4.5) Classifier on Six Testes Using Percentage 70 %....................................................

4.11

Accuracies of NB on the Six Versions of the Dataset Using 10-Fold Cross Validation……..………………..

4.12

73

Accuracies of NB on the Six versions of the Dataset Using Percentage 70% for Training and 30% for Testing

4.13

71

73

Experimental Results of Average Recall, Precision and F1-measure for NB Classifier on Six Tests Using 10Fold Cross Validation………………………………….

4.14

74

Experimental Results of Average Recall, Precision and F1-measure for NB Classifier on five Testes using percentage 70%.............................................................

4.15

74

Experimental Results for SVM, NB and C4.5NB Classifiers on Six Tests ………………………..………..

VIII

79

List of Figures

Figure

Page

Figure Name

No.

No.

1.1

The Main Text Classification Stage………………...…..

1

2.1

Text Classification Process……………………………...

17

2.2

An Example of Decision Tree [59]……………...………

28

2.3

SVM Builds a Hyperplane that Perfectly Separates …....

30

2.4

Diagram of k-fold Cross-Validation with k=10………...

33

2.5

The Kurdish Alphabet[5]……………………………...

37

3.1

The Frame structure of Sorani Kurdish TC……………..

40

3.2

Architecture of Sorani Kurdish Text Pre-process……….

44

3.3

Shows Example of Tokenizing Process…………………

44

3.4

Procedure Normalization Word from Kurdish Document

43

3.5

Overall Steps of Stemming of the Kurdish Sorani Documents………………………………………………

3.6

46

Flowchart for Removing Sorani Stop-Words from Kurdish Documents………….………………………….

50

3.7

Flowchart of Proposed System for Kurdish Documents..

51

3.8

Kurdish Preprocessing-Steps Algorithm………………

52

3.9

Illustration of Paice Evaluation Method……………...

55

3.10

WEKA Data viewer for KDC-4007 Dataset…………...

58

3.11

WEKA Data Format for Text Classification…………..

61

4.1

The Experimental Results of Kurdish Stemming Steps

4.2

Module…………………………………………………

64

The

69

Experimental

Results

IX

of

Kurdish

Text

Classification Using SVM…………………………...... 4.3

The

Experimental

Results

of

Kurdish

Text

Classification Using DT (C4.5)........................................ 4.4

The

Experimental

Results

of

Kurdish

Text

Classification Using NB…….………………………….. 4.5

75

Classification Accuracy of SVM, NB and DT (C4.5) on the Six Versions of the Dataset Using Fold=10………...

4.6

72

76

Time Taken to Building Classifiers of SVM, NB and C4.5 Without Preprocessing and with Preprocessing Using Fold=10…………………..…………………..….

4.7

The Effect of Preprocessing Steps on the Experimental Results for SVM, NB and C4.5 using Fold=10…………

4.8

76

Experimental

Accuracy

Percentage

Results

of

Classifiers on Datasets Using 10-Fold Cross Validation..

X

77

78

List of Abbreviations Abbreviation Meaning ARFF

Attribute Relation File Format

BOW

Bag Of Words

CCI

correctly Classified Instances

DF

Document Frequency

DMT

Desired Merge Total

DNT

Desired Non-Merge Total

FN

False negatives

FP

False Positives

GDMT

Global Desired Merge Total

GDNT

Global Desired Non-Merge Total

GUMT

Global Unachieved Merge Total

GWMT

Global Wrongly-Merged Total

ICI

Incorrectly Classified Instances

IDF

Inverse Document Frequency

IR

Information Retrieval

KDD

Knowledge Discovery in Databases

KE

Knowledege Engineering

ML

Machine Learning

NB

Naive Bayes

OI

Over-Stemming Index

SNN

Sulaymani News Network

SVM

Support Vector Machine XI

TC

Text Classification or Text Categorization

TF

Term Frequency

TN

True Negatives

TP

True Positives

UI

Under-Stemming Index

UMT

Unachieved Merge Total

VSM

Vector Space Model

WMT

Wrongly Merged Total

WS

Weight of Stemming

WSD

Word Sense Disambiguation

XII

Chapter 1

General Introduction

Chapter One General Introduction 1.1 Introduction With amount of text documents available on the World Wide Web (WWW) which grows rapidly in electronic forms, automatic data document classification is becoming an important field. Text categorization is a technique for organizing and managing these data documents, at the same time is improving the precision of retrieval. Thus, text categorization is a process of classifying unstructured documents into one or more pre-defined categories such as art or sport, and so on, based on linguistic features and content. Many application problems such as automatic web page categorization [1], word sense disambiguation [2], spam filtering [3], e-mail filtering [4] and others have been solved using text classification. There are three main phases in the text classification task as shown in Figure 1.1.

Figure (1.1): The Main Text Classification Stage

With more and more online information being available in Kurdish Sorani, a need was felt to develop tools for processing Kurdish language as approximately there are 40 million people who speak Kurdish in Iraq, Turkey, Iran, Syria, Lebanon, Armenia, Georgia, Kyrgyzstan, Azerbaijan, Kazakstan, 1

Chapter 1

General Introduction

and Afghanistan. Thus, development or improvement of the word search algorithm for this language is regarded as the actual and interesting task. At present, other languages presented many researches in text classification. For Kurdish, there are limited number of studies [5]. However, developing text classification systems for Kurdish documents requires more challenges. These challenges are due to the differences and the complex morphologies in Kurdish language dialects and the main factors behind these complexities are the large uses of inflectional and derivational affixes, likewise Kurdish Sorani dialect’s writing system, definiteness markers, possessive pronouns, enclitics, and many of the widely-used postpositions are used as suffixes. These challenges reveal the importance of an analysis and development of preprocessing techniques for Kurdish text classification. However, some recent works on literature showed that pre-processing methods such as stemming, stop words removing, and others, positively have great impact on performance of information retrieval and text mining systems for different languages [6], [7], [8], [16], [21]. Preprocessing data is an important and critical step in text mining, since the pre-processed data is used eventually for classification; thus, it would have great impact on results in terms of performance and accuracy. Text classification is unstructured and expressed in natural language which is extremely difficult to model. There are several techniques used in text mining for preprocessing the text categorization such as tokenization, normalization, stop word filtering, stemming and word weighting are most commonly used [19], [20], [21]. These methods allow us to transform this unstructured data into structured format that

2

Chapter 1

General Introduction

the data mining algorithms can work on. They are also frequently used in information retrieval domain. The process of converting words into their roots is called stemming. This is a vital process in digital text classification. It seems that text classification requires that a language stemming should take place prior to any compression algorithm or implementation. Stemming helps to decrease the space required for storing the configurations or indices of terms in the document. It also helps decrease the computational load of the used system. Stemming approaches have already been developed in different languages such as English, Iranian and Arabic [22], [23], [24], [25]. However, there have been little or no research works developed for building stemmers in the Kurdish language. It is of interest to note that stemming is language dependent; in other words, an English stemmer cannot be used in the Arabic language; likewise, an Arabic stemmer cannot be useful for other languages. And since the stemming technique is an important tool which supports the development of various natural language processing applications. It can be used in information extraction, search engines [26], automatic indexing, unstructured documents [27], machine translation, spell checking, and so on. Stemmers are language-specific tools. The contribution of this thesis is to describe the design of tools featuring all pre-processing steps in one package to be useful and efficient for all applications needed by Kurdish Sorani datasets. Then several classification algorithms will be used to identify the performance of all pre-processing steps.

3

Chapter 1

General Introduction

1.2 Literature Survey In the field of text classification problem, as mentioned in the earlier section, very few research works have been studied for the Kurdish language. Therefore, this field is at an early stage. It is worth noticing that due to the progress of the World Wide Web, and the increased number of non-English users, many research efforts for applying pre- processing approaches for other languages, the English and Arabic languages. Since Kurdish Sorani script is considered as the closest to the Arabic language, technically both have a written system that is from right to left. In this literature study, the research works in the text classification field have been sorted starting with the Kurdish language shadowed by the Arabic language and then followed by the English language: Mohammed et al. in 2012 used the N-gram Frequency Statistics for classifying Kurdish text. An algorithm called Dice’s measure of similarity was employed to classify the documents. A corpus of Kurdish text documents was build using Kurdish Sorani news articles collected from the online websites of several Kurdish newspapers. It consisted of 4094 text files divided into 4 categories: art, economy, politics, and sport. Each category was divided equally per its size (50% as training set and 50% as testing set). For the training and the test documents, the N-gram word level 1 gram and character level (2, 3, 4, 5, 6, 7, and 8) frequency profile were generated for each document and saved in text files. The Recall, Precision and F1 measure were used to compare the performance. The results showed that N-gram level 5 outperformed the classification the other N-gram levels [5]. Al-Harbi et al. 2008 in [6], proposed two popular classification algorithms SVM and C5.0 for Arabic text classifier. In general, documents used in this study consisted of 17,658 text documents which were collected from different 4

Chapter 1

General Introduction

sources. Arabic text classification were implemented to accomplish both feature extraction and selection tasks, then Chi Square technique was used to calculate features which were important. The text documents were divided into training and testing sets (70% for training and 30% for testing in each corpus). The results showed that C5.0 classifier produced average accuracy rate of 78.42% which was better than SVM, which produced average accuracy rate of 68.65%. Duwairi et al. in 2009 made comparison between two stemming approaches, namely; light stemming and word clusters to investigate the impact of stemming. The dataset consisted of 15,000 Arabic documents, and was categorized into three categories: sports, economics, and politics (5000 documents per class). The experiments were executed utilizing four different representations of the same datasets, stem-vectors, the light stem-vectors, the word clusters, and the original words. The authors reported that the light stemming procedure improved the accuracy of the classifier better than the other procedures [7]. In [8] Al-Kabi M et al. in 2011, conducted comparison between three classifiers Naïve Bayes Classifier (NB), Decision Tree using C4.5 Algorithm and Support Vector Machine (SVM) to classify Arabic texts. An in-house collected Arabic dataset from different trusted websites is used to estimate the performance of those classifiers. The dataset consisted of 1100 text documents and was divided into nine categories: Agriculture, Art, Economics, Health, Medicine, Law, Politics, Religion, Science, and Sports. Additionally, preprocessing included word stemming, and stop words removing was conducted to reduce the dimension of the feature vector space. The experiments showed that three classifiers achieved the highest accuracy in those cases that did not include stemming. While the accuracy was decreased when using stemming. 5

Chapter 1

General Introduction

This means that the stemming had impacted negatively on the performance of the classification accuracy of the three classifiers. In [9] Al-Shargabi et al. in 2011, evaluated the performance of three well known classification algorithms Support Vector Machines, Naive Bayes, and Decision Trees C5.0 classifier for classifying Arabic text based on stop words elimination. The outcomes demonstrated that the SVM classifier accomplished highest accuracy and the least error rate. In addition, the time needed to build the SVM model was much lower compared to the other two classifiers. In [10] Wahbeh A. H. et al. in 2012, aimed to compare three classification techniques for Arabic text documents. These three algorithms were namely, SVM classifier, Decision Trees C4.5 Classifier, and NB classifier. A set of Arabic text documents collected from different websites which lie into four classes, namely, Politics, Economics, Sports, and Prophet Mohammed Sayings (Al-Hadeeth Al-Shareef). The data sets are used in two ways, firstly divided into a training set and testing set, where 60% used for training phase and remaining 40% used for the testing phase, secondly 10-fold Cross-Validation used for training and testing. The performances of the three text classification techniques were based on accuracy and time. In terms of accuracy, results showed that the NB classifier achieved the highest accuracy, followed by the (SVM) classifier, and then, the Decision Trees (C4.5) classifier. In terms of time, the results showed that the time taken to build the SVM model was the lowest time, followed by the NB model, and then, the C4.5 classifier which took a highest amount of time to build the model needed. In [11] Zaki et al. in 2014 offered a new approach based on n-grams and the TF×IDF measure for offering information extraction techniques based on parts of words on classifying Arabic documents belonging to three categories namely, Sport, Politic, Finance & Economics. Preprocessing tasks including 6

Chapter 1

General Introduction

normalization, stop word removal, and stemming techniques were used. By comparing the acquired results, they found that the utilization of Radial Basis Functions enhanced the performance of the system. Ababneh et al. in 2014 used the k-nearest neighbor (KNN) algorithm to compare different variations of Vector Space Model (VSM) against seven Arabic Saudi data sets. For comparison purposes, these variations were Cosine Coefficient, Dice Coefficient, and Jacaard Coefficient and IDF term weighting method. Preprocessing tasks contained normalization, stop word removal in their experiment. The experiments showed that the Cosine Coefficient has achieved the best results compared to the other two vector space models using the same Arabic Saudi dataset [12]. Hmeidi I. et al. in 2015 conducted a comparison of the five best known algorithms for text classification were namely, Naive Bayes, Support Vector Machine, K-nearest neighbours, Decision Tree, and the Decision Table. The authors also studied the effects of using different Arabic stemmers (light and root-based stemmers) on the effectiveness of these classifiers. Moreover, they assessed the accuracy and scalability of two well-known data mining and machine learning tools (Weka and RapidMiner) to investigate their pros and cons for Arabic text classification. Arabic article corpus collected by Diab Abu Aiadh [13] as the sole dataset for training and testing purposes which is composed of 2700 documents equally spread across nine categories (Arts, Economics,

Health,

Law,

Literature,

Politics,

Religion,

Sports

and

Technology). The outcomes represented the good accuracy given by the SVM classifier, particularly when utilized with the light10 stemmer [14]. In [15] Mohammad AH et al. in 2016 studied the performance of three well-known machine learning algorithms Support vector machine, Naïve Bayes and Neural Network on classifying Arabic texts. The datasets consisted of 1400 7

Chapter 1

General Introduction

Arabic documents divided into eight categories collected from three Arabic news articles namely: Aljazeera news, Saudi Press Agency (SPA), and Alhayat. In terms of performance three evaluation measures were used (recall, precision and F1 measure). The results indicated that SVM algorithm outperformed NB and MLP-NN and F1-measure for three classifiers were 0.778, 0.754, and 0.717 respectively. Dumais et al. in 1998 compared five learning methods which are: Find Similar (a variant of Rocchio’s method for relevance feedback), Decision Trees, Naïve Bayes, Bayes Nets, and Support Vector Machines (SVM). The comparison is done based on the learning speed, classification speed in real time, and classification accuracy. The so-called Reuters-21578 collection was used and then, the ModApte split was followed, which 75% of the stories was used for training to build classifiers and the remaining 25% for testing. They reported that the SVM was the best among other classifiers due to being very accurate, fast to train, and fast to evaluate [16]. LAN M. et al. in 2005 proposed Support Vector Machines algorithm with various term weighting scheme methods to classify English texts into predefined categories based on their content. Also, they exhibited a new term weighting scheme TF×RF to improve the term's discriminating power. The two widely benchmark data sets used in the experiments which were the Reuters21578 corpus and the 20 Newsgroups corpus. The experimental results demonstrated that this newly proposed TF×RF scheme was significantly superior to other widely-used term weighting schemes [17]. In [18] Toman et al. in 2006 analyzed the effects of normalization (stemming or lemmatization) and stop-word removal on English and Czech datasets. They used two datasets for experiments. The first was Reuters corpus which consisted of 8000 documents in English fell into six categories and the 8

Chapter 1

General Introduction

second was Czech News Agency corpus that contained 8000 documents in the Czech language belonging to five categories; moreover, the multinomial Naive Bayes classifier used for classification tests. The results showed that stop-word removal improved the classification accuracy in most cases while the influence of word normalization (Stemming and lemmatization) improved slightly the classification accuracy. Also Pomikálek et al. in 2007 examined the effect of preprocessing tasks including stop-word removal, and stemming on the English documents. They used three data sets for categorization. They were Reuters-21578 ModApte, 20Newsgroups and Springer.

The comparison was made using different

classifiers including Support Vector Machines, Naive Bayes, K-Nearest Neighbour, Neural Networks, C4.5, Simple Linear Regression, Voted Perceptron, and RepTree. It was seen that utilizing stemming and stop-word removal would have very little effect on the general classification results. Also, they noticed that using stemming and stop-word removal would have very little impact on the overall classification results [19]. In [20] Zhang W et al. in 2011, the authors evaluated the performance of three methods document representation methods: TF.IDF, LSI and multi-word in text classification. In their study, two different languages where used, namely Chinese and English. The dataset for the Chinese corpus was TanCorpV1.0 which consisted of 14,150 documents with 20 categories, randomly from the original corpus, four categories were selected, thus, totally 1200 documents were used; and for the English corpus was Reuters-21578 distribution 1.0 which contained 21,578 documents with 135 categories, also, four categories assigned, thus, totally 2042 English documents were used. Stop-word was eliminated from English documents. Support vector machine was used to estimate the performances of the above methods. Also, Information 9

Chapter 1

General Introduction

gain feature selection method was applied. Experimental results demonstrated that LSI would have better performance than other methods in both document collections in text categorization. In [21] Mohsen AM et al. in 2016 conducted study to compare the performance of different well known machine classifiers to classifying emotion documents. The ISEAR dataset was applied. It consisted of 7,666 documents belonging to five categories namely: Anger, Disgust, Fear, Joy and Guilt. Tokenization, stop word removal, stemming and lemmatization were used as preprocessing tasks and TF.IDF as term weighting. Similarly, two lexicons were used which are NRC emotion lexicons (National Research Council of Canada) and SentiWordNet sentiment lexicons. Based on the obtained results, the authors concluded that LMT was the most appropriate classifier for English emotion document classification in comparison with the other algorithms. Most of the past studies proposed the utilization of preprocessing tasks to decrease the dimensionality of feature vectors without completely looking at their commitment in advancing the effectiveness of the text classification framework, which makes this study a novel one that leaned towards to the effect of preprocessing assignments on the performance of the Kurdish Sorani classification system.

1.3 The Problem Statement Online Kurdish text categorization is not of high quality. When articles are accessed in a collection instead of in a newspaper (site), it is difficult to browse by class. Moreover, there is no basic and acknowledged categorization for the Kurdish texts. Even when a classifier is selected, there isn’t best demonstrated techniques for classifying Kurdish Sorani text. Additionally, there is no simple and best strategy for Kurdish word stemming that will improve Kurdish text 10

Chapter 1

General Introduction

classification based on given classes. In this context, a robust preprocessing technique proposed to attempt to pick the minimum number of terms (features) to improve accuracy and reduce the runtime when building a classification model.

1.4 The Aims of the Thesis The main objectives of this research work are to develop and implement an efficient approach for large volume of Kurdish Sorani text that achieves the enhanced level of speedup and preserves the required accuracy of classification algorithm. Similarly, this thesis aims to determine and collect a Kurdish corpus of text documents with various domains. It also purposes to design the most suitable text preprocessing techniques such as stemming, and finally, it intents to evaluate and compare the speedup and accuracy of the proposed approach with several machine learning classifiers.

1.5 Thesis Contributions The main contributions of this thesis are as follows: 1. One of the contribution points of this research is to compile representative Kurdish Sorani datasets that cover different text types which will be utilized as part of this research and later as a benchmark. Therefore, the datasets are collected from various sources and different domains. A new dataset named KDC-4007 is created, which can be widely used in the studies of text classification about Kurdish news and articles. The KDC-4007 dataset can be utilized for various computational linguistics researches including text mining, information retrieval, and in advancement stages in the field of computational linguistics. 11

Chapter 1

General Introduction

2. Using linguistic expertise to design a stemming-step module to strip prefixes, suffixes and postfixes from the given word by steps until catching potential roots. So, developing and integrating Kurdish Sorani morphological analysis tools (stemming) in one package. 3. Grouping Kurdish Sorani words to experiment with further stemming methods and use both over-stemming and under-stemming to investigate the extent and significance of the performance. 4. Cover the issues of the normalization process of Kurdish Sorani which can be caused by discrepancy issue which is generated via typescripts of multiple unicodes. Also, these issues open challenges in the stemming field which can be faced with taking care of normalization process to get the more standardized texts. 5. Collect and manage intensive stop-word list to reduce the size of vocabulary as much as possible, also provide light stop-word list which slightly affects the recall. 6. Demonstrating the effectiveness of the hybrid steps for pre-processing compared to not using the hybrid steps, we provide comparative results that demonstrate the relative performance of classifier methods (e.g. SVM, C4.5, and NB) to classify a hold test set. 7. Since weighting schemes have not been addressed in the literature for Kurdish language yet. Thus, applying a feature weighting schemes (Term Frequency, Inverse Document Frequency) on Kurdish Sorani dataset and investigating its impact on Kurdish Text Classification before (e.g. original dataset) and after the developed pre-processing steps for Kurdish documents in text representation.

12

Chapter 1

General Introduction

1.6 Thesis Layout The thesis is organized as follows:

Chapter Two: [Text Classification Basics] This chapter presents a background (such as the concept of text mining in general and text categorization in particular, and techniques used to transform text documents into a form that is suitable for automatic processing) needed to develop an efficient Kurdish Sorani text classification system. In addition, it describes the well-known machine learning classifiers for text classification. It also discusses the types of term weighting techniques. This chapter ends with reviewing of general Kurdish formulation.

Chapter Three: [Proposed Approach for Kurdish Sorani Text Classification] This chapter describes the overall process of Kurdish data collection and documents pre-processing stage, used in this thesis. It also explains in detail the proposed methods through the algorithm, flowchart and block data diagram.

Chapter Four: [Test Results Evaluation] This chapter present the impact of using documents pre-processing stage on three classification algorithms. Results are provided in such a way that makes it easy for understanding also for comparing.

Chapter Five: [Conclusions and Future Works] This chapter summarizes and discusses the work of this thesis. It also suggests some ideas for future work, and the future steps to be taken to solve the problems left open by this research.

13

Chapter 2

Text Classification Basics

Chapter Two Text Classification Basics 2.1 Introduction This chapter presents the key concepts related to this thesis, including text mining in general, and text classification with its applications. It explains the fundamental parts of text classification such as text pre-processing, text representation, term weighting and the most common machine learning algorithms utilized in text classification. In addition, it introduces different performance measures in terms of evaluation. Also, reviewing of general Kurdish formulation are being covered in this chapter.

2.2 Text Mining In recent years, there is an enormous amount of machine readable data stockpiled in files and databases in forms of text documents. Texts are one of the most common and convenient technique for information exchange due to the fact that much of the world’s data can be found in text form such as newspaper articles, emails, literature, web pages, and others. The rapid growth of the text databases due to direct amounts of information available in electronic forms, such as e-mails, the World Wide Web, electronic publications, and digital libraries. Text mining can be defined as the process of discovering meaningful and interesting linguistic patterns from large collection of textual data, and it is relevant to both information retrieval (IR) and knowledge discovery in databases (KDD) [28],[29]. In general, data mining is an automatic process of finding useful and informative patterns among large amounts of data or detecting new 14

Chapter 2

Text Classification Basics

information in terms of patterns or rules from that enormous amount of data. Data mining usually deals with structured data, but information stored in text files is usually fairly unstructured and difficult to deal with. Thus, for dealing with such data, a pre-processing is required to convert textual data into an appropriate format for automatic text processing. It can be used on a variety of data types, include structured data (relational), multimedia data, free text, and hypertext [28], [30]. In data mining, knowledge or information is usually invisible and unknown, so automatic techniques are required in order to simplify the extraction of these data. In text mining, the data is obvious in the text documents but the biggest problem is that this information or data is not represented in a suitable technique for processing by a computer. One of the text mining goals is to represent text data stored in a suitable form since there are the richness and ambiguity of natural language texts used in most of the obtainable documents [25]. Extracting and analyzing useful information from text data is a difficult task, besides, it is not easy to retrieve relevant and efficient queries without knowing what could be in the documents. Text mining uses techniques from information retrieval, information extraction as well as natural language processing (NLP) and joins them with the algorithms and methods of data mining, KDD, machine learning and statistics [28]. The purpose of text mining is to process unstructured textual data, and extract non-trivial patterns or meaningful pattern from the text data. In addition, make the information included in the text accessible to the different data mining algorithms and reduce the effort required of users to obtain useful information from large computerized text data sources [28], [29]. Text mining is generally a multidisciplinary domain, research in text mining involves dealing with problems such as text representation, text analysis, text 15

Chapter 2

Text Classification Basics

summarization, information retrieval, information extraction, text classification and document clustering. In all of these problems, data mining techniques and statistics are used to process textual data [28], [31].

2.3 Text Classification With the amount of text documents available on the World Wide Web (WWW) which grows rapidly in electronic forms, automatic document classification is becoming an important field. Text classification is a process of automatically assigning sets of documents into classes (categories) labeled based on their contents, and it is also considered as an important element in the management of tasks and organizing information [32]. It is the process of classifying unstructured documents into one or more pre-defined categories such as science, art or sport, based on linguistic features and content. It can be described as a natural language problem in which the goal is to decrease the need of manually organizing the huge amount of text documents. There are two main approaches for the text classification problem: the knowledge engineering (KE) approach and the supervised learning approach. The first approach, KE is based on manually defining a set of rules constructed by domain experts to building a classifier C. Whereas in the second approach, the classifiers are automatically built from a set of labeled (already categorized) documents by applying machine learning techniques and there is no need for a manual definition by domain experts [33]. Manual classification is undesirable due to the time consumption and it requires high accuracy. Automated text classification makes the categorization process fast and more efficient since it automatically classifies text documents. At present, machine learning techniques have become the most common approach for solving the text classification problem automatically [32], [33], [34]. 16

Chapter 2

Text Classification Basics

Many of the sub-problems are branched out from the text classification problem and have been studied intensively in the literature such as the document indexing, the weighting assignment, dimensionality reduction, document clustering, and the type of classifiers created. The process of text classification consists of several key steps, such as text pre-processing, feature selection and then building the classifier model on the training data and assessing it on the testing data. Figure 2.1 explains the general steps of the text classification process.

Figure 2.1: Text Classification process.

In the pre-processing steps, a tokenization procedure is performed in order to remove non-informative words such as digits and punctuation marks. The main objectives of pre-processing are to obtain the key features or key terms 17

Chapter 2

Text Classification Basics

from the stored text documents and to improve the relevancy between word and text document as well as the relevancy between word and class. The twocommon feature selection (reduction) approaches that are used in both supervised and unsupervised applications are the stop word removal and stemming. In stop-word removal, the common words in the documents which are not specific or discriminatory to the different classes are determined. In stemming, different forms of the same word are consolidated into a single word. In this process, the root/stem of a word is found. The purposes of this method are: to remove various suffixes, to reduce number of words, to have exactly matching stems, to save memory space and time. For example, ‘plays’, ‘playing’, ‘played’, ‘player’ are reduced to stem ‘play’. A set of the most informative words is kept and then used to represent the text document as vectors of features. Essentially in document classification, the goal of feature selection is to improve the classification accuracy and computational efficiency by discarding irrelevant and noisy terms (features) that do not have enough information to assist with text classification. Finally, the last stage is to build a classification model on training data using the best subset of features and then evaluating its performance on a separate test data. The performances of the classifiers are evaluated using many performance metrics such as Precision, Recall, F-measure, etc. [32], [33], [34], [35].

2.4 Applications of Text Classification Many useful applications have been found for text categorization approach. The following are a few examples of its applications. 2.4.1 Hierarchical Classification of Web Pages: the amount of web pages or sites on the World Wide Web is growing rapidly, making the task of finding specific information on the Web more difficult without organizing web pages. 18

Chapter 2

Text Classification Basics

It is evident that web pages or sites are organized under the hierarchical classification, this would make a web search engine easier and faster to start navigating and then restricting the search to a category that contains the required information. Whereas manual categorization of web pages is infeasible and costly. Text categorization techniques have been used for classifying web pages under hierarchal categories to facilitate the web navigation for automatic web classification [1]. 2.4.2 Word Sense Disambiguation: Word sense disambiguation (WSD) is the process of identifying the meanings of words in contexts. WSD is very important for many tasks, including natural language processing, machine translation, and indexing documents by word senses rather than by words for information retrieval purposes. Text classification techniques have been applied successfully to the WSD problem [2]. 2.4.3 Email Classification and Spam Filtering: A spam filter is an automated technique to excellence a spam from a non-spam for preventing its delivery [3]. The unsolicited bulk messages randomly sent by spammers that is creating a persistent problem for internet service providers and users. As a result of this growing problem automated methods for filtering such spam from valid email are becoming necessary. The most studied for the spam is junk mail or e-mail spam [4]. Text classification technique is used to characterize incoming e-mail as positive (i.e. spam) or negative (non-spam) and to reject those messages that contain spam. Several laboratory and field studies are introduced to achieve the best result based on text classification algorithm on how to distinguish between spam and non-spam messages [36] [37]. 2.4.4 Automatic Indexing for Boolean IR Systems: Document indexing automatic for IR systems depend on a controlled dictionary. One of the 19

Chapter 2

Text Classification Basics

prominent examples is Boolean systems via which each document is assigned one or more keywords or key phrases describing its content, where these keywords and key phrases belong to a finite set called controlled dictionary, often consisting of a thematic hierarchical thesaurus (e.g. The MESH thesaurus for medicine or the NASA thesaurus for the aerospace discipline). Automated metadata generation viewed as a problem of document indexing with controlled dictionary and thus tackled by means of text categorization techniques [38].

2.5 Text Pre-processing Steps Dataset pre-processing is an important stage in text mining. A huge number of features or keywords in the documents can lead to a poor performance in terms of both accuracy and time. For the problem of text classification a document, which typically has a high dimensionality of the feature space and most of the features (i.e., terms) are irrelevant to the classification task or non-informative terms. The main objective of the preprocessing steps is to prepare text documents for the next step in text classification which are represented by a great number of features [39], [40]. The proposed model of the pre-processing steps for Kurdish Sorani text document will be explained in detail in Chapter three. The most common steps for text pre-processing are described below. 2.5.1 Tokenization: It is responsible for tokenizing the document into a sequence of tokens (features) delimited by whitespace, punctuations, tab, new line, and so on. In fact, this rough structuring serves as a basis for later processing within the text classification [28]. 2.5.2 Stop Word Filtering: A stop words (function words) are based on the previously specified language information. They are occurring frequently in a 20

Chapter 2

Text Classification Basics

text document that has little semantic content and they do not help in discrimination between classes [28]. For instance, words in English language such as ‘a’, ‘an’, ‘the’, ‘and’....... etc. are stop-words, those words are increasing the noise of the results because they are so common terms that provide only a little of meaning and contribute only in a syntactic function without indicating any paramount of matter or subject [44]. These stop words are recognized and filtered out by the process called stop word removal [34], [35], [39]. The removal of stop words changes the feature (word) length and minimizes the memory of the process. 2.5.3 Stemming: Is a technique for reducing an inflected or derived word to their common base (root) to keep only their semantically relevant essence while grammatical affixes are stripped off. For example, the word ‘plays’, ‘playing’ and ‘player’ can be reduced to their root ‘play’. The purpose of this application of language specific algorithms is to allow to handle variants of the same word in a more abstract manner and produce in a more robust extraction of document characteristics. This method is not specific to the case of the classification problem which is often used in a sort of unsupervised applications such as clustering and indexing [43]. In the classification problem case, it makes sense to supervise the feature selection process with the use of the class labels. This kind of selection process ensures that those features which are extremely skewed towards the turnout of a class (label) are picked for the learning process, and therefore, optimizes the text classification accuracy [41], [42]. In general, stemming has many advantages which reduces the size of document and increases the speeding process, and it can also be used in information retrieval systems to reduce variant word forms to common roots to improve retrieval effectiveness [44].

21

Chapter 2

Text Classification Basics

2.6 Text Representation Before classifying, documents must be transformed into a numerical vector to be ready for automatic processing. The text representation models, which is the process of transforming a document from a series of characters into sequences of words to be appropriate for the learning algorithm and the classification task. This type of simple representation has a strong effect on the generalization accuracy of a learning system [45]. The representation of text document can be coded as a form of a matrix, where columns indicate words that distinguish the objects (text documents) stored in each row, where each apparent word is a feature and the number of times the word occurs in the text document is its value [46]. The most common technique for text representation in the text categorization task is bag of words (BOW) [45], [46]. This method of document representation is also known as Vector Space Model (VSM); it is the most common and easy way of text representation. In BOW case, the representation of term (feature) is a single word. Each text document Di is represented as a vector Di = where N denotes the number of distinct features (terms) in the document collection [47]. The value of vNj between (0, 1) represents how much the term tk contributes to the semantics of document Di [47], [48]. In the context of text classification, researchers have tried to use another approach based on phrases to represent text documents instead of single words as features [50]. In general, the advantage of using phrases for text representation is that phrases have greater meanings than single words [49] [51]. For instance, in [49], authors use multi words for text representation in text classification problem. Based on the syntactic structure, a practical method was used to extract the duplication patterns of two sentences and then, the regular expression method was used for multi-word to extract noun phrases. In 22

Chapter 2

Text Classification Basics

addition, information gain was applied as a feature selection method and SVM was implemented as a text classifier. However, the results showed that using multi-words for text representation did not introduce better classification accuracy in comparison with case of single word representation. In [50], a statistical phrase of different lengths of n-grams such as unigrams and bi-grams were used to represent text documents, after stop words removal. Also, in this study, various filter approaches were used as features selection technique. Furthermore, the Racchio algorithm was implemented as a text classification model. Nevertheless, experimental results on the Reuters21578 benchmark showed that applying statistical extracted uni-grams and bigrams as features did not produce better results than using the single word representation. In this thesis, an extensive study of the approach to adjust Kurdish terms (features) is performed on any text representation during preprocessing steps to improve Kurdish text classification.

2.7 Term Weighting In text representation, terms are words, phrases, or any other indexing units used to recognize the contents of a document [51]. Each term that appear in documents must be represented to machine learning classifiers as realnumber vectors of weights [48]. According to a text classification research, weighting of term can be divided into three major approaches [51]: 1. Boolean or binary weighting: Binary weighting is the simplest way of encoding for the term weighting. If the corresponding word (term) is used in the document dj at least once then it is set to 1 otherwise 0 [48] [51].

23

Chapter 2

Text Classification Basics

2. Term Frequency TF: In term frequency weighting scheme, an integer value indicates the number of times in which the term it appears in a specific document dj [48] [51]. 3. Term Frequency Inverse Document Frequency TF.IDF: TF×IDF can be considered as the most accurate application for text categorization domains with more than two categories. The results are usually in higher performance, which interprets why the TF×IDF weights are typically preferred over the other two options [45], [52]. It is a straightforward and efficient method for weighting the terms in text documents categorization purposes; similar algorithms have been used in [39], [45]. In this work, the TF×IDF weighting function used and it is based on the distribution of the terms within the document and within the collection, where the higher value indicates that the word occurs in the document and does not occur in many other documents, and it is in inverse amount to the number of documents in the collection, for which the word occurs at least one time [52]. Can be calculated as follows: (2.1)

Where TF is the frequency of term in document dj and DF (ti) is the number of documents that contains term ti, after stopping word removal and word stemming, and N is the total number of documents.

24

Chapter 2

Text Classification Basics

2.8 Machine Learning Classifiers for Text Classification Task In this thesis, three types of machine learning classifier are used for classifying the Kurdish Sorani text documents into several classes. These classifiers are Naïve Bays, Decision Tree and Support Vector Machines. A description of each type is demonstrated in the following subsections.

2.8.1 Naive Bayes Classifier Naive Bayes classifier is a probabilistic based approach which is based on Baye’s theorem. It is simple and efficient for implementation [9] [55]. Naive Bayes classifier underlies on the assumption that the features (words) in the dataset are conditionally independent which computes the probability of each by calculating the frequency of features (words) and the relevance between them in the dataset [9]. Suppose a set of training samples where each sample X is determined by a set of feature values < F1; F2; … Fn > and let C be a set of classes describing the target function. However, based on the feature values given a test sample t. The test sample to the class is assigned in NB per the highest probability. The probability that the test samples t belongs to a class Cj can be evaluated as follows: (2.2) is the probability of the class

given a test example t. P(t) is equal

for all classes so that it can be ignored [55]. (2.3) Based on the Bayes theorem assumption which says the features are conditionally independent, the probability of class Ci can be given as below:

25

Chapter 2

Text Classification Basics



(

)

(2.4)

n is the number of features fi that form training samples. The class of the test instance t is defined by NB classifier: ∏

(

)

(2.5)

VNB is the product of Naive Bayes classifier which indicates to the class of test instance. Despite the features independence assumption is unrealistic, Naive Bayes has been discovered extremely effective for many functional applications for example, text classification and medical diagnosis even when the dimensionality of the input is high [55, 56]. Appendix-A1 explains steps of this algorithm. Below shows the description of advantages and disadvantages of Naïve Base classifier.

a) Advantages of NB 1) Include simplicity and efficiency. 2) Robustness and interpretability. b) Disadvantages of NB 1) It does not work properly with data having noise present with them. 2) Remove all the noise before applying NB classifier is requirement [57].

26

Chapter 2

Text Classification Basics

2.8.2 Decision Trees Classifier (DT) Decision tree algorithm is widely used in machine learning and data mining fields, this is because it is simple and can be easily understood and converted into a set of humanly readable if-then rules [58]. ID3 algorithm is one of the most well-known decision tree algorithms. C4.5 is an extension of ID3. In this study, the C4.5 algorithm is used via which the decision tree mechanism is applied for classifying unseen instances to test at each node some feature values for finding the class of unseen instance where the test starts at the root node and goes down to a leaf node. A decision tree is reached when that leaf node determines the class of unseen instance [59, 60]. Consider < a1; b2; a3; b4 > be an unseen instance where decision tree classifier is built based on the classification of unseen instance is yes [59]. Figure 2.2 is an illustrative sample of a decision tree for the set of training samples appeared in Table 2.1.

Table 2.1: Set of training examples [59] F1 a1 a1 a1 a1 c1 c1 c1 c1

F2 a2 a2 b2 b2 c2 c2 c2 c2

F3 a3 a3 a3 b3 c3 c3 c3 c3

F4 a4 a4 a4 b4 a4 c4 c4 c4

27

Classes Yes Yes Yes No Yes No No No

Chapter 2

Text Classification Basics

Figure 2.2: An Example of Decision Tree [59]

Information gain (IG) is a great measure for choosing the best feature where the feature with highest information gain is chosen to be the root node [59]. Appendix- A2 explain the basic steps of this algorithm. Below shows the description of advantages and disadvantages of Decision Tree classifier. a) Advantages of the DT 1) Include straightforwardness, interpretability and capacity to handle feature interactions. 2) In addition, DT are nonparametric, which makes issues like exceptions and whether the dataset is linearly divisible [9]. Nonetheless, there is Pruning (Reducing errors) which is a sort of machine learning systems that decreases the size of decision trees through expelling those sections of the tree that give little energy to 28

Chapter 2

Text Classification Basics

classifying instances. It is utilized to minimize the complexity of the final classifier. It can be utilized to decrease the over fitting situation and remove the noisy or wrong information from the tainting set [60]. b) Disadvantages of the DT 1) DT lacks the support for online learning and suffers from the issue of overfitting, which can be handled using different strategies like random forests (or boosted trees) or the problem of overfitting could be avoided by pruning the tree [14].

2.8.3 Support Vector Machine Support Vector Machine (SVM) is a supervised machine learning algorithm which was proposed to handle text classification by [61]. Researchers have used SVM widely in a text categorization task, such as in [14], [62]. In Ndimensional space, input points are mapped into a higher dimensional space and then a maximal separating hyperplane is found SVM technique classification depends on the Structural Risk Minimization principal [63]. Suppose that there is a set of training vectors space belongs to a set of classes’ 𝑦

n,

i= 1…. L, in the feature

{−1, 1}. Figure 2.3 is a case of an optimal

hyperplane for separating two classes. From Figure 2.3, SVM builds a hyperplane that perfectly separates a set of positive patterns from a set of negative patterns with a maximum margin in case of linear [59].

29

Chapter 2

Text Classification Basics

Figure 2.3: SVM Builds a Hyperplane that Perfectly Separates Classes.

For linearly separable vectors, the function can be expressed as: (2.6) w is weight vector for optimal hyperplane, b is known as the bias and x is test instance. Then, the class of x can be found using the following linear decision formula [63]: 𝑦

(2.7)

One of the issues of SVM is to find the most extreme separation between the closest training samples which means the decision boundary should be as far away from the data set of all classes as possible; in another significance, the margin m should be maximized. The distance is given by: (2.8) Therefore, the hyperplane which minimizes w is considered as the optimal hyperplane and it is written as the following formula [64]: (2.9) 30

Chapter 2

Text Classification Basics

The maximal margin hyperplane can be given as following when using Lagrangian formula [64]: ∑

𝑦

(2.10)

Where, the kernel function is k(x; xi) , yi is the class label of support vector xi, x is a test vector, ai is a Lagrange multiplier for every training vector; vectors for that ai > 0 are called support vectors, b is a numeric parameter and n is the number of support vectors. In context of the text classification problem, xi symbolizes the ith document in the training set and yi indicates the class of that document (e.g. art, sport, religion, science, etc.) [63], [64]. For non- linearly classification data, different kernel functions can be used. The most widely kernel functions are [63]: (

Linear:

)

(2.11)

Sigmoid:

(

)

(2.12)

Polynomial:

(

)

(2.13)

Radial Basis Function: (

)

(



‖ )

(2.14)

Where, γ, r, and d are kernel parameters. The linear kernel function is used in this thesis scope since there are a very large number of features as in the document classification problem, so that is the proper choice and there is no need to map the data [69]. Below shows the description of advantages and disadvantages of SVM classifier [14]. a) Advantages of SVM 1) Involve high accuracy. 2) Great theoretic certifications about overfitting and, with a suitable kernel. 31

Chapter 2

Text Classification Basics

3) Work well regardless of whether the data is linearly separable or not. Property in text categorization problems their ability to learn can be independent of the dimensionality of the feature space.

b) Disadvantages of SVM 1) Involve complexity 2) Poor interpretability 3) High memory requirements.

2.9 Evaluation and Performance Measures In text mining, assessment of the exactness of machine learning calculations is a fundamental stride. To category given data, a set of training data is utilized to construct the classification model. For assessing the precision of the acquired classifier, two basic approaches, hold-out and crossvalidation are utilized to survey the capacity of the classifier to predicate the right class or category of unseen instances.

2.9.1 Hold Out In the hold out strategy, the accessible data is subjectively divided into two separate sets, training set and a testing set. Normally 66% of the data are held for training further. An issue may emerge when one of the classes is not represented in the training portion of data. This issue is illuminated utilizing what is called stratified hold out. For this situation, the chosen test contains cases from all classes of the data. As it were, all classes are spoken to in both data sets [65] [66].

32

Chapter 2

Text Classification Basics

2.9.2 K-fold Cross Validation In this technique, the information is randomly part into K subsets or folds, and the holdout method is repeated k times. Each time, one of the k subsets is used as the test set and the remaining k-1 subsets are put together to form a training set. Then, the average error over all k trials is calculated. Figure 2.3 Shows diagram of k-fold cross-validation with k=10. Commonly, the stratified version of K -fold cross validation is utilized to guarantee that all given classes are represented in all folds [65], [67], [68]. Below shows the description of advantages and disadvantages of K-fold Cross validation. a) Advantage: 1) It matters less how the data gets divided, each data point gets to be in a test set completely once, and gets to be in a training set k-1 times. The variance of the resulting estimate is reduced as k is increased [67], [68]. b) Disadvantage: 1) The training algorithm has to be rerun from scratch k times, which means that it takes k times as much calculation to make an assessment [67], [68].

Figure 2.4: Diagram of k-Fold Cross-Validation with K=10.

33

Chapter 2

Text Classification Basics

2.9.3 Error Rate Error rate is the rate of mis-classified sample in a given test set. Consider that a test set G comprises of N samples, and r is the number of mis-classified samples by a classifier. The accuracy of the classifier for correctly predicting the classes of the sample in G can be assessed as follows [66]: (2.15) For more credible valuation, normal appropriation is utilized to evaluate the accuracy. In the event that the dataset size is not small, the evaluated accuracy is given as [66]: (2.16) The accuracy is in the range: (2.17)

The disadvantage of this method; the error rate strategy is that it disregards the cost of wrong prediction which is critical in machine learning. This issue can be avoided using F-measure [66].

2.9.4 F-measure F-measure is the most commonly used method to evaluate the text categorization effectiveness [31]. The F-measure, introduced by [71], is the harmonic average of both precision and recall. Consider the documents in the test set that is category C. The classifier predicts a class for each document, and these expectations will fall into four classes as for category C [66].

34

Chapter 2

Text Classification Basics

-True Positives (TP): the number of documents that are in category C, and were correctly assigned to be in the category C. -True Negatives (TN): the number of documents that are not in category C, and were assigned not to be in the category C. -False Positives (FP): the number of documents that were predicted to be in category C, but in fact they are falsely assigned to the different category. -False Negatives (FN): the number of documents that were predicted not to be in category C, but are actually in category C. Precision is the proportion of predicted category C documents that were correctly predicted, as calculated using the equation: (2.18) Recall is the proportion of actual category C documents that were correctly predicted, as calculated using the equation: (2.19) The F-measure is computed according to the following formula: (2.20)

35

Chapter 2

Text Classification Basics

2.9.5 Confusion Matrix In general, confusion matrix is adopted for effectiveness evaluation in classification problems to estimate correctly and incorrectly classified for each class. The confusion matrix has only two classes – positive and negative for a binary classification problem [69]. Table (2.2) is describing Confusion Matrix. Table 2.2: Confusion Matrix. Class

Positive

Negative

Positive

TP

FN

Negative

FP

TN

In machine learning and statistics to examine the performance of classification algorithms, the accuracy rate (Acc) and error rate (Err) are used. The accuracy rate is the percentage of correct predictions of the classifier and the error rate is the percentage of incorrect predictions of the classifier also called mis-classification [69] [70]. They are calculated as the following: 𝑦

(2.21)

(2.22)

36

Chapter 2

Text Classification Basics

2.10 General Kurdish Formulation Kurdish language is in the Indo-European family of languages. This language is spoken in Kurdistan, a large geographical region spanning the intersections of Iran, Iraq, Turkey, and Syria [75]. The Kurdish language is generally divided into two dialects: Sorani and Kermanji. The writing script of Sorani Kurdish alphabets reads from right to left. This is similar to Persian, Arabic, and the Urdu languages. The Kurdish alphabet consists of 33 characters as shown in Figure 2.5. Researchers in [76], [77] and [78] have identified that the morphology of the Kurdish Sorani dialect, which has not been otherwise classified.

‫ا ه ب پ ت ج چ ح خ د ر ڕ ز ذ س ش ع غ ف ظ ق ک گ ل ڵ م ن و ۆ وو هـ ی ێ‬

Figure 2.5: The Kurdish Alphabet [5].

The syntactic structure of the Kurdish Sorani dialect depends on the basic word or root. The root word produces both nouns and derivational verbs via adding affixes to the base (root). It is worth mentioning that affixes in the writing system of Sorani can be in the form of nouns that are attached to their roots; for instance, “‫ ”دارتاش‬means carpenter, which can be further described as “‫ ”تاش‬which is the root and “‫ ”دار‬which is the noun. Or prepositions can be attached to roots such as “‫”ياريدا‬, which means in play, which can be further explained as “‫ ”يارى‬which is the root play and “‫ ”دا‬which is the preposition “in”.

37

Chapter 2

Text Classification Basics

Generally speaking, in the Kurdish language affixes can take three forms: 2) Prefixes are attached to the beginning of a word. 3) Suffixes are attached to the end of a word. 4) Postfixes are attached to the end after suffixes. Therefore, the words in Kurdish language can get quite complicated, if all these affixes are attached to their roots for example, a word “‫”لەياريگايەکان‬ (layarigakan), means “from playgrounds ". Table 2.3 shows a word and its affixes. Table 2.3: Example of Kurdish Affixes. Word

Root(base)

Prefix

Suffix

Suffix

Postfix

‫لەیاریگایەکان‬

‫یاری‬

‫لە‬

‫گا‬

‫یـ‬

‫ەکان‬

Layarigakan

Yari

La

Ga

I

Akan

in play grounds

The word"play"

From

PlaceMarker

For the existence two vowel( ‫ا‬and ‫)ە‬

Plural definite Marker

Accordingly, from the above example, it can be seen that the Kurdish Sorani dialect has a comparatively complex morphology. Thus, affixes can be removed from a word, then the stemmed word or the root is produced.

38

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

Chapter Three Proposed Approach for Kurdish Sorani Text Classification 3.1 Introduction In the previous chapter, the key ideas of the text mining field and its significance were identified and investigated, and in particular in the text classification. In addition, different procedures and techniques that have been utilized previously as part of text classification. Besides, the Kurdish language formulation and the morphology of the Kurdish Sorani dialect are presented. In this chapter, the structure of the proposed procedure for Kurdish Sorani text classification system will be depicted. Beginning with introducing the architectural plan of the system, and then clarifying the phases of executing the system.

3.2 System Structure The Kurdish text classification framework in this work can be divided into five main stages: data collection, data pre-processing, data representation and term weighting, experimentation, classifier construction to predict category and evaluation and comparison. Figure 3.1 shows the frame structure of the proposed Kurdish Sorani text classification system. Every stage consists of several steps, or modules. First, a dataset has been collected from different Kurdish sources. The collected Kurdish data is contained in three samples to test the preprocessing stage and overall of system. 39

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

Preprocessing is performed to prepare the documents which consist of the following steps: Normalization, Kurdish Stemming Steps module, and Stop words Removal which plays an important role in the project. The result of the preprocessing is used as an input for the next stage (Term Weighting) as described in section 2.7.3.

Figure 3.1: The Frame Structure of the Kurdish Sorani Text Classification System 40

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

The first step in the classification of the text is to convert the documents, which are usually strings of characters, to an appropriate representation for the learning algorithms and the classification task. Different experimentations are applied onto three classifiers namely; SVM, NB and C4.5 classifiers which are discussed in the sections 2.8. Each Classifier is tested in two ways namely; cross validation and conventional validation (percentage split). In the percentage split, the data set is divided into two parts, the first part is 70% of the data set which can be used as a training set, and the second part is 30% of the data set which can be used as a testing set. There is no standard for choosing the size of the training and testing sets, however, more and more data is required for the training. In the 10-fold cross validation, which is very common strategy for the assessment of classifier performance, the data is divided into 10 folds; nine folds of the data are used for the training, and one-fold of the data can be used for testing. Further details on both ways of training are explained in sections 2.9.2 and 2.9.3. The final stage is to build a classifier and the prediction module for testing the new and unseen documents. Detail a descriptions of the above five stages are in the following sections.

3.2.1 Data Collection Since there are no standard test collections available for the Kurdish Sorani dialect, the data collected especially from three sources, these are namely; Rudaw, which is a Kurdish daily online news and information website about Kurdistan (http://www. rudaw.net/Sorani), and Nrttv, which is another Kurdish daily

online

news

(http://www.nrttv.com), Sulaimany

News

Network

(http://www.snnc.co), and other websites. After collecting the data sets from these sources, one document collection is created, which contains 4,007 pieces of text data and a total of 85,830 words. 41

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

After the tokenizing process, it is reduced to 23,389 unique words. The pseudocode of the unique words is explained in (Appendix B). As a result of this process, these documents can easily be used for experiments and further study. Since datasets in general are not accessible for tests and text classification studies, this new dataset called KDC-4007 is created. The most important feature of this dataset is its simplicity to use and its being well-documented, which can be widely used in various studies of text classification regarding Kurdish Sorani news and articles. The documents consist of eight categories, which are Sport, Religion, Art, Economic, Education, Social, Style, and Health. Each of them consisted of 500 text documents, where the total size of the corpus is 4,007 text documents. The dataset and documents have become freely accessible in order to have repeatable outcomes for experimental assessment on KDC-4007 dataset in [81]. Table 3.1, shows the distribution of the documents among these eight categories. Three samples are created from the collected data: 1. Sample I: 85,830 words are taken to test Kurdish stemming and stop words in the whole document. 2. Sample II: This sample consists of 350 groups of words (root words) and 1,702 derivational words. In this sample, the words are linguistically grouped according to their roots taken from the document, and the assessment of grouping is done manually. 3. Sample III: The KDC-4007 dataset of Kurdish text collected from different websites which consisted of eight categories used to test the Kurdish Sorani text classification.

42

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

Table 3.1: Number of Classes and Documents in Selected Sources. Class

#Docs

#Classes

Sources

1047

3

http://www.rudaw.net/sorani

755

3

http://www.nrttv.com

1705

4

http://www.snnc.co

Religion

500

1

randomly compiled from Internet

Total

4,007

8

Sport Health Economy Sport Economy Education Style Socials Education Art

3.2.2 Data Pre-processing In this section, the proposed Kurdish Sorani stemming approach of the preprocessing stage is presented. Clearly, the complex morphology of Kurdish Sorani makes it very difficult to develop and handle the processing of natural language for digital information retrieval. Accordingly, the stemmer is a tool that can be useful for normalization and stop-word removal. In addition, it is useful for information retrieval in combating the problem of vocabulary mismatch. This problem commonly occurs in numerous applications, and can be tackled during the pre-processing stage. The architecture of the proposed approach for Sorani texts is shown in Figure 3.2. When the texts are collected, they are sorted and coded in a variety of ways that make evaluation difficult. Thus, before they can be subject to electronic searching, all documents and texts must be standardized and encoded into Unicode (UTF-8). The proposed

43

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

approach involves four main steps which can be clarified by the following subsection:

Figure 3.2: Architecture of Kurdish Sorani Text Pre-process.

3.2.2.1Tokenization Tokenization is regarded as a basic, and significant step in the natural language process [79], [80]. The simple reason for this process is to transform a text stream into tokens by segmenting the texts into smaller units of meaning. The process of tokenization involves breaking down the sentences in the text document file into words delimited via tabs, different lines, or white space [79], [80]. Tokenization results in a list of valuable semantic tokens. Therefore, this process helps using a given word in the next phase. Figure 3.3 shows an example of this stage.

Figure 3.3: Shows Example of Tokenizing Process. 44

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

3.2.2.2 Kurdish Normalization The features of Unicode characters are considered as a base for the process of normalization. It is notable that Kurdish writing uses Arabic electronic texts. This will cause differences in letters to some extents. Plainly, this sort of discrepancy issue is generated via typescripts of multiple Unicodes, and eventually, recall in retrieving relevant information is negatively impacted. For that reason, written texts are unified to tackle this issue. This sort of unification improves subsequent steps of pre-processing. Furthermore, this similarity helps to remove affixes, and relates words with each other when removing stop words. The main aim of the proposed approach is to prepare consistent words for the next steps in the pre-processing phase [82]. Essentially, the normalization process is conducted via the following steps to tackle Kurdish Sorani dialect as shown in Figure 3.4. Algorithm 3.1: Normalization Algorithm. Purpose: Normalize Kurdish Sorani Words. Input: Dataset. Output: Normalized Dataset. Procedure: 1. Converting dataset into UTF-8 encoding

2. Replace Arabic letter DOTLESS_YAA (‫ )ي‬with Arabic letter KURDISH_YEH (‫)ی‬. 3. Replace Arabic letter ALEF MAKSURA (‫ )ى‬with Arabic letter KURDISH_YEH (‫)ی‬. 4. Replace Arabic letter KAF (‫ )ك‬with Kurdish letter KEHEH (‫)ک‬. 5. Replace (“‫ "ـە‬which is consisted of “ZWNJ + HEH (‫ )")هـ‬with Kurdish letter AE (‫)ه‬. Figure 3.4: Procedure Normalization Words from Kurdish Documents.

45

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

3.2.2.3 Kurdish Stemming- Steps The proposed approach in this paper is used for removing affixes. This is a stemming step analysis, which is a step-based approach to stages a word goes through before arriving at the extracted root of the word. Some conditions have been re-adjusted for the letters which are affixes positioned at the end of words in the Kurdish Sorani dialect. Accordingly, this stemmer is specified to the words that have several affixes (for example, a word that has: “prefix” + “root” + “suffix 1”+ “suffix 2” +..........”suffix N”). Thus, a given word can go through a group of simplified guidelines depending on conditions to get the root of word. This approach depends on collections of potential prefixes and suffixes, which are commonly utilized in Kurdish text documents. Table 3.2 demonstrates Sorani prefixes, and suffixes which require exclusion [82].

Table 3.2: Prefixes, Suffixes and Postixes Removed by Kurdish Stemming- steps. Prefixes

Suffixes and Postfixes

‫ ڕا‬,‫ لە‬,‫دە‬,‫ سەر‬, ‫دەر‬, ‫هەڵ‬

, ‫دن‬, ‫ ان‬, ‫ ین‬, ‫ تن‬, ‫ەوە‬, ‫ وە‬,‫ یە‬, ‫ هکە‬, ‫ ێک‬, ‫ و‬,‫هکان‬,‫ مان‬,‫ تان‬,‫ یان‬,‫ گا‬, ‫ ی‬, ‫ ش‬,‫دا‬ ‫ هات‬,‫ كار‬, ‫ بوون‬,‫ بوو‬,‫ ترین‬,‫ تر‬, ‫ ێت‬,‫ م‬,‫ن‬, ‫ە‬

This approach starts validating a word that ends with affixes. The Kurdish Stemming-step module is designed to strip prefixes, suffixes and postfixes from the given word to catch potential roots. The given word will be checked through all the steps in the Kurdish Stemming-step process to map the string of letters that are positioned at the beginning or the end of the root of the word.

46

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

It is worth noticing that the Kurdish Stemming-Steps technique does not use a dictionary for checking the root. For instance, the word (‫ )لەهەنگاوهکانیان‬is reduced to its root ("‫"هەنگاو‬means "step") right through step1 which removes prefixes (‫)لە‬, and then goes through the next step for mapping it up until it arrives at step7 to remove the suffix (‫)يان‬, then produces (‫(هەنگاوهکان‬. After that, the approach associates the suffix of the given word with suffixes in the steps followed later to arrive at the step where it finds a match, and removes it from the word at step10. Thus, the suffix (‫ )کان‬is removed. Finally, in the step 13, the suffix (‫ )ه‬is matched, and then removed. The overall process of Kurdish stemming for a given word can be shown in Figure 3.5. This approach is not only used to remove affixes from nouns and verbs as it is used in other languages, but also it removes affixes from the stop words which are widely used in Kurdish Sorani, for example, see Table 3.3. Table 3.3: Example of Stop Word Affixes. Stop word ‫ پاشانەوە‬,‫ لەپاشاندا‬, ‫لەپاشی‬

Root of stop word ‫ پاشان‬, ‫پاش‬

English meaning After

‫ ئێوەش‬, ‫ ئێوەى‬,‫ ئێوەمان‬,‫ئێوەیان‬

‫ئێوە‬

Your

‫ چەندە‬, ‫چەندین‬

‫چەند‬

Some

‫ ئەوەتان‬, ‫ ئەوەی‬, ‫ئەوەیە‬

‫ئەوە‬

That

Not only these Stop words that were repeated here, but there are many other Stop words in Kurdish Sorani texts that have affixes. These are just examples for clarification.

47

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

The algorithm for stemming the Kurdish Sorani words from a given Kurdish documents is as follows: Algorithm 3.2: Proposed Kurdish Stemmer-steps Algorithm. Purpose: Stemming Kurdish Sorani Words. Input: Dataset Output: all words stemmed in Dataset. Procedure:

Figure 3.5: Overall Steps of Stemming of the Kurdish Sorani Documents.

48

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

3.2.2.4 List of Kurdish Stop Words In order to make the environment of information retrieval precise, some of the words that are regarded as not significant must be removed, due to their meanings in the Kurdish sentences. These words are common in Kurdish Sorani texts, and as a result, they intensify the noise of the results. Most of these words are prepositions or pronouns. A predefined list can be prepared to contain these words that do not serve the process of information retrieval, but which are used very regularly in the Kurdish texts. (Appendix - C) shows a list that contains nearly (240) Stop-words, and the list of stop words is developed for two main reasons:

1) The words that correspond between a term and a document need to be kept. This depends considerably on the words that hold essential meaning. Thus, noise words should be removed. Retrieving documents which contain words such as ( ‫ بۆ‬meaning “to”), (‫ ئێوە‬meaning “yours”) and (‫ نێو‬meaning “in”) in the identical request do not establish a relevant intelligibility. These noise words are non-significant, and they may cause damage to the performance of retrieval, as they do not distinguish between relevant and non-relevant documents. 2) Besides, the richness of the stop words in Kurdish Sorani increases the size of the feature vector. The proposed approach attempts to reduce the size of the file from 35 % to 50%.

49

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

The procedure in Figure 3.6 for filtering the Kurdish Sorani stop words from a given Kurdish documents.

Figure 3.6: Flowchart for Removing Sorani Stop-Words from Kurdish Documents.

The output of the above procedure is a set of words (features) without stopwords in the corpus.

50

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

However, developing and integrating Kurdish Sorani morphological analysis tools (preprocessing-steps) in one package which can be explained as shown in Figure 3.7. Using linguistic expertise to design a preprocessingstep module.

Figure 3.7: Flowchart of Proposed System for Kurdish Documents. 51

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

In Figure 3.8 the description of the pseudo code about the whole process that this thesis is following and suggesting for purposed Kurdish preprocessing.

Algorithm 3.3: Kurdish Sorani preprocessing-steps Algorithm Purpose: Stemming Kurdish Sorani Words Input:  Dataset  Stop-Word list Output: Normalizing, Stemmed Dataset, removing stop- words. Procedure: 1. Converting dataset into UTF-8 encoding 2. Read input files 3. Remove punctuation, diacritics, non-letters, and non-Kurdish word. 4. Apply normalizing Algorithm 3.1 5. Apply Stemming- step to remove prefixes and suffixes Algorithm 3.2 6. Apply Stop-word filtering as shown in section 3.2.2.4 Figure 3.8: Kurdish Preprocessing-Steps Algorithm

3.2.2.5 Evaluation of the Stemmer Algorithm The evaluation process which is proposed by Paice [72] is performed on the proposed approach. Paice evaluated various English stemming approaches isolated from the context of information retrieval systems. Instead of using the traditional precision, or recall, parameters, he relied on new parameters; namely, the over-stemming index (OI) and the under-stemming index (UI), and their ratios and weight of stemming (SW) [73], [74]. 52

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

To test the stemmer, sample II is generated from different words partitioned into concept groups, each of which contains forms, which are both morphologically and semantically related to one another. A flawless stemmer can stem all words in a group to the same stem, and that stem must not then be found in any other group. An under-stemming error can occur if a stemmed group has more than one unique stem. This corresponds to an undesirable outcome on recall in information retrieval systems. By the same token, overstemming errors can occur if a stem of a particular group also happens in other stemmed groups, in which precision is reduced. Accordingly, it is preferable to have a stemmer that can feasibly generate as few under-stemming and overstemming errors as possible [74]. For each concept group g, two totals can be calculated: A flawless stemmer has to merge every member of a concept group with each other. The total number of different possible words form pairs in the particular group describes the desired merge total. This can be expressed as follows [72]: (3.1) Ng, represents, the number of words in the group. A flawless stemmer would not unify any member of the present concept group with any word that is not in the group. Consequently, a desired nonmerge total which counts the possible word pairs formed by a member and a non-member word for every group can be expressed as follows [72]: (3.2) W represents the total number of words. Ultimately, both the global desired merge total (GDMT) and global desired non-merge total (GDNT) can be obtained right after summing the DMT and DNT for all groups in the word 53

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

sample. It is found that some of the groups still contain two or more distinct stems when a stemmer to the sample group is applied. Thus, there are understemming errors to be counted in such groups. Assume that a concept group of size Ng contains s distinct stems after stemming, and the number of instances of these stems are U1, U2, and U3 respectively [72]. The Unachieved Merge Total (UMT) counts the number of under-stemming errors for that group and it can be expressed as follows [72]: ∑

(3.3)

The Global Unachieved Merge Total (GUMT) is obtained by summing the UMT for each group. The under-stemming index (UI) is given by: (3.4) Over-stemming errors can be caused when a stemming might find cases where the same stem occurs in two or more different groups. Part of two or more different concept groups in any stem group can contain over-stemming errors which are needed to be counted through the Wrongly Merged Total (WMT). The WMT value for that group can be zero if a group does not contain over stemming errors. Assume a stem group of size Ns items which can be derived from t different concept groups and the number of each original concept group within this stem group can be represented via V1, V2, V3…, Vi as shown below: ∑

(3.5)

The Global Wrongly-Merged Total (GWMT) is obtained via summing the WMT for each group. The over-stemming index (OI) can be expressed as follows:

54

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

(3.6) Clearly UI and OI should be low for a heavy stemmer. The stemming weight is defined as the ratio of these two [72]: (3.7) SW is used as a parameter to measure the strength of a stemmer. A stemmer is weak when the value of SW is low and the stemmer is strong when SW is higher. Figure 3.9 illustrates how this evaluation method works.

Figure 3.9: Illustration of Paice Evaluation Method.

55

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

3.2.3 Experimentation of Methodology This section explains the methodology and classification process for three classification algorithms. We proposed to experiment the same data, but differently, this means that the testing is done for more than once for three algorithms so as to evaluate them. The three classifiers are conducted using six different representations of the same dataset. In general, all common operators, punctuations and non-printable characters are removed. The original dataset without any preprocessing task is used in the first experiment (Test1), in the second experiment (Test2), normalization and Kurdish Stemming-step module are used as the preprocessing task of datasets. In the third experiment (Test3), normalization and the stop word removal were applied in the datasets. In the fourth experiment (Test4), all steps of preprocessing were used which involve normalization, Kurdish Stemming-step module and stop word removal. In the fifth experiment (Test5), all steps of preprocessing were also used besides the application of TF×IDF weighting. Finally, on the original dataset without any preprocessing TF×IDF weighting was used as a last experiment (Test6). Test5 and Test6 experiments respectively, studied the impact of TF×IDF weighting on the accuracy of Kurdish Sorani document classification to reveal all possible interactions for TF×IDF feature weighting method. Table 3.4, shows the details of the experimentations about KDC-4007 dataset version. The other observation related, while using two popular process, which are 10-Fold Cross Validation and Percentage Split that they give the different performance accuracy value compared with the original dataset for each classifiers.

56

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

Table 3.4: The Dataset Versions of Experimentation of Methodology Dataset name

Normalization

KStemmingstep module

Stop- word filtering

TF-IDF weighting

Number of documents

Number of Feature

Test1

No

No

No

No

4,007

24,817

Test2

Yes

Yes

No

No

4,007

13,309

Test3

Yes

No

Yes

No

4,007

20,150

Test4

Yes

Yes

Yes

No

4,007

13,128

Test5

Yes

Yes

Yes

Yes

4,007

13,128

Test6

No

No

No

Yes

4,007

24,817

3.2.4 Data Representation and Term Weighting In this section, each text document in dataset was transformed into a feature vector. In this way, each text file in the dataset is represented as a vector of features (words) and every dimension compares to a separate term (word). In the event that a term happens in the record, then its value becomes non-zero in the vector. When it is considered from text classification point of view, the objective is to build vectors containing features per class by utilizing a training set of the documents. However, text document contents in all variants of experiments (Tests) are changed into document term matrix by using text to ARFF, which is a format file of WEKA. Then, every matrix having a place with dataset adaptations are changed over into attribute relation document organize (ARFF) that is a legitimate organization for WEKA to be executed. Term weighting is one of the critical and fundamental steps in text classification that depends on the statistical analysis approach, more details are given in 2.7. In both experiments, Test5 and Test6, for all classifiers, we endeavor to utilize

×

weighting schemes to mirror the relative

significance of every term in a document and to reduce the dimensionality of the feature space. Accordingly, a term that has a high 57

×

weighting value

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

must be simultaneously remarkable in this document and must show few times in alternate documents. It is considerably the situation where a term associates to an important and unique of a document. Figure 3.10 shows the KDC-4007 dataset after using TF×IDF Term Weighting.

Figure 3.10: WEKA Data Viewer for KDC-4007 Dataset after Using TF×IDF Term Weighting.

3.2.5 Machine Learning Classifiers In a general description, the point of text classification is classifying uncategorized documents into predefined categories. When we look from machine learning point of view, the aim of text classification is to learn classifiers from labeled documents and satisfy categories on unlabeled 58

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

documents. In literature, there is an affluent set of machine learning classifiers for text classification. The determination of the best performing classifier relies on various parameters, for example, dimensionality of the feature space, number of training examples, over-fitting, feature independence, straightforwardness and system’s requirements. Taking into consideration the high dimensionality and over-fitting aspects, three well-known classifiers (C4.5, NB and SVM) are chosen among all classifiers in our experimentation.

3.2.6 Evaluation Metrics In the field of machine learning, there are various evaluation criteria used to assess classifiers. In this study, the four popular evaluations; accuracy (ACC), precision, recall and F1-score are utilized. Their mathematical equations are illustrated in section 2.9.4 and 2.9.5. Accuracy is the most widely used on a large scale to assess the standard of performance, which is the proportion of the total number of class files that are properly classified. In addition, the time to build the model is involved in the analysis of comparatives. The classifiers compare the effectiveness of the proposed approach to measures how accurate the classification was by counting the number of correctly classified instances (CCI) and the number of incorrectly classified instances (ICI).

3.3 Implementation and Practical Work A software is built to develop pre-processing for Kurdish Sorani text using java programming language. The developed pre-processing contain three classes for Normalization, Stop-Words Remover, and Kurdish Stemming Steps module which plays an important role in the project; and then main class to load and save the Kurdish Sorani documents as text. 59

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

WEKA Tool is used for creating and evaluating classifiers [83]. It’s a collection of machine learning algorithms for the well-known data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. In our experiment, WEKA Tool called in the Java code, the structure of working is explained in (Appendix - D). Three learners were used,

namely; SVM, NB and C4.5 with the standard WEKA settings. However, in our implementation we utilize the hardware of a custom build PC with the following specifications:  CPU: Intel Core i3 and speed of 2.40 GHz.  Memory: 4 GB.  OS: Microsoft Windows 7 32-bit operating system. For software we utilize the following:  Programming Language: Java with JDK8.  IDE: NetBeans IDE 8.1 version.

The general format of the ARFF files have two main sections, relation and data. As in the example shown in Figure 3.11., the relation section starts with the relation name “e.g. KurdishSoraniDocuments-KDC-4007”, and then, defines list of attributes for input features with their type (e.g. ‫ ئادەم‬numeric, ‫ئارام‬ numeric, ‫ ئازار‬numeric), class (output) attribute with its type (e.g. class {religion, sport, health, education, art, social, style, economy}). The last part of the relation section defines the possible classes of instances. Data section contains the data instances where each line represents an instance while each column is an attribute value. The last column refers to the classification of instance as shown in Figure 3.11. In some cases as in text document representation, most of attributes values have values of zero. Text file usually has a small number of words in comparison with the number of features used to 60

Chapter 3

Proposed Approach for Kurdish Sorani Text Classification

represent texts in text mining. Most of the values in such case are zero as in Figure 3.10. For example: {0, 0, 1, 1, 1, 0, 1, 0, art}. {0, 1, 0, 0, 1, 1, 1, 0, 0, economy}. In WEKA, there is another way called sparse ARFF can be used for data representation. In this method, attributes are indexed starting from zero, each nonzero attribute is identified with the index of attribute. The above example can be rewritten in ARFF sparse file format as in Figure 3.11. {988 1, 1284 1, 1545 1, 1608 1, art}. {414 1, 505 1, 692 1, 1067 1, economy}. The first element (988 1) means the index is 988 and the attribute value is 1.

Figure 3.11: WEKA Data Format for Text Classification.

61

Chapter 4

Test Results Evaluation

Chapter Four Test Results Evaluation 4.1 Introduction In this chapter, the most important testing results are presented and discussed to reflect the performance capabilities of the pre-processing steps on Kurdish Sorani text classification. Sample I and Sample II from sources were taken and used for assessing the quality of the suggested stemmer and stopword list. As part of the Kurdish text classification, removing stop words and stemming of documents before and after applying classifier algorithms are evaluated. For this purpose, Sample III is chosen as a variant of experiments to test the impact of our developed pre-processing on the performance of Kurdish text classification. The results give a comparable measurement, which can be used for selecting an accurate algorithm for Kurdish text classification and for identifying the causes of getting a low rate of accuracy in some Tests.

4.2 Analysis of Sample Results for Kurdish Sorani Text In this work, Kurdish text documents have been evaluated according to the following samples:

4.2.1 Experimental Results for Sample I As part of the pre-processing steps, removing stop words from documents before and after applying the stemmer is assessed. Firstly, sample I of data collection is taken to measure how many stop words are deleted correctly from documents. 62

Chapter 4

Test Results Evaluation

The results of the experiments of sample I can be seen in Table 4.1, which shows the average number of words per document is greatly reduced after stemming. This shows that the removal of the highly frequent stop words from the text after applying the stemmer is increased. Therefore, it can be suggested that the stop-word list can be likewise useful in pre-process steps to improve retrieval evaluation. Table 4.1: Stop Words Removed from Documents Test on Sample I. Analysis Of Stemmers Words in Doc Total No. of word Total No. of Stop words removed

Without Stemming

With Stemming

85,830 words

23,389 unique word

13,005 unique word

83,870 words

62,679 words

64,589 words

Figure 4.1 (a) describes the stop words that were removed from the document before and after stemming. These words are non-informative data in the documents and affect the process accuracy and retrieval. They detract from the effectiveness of the process. Therefore, from here we note that our approach has an impact on the pre-processing step to reduce the dimensions of feature space by eliminating noisy, irrelevant, and non-informative data while retaining relevant and informative items. In another experiment on Sample I, the effectiveness of grouping various types of a word to the same stem and word reduction in a document is evaluated after removing all stop words. In sample I 23,389 unique - word are obtained. This process is used to reduce the nonconflated words from conflated words. For example, if the document contains the words "playing”, “played", and "player", they are reduced to one word (the root), which is "play". The Kurdish stemming-step module achieved 55% of word reduction. This process affects a high dimension of features and thus in turn affects the accuracy in the classification process as well as the retrieval of the text. 63

Chapter 4

Test Results Evaluation

Figure 4.1 (b) shows the number of words without stemming and with stemming using the Kurdish stemming-step module. Word reduction is calculated via the following equations: (4.1)

Figure 4.1: The Experimental Results of Kurdish Stemming Steps Module: (a)The Removal of Stop Word; (b) The Word Reduction.

4.2.2 Experimental Results for Sample II Paice’s Evaluation Method (group collections) detail in section 3.2.2.5 is used to evaluate sample II, which has been prepared to be consistent for this sort of evaluation. Table 4.2 contains the test results of the over-stemming, understemming and the weight measures. Although the results indicate that the Kurdish Stemming-step formulation is very strong and effective with inflection and derivation, it still transforms some of the root words to incorrect stems, which is caused by both over-stemming and under-stemming errors. In all, the 64

Chapter 4

Test Results Evaluation

occurrence of over stemming errors is greater than under stemming errors. The test shows that the Kurdish Stemming-step module is strong, and this appeared obviously on the stemming weight, which is a large value. A weak stemmer produces more under-stemming than over-stemming errors, and a strong stemmer does the reverse [74]. Table 4.2: Results of Paice’s Evaluation Method using Sample II. Kurdish Stemmer-step

Result

Over-Stemming (OI)

0.34

Under-stemming (UI) Stemming Weight (SW)

In addition, sample II is tested after applying the Kurdish Stemming-Step formulation to measure how many words are stemmed correctly, which is calculated by counting the correct stem words coming from the Kurdish Stemming-step module, then dividing them by the whole number of words in the collection and then finding the percentage. The Kurdish Stemming-Steps formulation produces better indicators of correctly stemmed words and combining variant words of the same group to correct stem; it reaches nearly 78% of correct stems.

4.2.3 Experimental Results for Sample III In this sample, three different classifiers are used to study the effect of each of the preprocessing tasks. The three classifiers are conducted using six different representations of the same dataset. After conducting a comprehensive study and comparison on this dataset, some insightful thoughts and conclusions can be discussed. The objective of this set of experiments was to compare the

65

Chapter 4

Test Results Evaluation

performance of the considered classifiers for each of the six different TESTs of the dataset. Drawn in the following:

4.2.3.1 The Results on SVM SVM builds a hyper plane that perfectly separates a set of positive patterns from a set of negative patterns with a maximum margin in case of linear. The aim of this set of TESTs was to compare the performance of this classifier for each of the six-different preprocessing on the dataset. Table 4.3 and 4.4, show the accuracy for six different representations, the number of correctly classified instances (CCI), the number of incorrectly classified instances (ICI), and time spent to build mode. Table 4.3: Accuracies of SVM on the six versions of the dataset using Fold=10. Trails

Accuracy

CCI

ICI

Time (Sec)

Test1

87.17 %

3493

514

4:27

Test2

91.39 %

3662

345

3:33

Test3

87.62 %

3511

496

4:20

Test4

91.44 %

3664

343

3:33

Test5 Test6

91.48 % 87.72 %

3666 3515

341 492

4:23 5:3

Table 4.4: Accuracies of SVM on the Six Versions of the Dataset Using Percentage 70% for Training and 30% for Testing. Trails

Accuracy

CCI

ICI

Time (Sec)

Test1

86.35 %

1038

164

1:37

Test2

91.09 %

1096

107

59 Sec

Test3

87.85 %

1056

146

1:30

Test4

92.01 %

1106

96

58 Sec

Test5 Test6

92.26 % 87.60 %

1109 1053

93 149

1:2 1:49

66

Chapter 4

Test Results Evaluation

Per the proposed technique, using normalization, stop-word removal, and Kurdish Stemming-step module produced a positive impact on classification accuracy in general. As shown in Table 4.3, applying normalization and Kurdish Stemming-step module provided a dominant impact and generated a significant improvement in classification accuracy with SVM classifier. This can be seen from the experiences of Test2, Test4, and Test5 respectively. On the other hand, stop word removal provided a slight improvement in classification accuracy with SVM classifier. However, normalization helped in gathering the words that contained a similar importance, a smaller number of features with further achieved discrimination. For any classification system, the model building time is a critical factor. As expected, the learning (model building) times for six Tests were generally low compared with NB and DT (C4.5). The other perception identified with the learning times, while utilizing Kurdish Stemming-step module reduced the building times for classifier compared with the original dataset. Tables 4.5 and 4.6 demonstrated the weighted averages of the Precision, Recall and F-measure values. The obvious thing in the two tables is how close these values are among the considered tests. Table 4.5: Experimental Results of Average Recall, Precision and F1measure for SVM Classifier on Six Tests Using 10-Fold Cross Validation. Trails Test1 Test2 Test3 Test4 Test5 Test6

Precision

Recall

F1-Measure

0.87 0.91 0.88 0.92 0.92 0.88

0.87 0.91 0.87 0.91 0.91 0.87

0.87 0.91 0.87 0.91 0.91 0.87

67

Chapter 4

Test Results Evaluation

Table 4.6: Experimental Results of Average Recall, Precision and F1measure for SVM Classifier on five Testes Using Percentage 70 %. Trails

Precision

Recall

F1-Measure

Test1 Test2 Test3 Test4 Test5

0.88 0.91 0.89 0.92

0.86 0.91 0.87 0.92

0.86 0.91 0.88 0.92

0.92

0.92

0.92

Test6

0.89

0.87

0.88

In addition, the average precision and recall of the eight categories can be seen for Test2, Test4 and Test5 worked well compared to the original dataset which stemming processing was used which reduced size of feature that effected on the final performance of Kurdish text classification. On the other hand, we can notice that the precision, recall and F-measure for normalization and stop word removal did not affect, or slightly affected in Test3. Per Test5 results, SVM with TF×IDF term weighting yielded better than DT (C4.5) and NB with TF×IDF term weighting. The other related observation that can be seen while using two popular processes, which were 10-Fold Cross Validation and percentage split that they gave different performance accuracy value for this classifier compared with the original dataset. In conclusion, Kurdish Stemming-step module in general can considerably increase accuracy and reduce the learning times for SVM classifier. Figure 4.2 shows the Classification accuracy of SVM for six Tests for 10-Fold Cross Validation and percentage split methods.

68

Chapter 4

Test Results Evaluation

Fold K=10 (b) Accuracy

Accuracy

Percentage Split 70% (a)

94.00% 92.00% 90.00% 88.00% 86.00% 84.00% 82.00% Test6

Test5

Test4

Test3

Test2

92.00%

Test1

90.00%

Test6

Test5

88.00% Test4

86.00%

Test3

84.00%

Test2

Test1

Figure 4.2: The Experimental Results of Kurdish Text Classification Using SVM: (a) Using Hold out Strategy 70% for Training and 30% for Testing; (b) Using 10 – Fold Cross-Validation.

4.2.3.2 The Results on DT Tables 4.7 and 4.8 depict the comparison results for the percentage split and the cross-validation methods obtained for the six proposed experiments using C4.5 decision tree. From these results, we can conclude that Test4 had the best performance in general. On the other hand, the Test5 included feature-weighting TF×IDF produced achieve accuracy that was almost the same as the Test4 using cross validation. Table 4.7: Accuracies of DT (C4.5) on the Six Versions of the Dataset Using 10Fold Cross Validation. Trails

Accuracy

CCI

ICI

Time (Sec)

Test1

64.88 %

2600

1407

231:33

Test2

79.81 %

3198

809

155:31

Test3 Test4 Test5 Test6

64.26 %

2575

1432

228:49

80.58 % 80.53 % 64.73 %

3229 3227 2594

778 780 1413

150:29 164:29 217:47

69

Chapter 4

Test Results Evaluation

Table (4.8): Accuracies of DT (C4.5) on the Six Versions of the Dataset Using Percentage 70% for Training and 30% for Testing. Trails

Accuracy

CCI

ICI

Time (Sec)

Test1

62.22 %

748

164

32:29

Test2 Test3 Test4

80.61 %

969

233

12:42

65.80 %

791

411

22:31

81.11 %

975

227

11:26

Test5 Test6

81.11% 60.98 %

975 733

227 469

10:40 23:22

The results in Test1 (the original dataset is used) were very small, whereas the performance for same dataset and the same classifier used in Test2 and Test4, increased significantly compared to the original dataset. The reason for this is that the two tests in the preprocessing step contained Kurdish Stemmingstep module technique in the preprocessing steps; whereas, the performance for Test3 increased marginally which contained normalization and stop word removal in the preprocessing stage. Another measure which obtained from the experiments was the amount of time taken for building the models. As shown in the two Tables 4.7 and 4.8, the DT (C4.5) required a huge amount of time to build the needed model for six different datasets in general. While the time for building the models in Tests contained the preprocessing stage which decreased very significantly compared to the original dataset. As illustrated in Tables 4.9 and 4.10, the weighted averages for the precision, recall and F1- measure in Test1, (which is the original dataset involved) are very small for the percentage split and the cross-validation method.

70

Chapter 4

Test Results Evaluation

Table 4.9: Experimental Results of Average Recall, Precision and F1-measure for DT (C4.5) Classifier on Six Tests Using 10-Fold Cross Validation. Trails Test1 Test2 Test3 Test4 Test5 Test6

Precision

Recall

0.65 0.80 0.68 0.81 0.81 0.65

F1-Measure 0.64 0.79 0.64 0.80 0.80 0.64

0.65 0.79 0.64 0.80 0.80 0.64

Table 4.10: Experimental Results of Average Recall, Precision and F1measure for DT (C4.5) Classifier on Six Tests Using Percentage 70 %. Trails Test1 Test2 Test3 Test4 Test5 Test6

Precision

Recall

0.64 0.81 0.70 0.82 0.82 0.62

F1-Measure 0.62 0.80 0.65 0.81 0.81 0.61

0.62 0.80 0.66 0.81 0.81 0.61

Whilst the F1- measure for same dataset and same classifier was used in Test2, Test4 and Test5, it increased significantly compared with the original dataset. The reason for this is that the three tests in preprocessing step contained stemming, thus it can be inferred that the Kurdish Stemming-step module improved the Precision and Recall for the classifier. However, the Precision and Recall for Test6 in Table 10.4 decreased slightly which contained feature weighting TF×IDF produces. On the other hand, the F1- measure of Test5 that included feature weighting TF×IDF produced same levels in comparison with Test4 for the percentage split and the cross-validation method.

71

Chapter 4

Test Results Evaluation

In fact, if one carefully looks at both Figure 4.2 and 4.3, which showed the performance for the SVM experiments and DT (C4.5) experiments, it is obvious that DT (C4.5) was more responsive for certain preprocessing steps. Figure 4.2 showed the classification accuracy of DT (C4.5) for six Tests using 10-Fold Cross Validation and percentage split methods.

Percentage Split 70% (a) Accuracy

Accuracy

Fold k=10 (b)

100.00% 80.00% 60.00% 40.00% 20.00% 0.00%

100.00% 80.00% 60.00% 40.00% 20.00% 0.00%

Test6

Test5

Test4

Test3

Test2

Test1

Test6

Test5

Test4

Test3

Test2

Test1

Figure 4.3: The Experimental Results of Kurdish Text Classification Using DT (C4.5): (a) Using Hold Out Strategy 70% for Training and 30% for Testing; (b) Using 10 – Fold Cross-Validation.

4.2.3.3 The Results on NB Different experiments examined to evaluate the performance of the NB classifier on preprocessing steps of the Kurdish Sorani text classification. As indicated by the data introduced in Tables 4.11 and 4.12, the highest accuracy (87.10 %) achieved when all the pre-processing steps were used with Test 4 using the percentage split method. Also in case of the cross-validation method, the highest accuracy (86.74 % and 86.42 %) achieved with Test2 and Test4 when normalization and Kurdish Stemming-step module technique were used respectively. After performing feature weighting TF × IDF, NB classifier 72

Chapter 4

Test Results Evaluation

obtained the worst accuracy results in Test5 and Test6 compared to the values obtained from other two classifiers, where it was worse than Test1 (i.e. original datasets) which is unexpected. Table 4.11: Accuracies of NB on the Six Versions of the Dataset Using 10-Fold Cross Validation. Trails Test1 Test2 Test3 Test4 Test5 Test6

Accuracy

CCI

ICI

Time (Sec)

76.89 %

3081

926

11:32

86.74 % 79.13 % 86.42 % 82.48 % 70.00 %

3476 3129 3464 3305 2805

531 881 544 702 1202

9:7 14:5 9:36 11:58 9:34

Table 4.12: Accuracies of NB on the Six Versions of the Dataset Using Percentage 70% for Training and 30% for Testing. Trails Test1 Test2 Test3 Test4 Test5 Test6

Accuracy

CCI

ICI

Time (Sec)

76.53 %

920

282

4:55

86.02 %

1034

168

3:4

77.78 % 87.10 % 81.94 % 70.96 %

935 1047 985 853

267 155 217 349

4:37 3:2 2:54 4:50

As expected, the building times for classifier like NB, required a small amount of time to complete the model compered to DT (C4.5) in both methods. Consequently, it can be noticed from Tables 4.13 and 4.14, that the results gave the highest average Precision, Recall and F1- measure when the pre-processing steps were used. Also, it can be noticed that when feature-weighting TF × IDF was used, the F1- measure was decreased for the same dataset and the same

73

Chapter 4

Test Results Evaluation

classifier. Figure 4.4 clearly shows the classification accuracy of the NB classifier for six different Tests.

Table 4.13: Experimental Results of Average Recall, Precision and F1measure for NB Classifier on Six Tests Using 10-Fold Cross Validation. Trails

Precision

Recall

F1-Measure

Test1

0.77

0.76

0.77

Test2 Test3 Test4 Test5 Test6

0.87

0.86

0.86

0.78

0.78

0.78

0.86 0.82 0.71

0.86 0.82 0.70

0.86 0.82 0.70

Table (4.14): Experimental Results of Average Recall, Precision and F1measure for NB Classifier on Six Tests Using Percentage 70 %. Trails

Precision

Recall

F1-Measure

Test1

0.77

0.76

0.76

Test2 Test3 Test4 Test5 Test6

0.86

0.86

0.86

0.79

0.77

0.77

0.87 0.81 0.73

0.87 0.82 0.71

0.87 0.81 0.71

74

Chapter 4

Test Results Evaluation

Test6

50.00% Test5

Test4

Test3

Test2

Accuracy

100.00%

Fold k=10 (b)

Accuracy

Percentage Split 70% (a)

0.00%

100.00% 80.00% 60.00% 40.00% 20.00% 0.00%

Test1

Test6

Test5

Test4

Test3

Test2

Test1

Figure 4.4: The Experimental Results of Kurdish Text Classification Using NB: (a)Using Hold out Strategy 70% for Training and 30% for Testing; (b)Using 10 – Fold Cross-Validation.

4.2.4 Results and Discussion Sample III The dataset was experimented using two methods for measuring accuracies which are percentage split method, where 70 % of the dataset is used as a training set and the remaining 30 % is used as a testing set and k-fold cross validation method. The data was divided into 10 folds, a fold was used as testing and the remaining folds were used as training. All documents for training and testing involved a pre-processing step, which included tasks of function of word removal, word stemming, and feature weighting TF×IDF separated into Tests. The goal of experimental studies was performed to evaluate the performance of text classification classifiers on dataset versions created by using different Tests. As illustrated in Figure 4.5, the best result obtained was through SVM classifier, followed by NB classifier, and then DT (C4.5) classifier for each of the six different Tests.

75

Chapter 4

Test Results Evaluation

Analysis Sample III 100.00%

accuracy

80.00% 60.00% SVM 40.00%

NB

20.00%

DT(C4.5)

0.00% Test1

Test2

Test3

Test4

Test5

Test6

Trails

Figure 4.5: Classification Accuracy of SVM, NB and DT (C4.5) on the Six Versions of the Dataset Using Fold=10.

As expected, the learning times for a classifier like SVM classifier was generally low whereas, DT (C4.5) classifier took considerably longer to build, such longer building times do not necessarily correspond to higher accuracies. Figure 4.6 showed results of building times.

Figure 4.6: Time Taken to Building Classifier of SVM, NB and C4.5 without Preprocessing and with Preprocessing Using Fold=10. 76

Chapter 4

Test Results Evaluation

From the experimental results, as in Figure 4.7, it is obvious that the Kurdish Stemming-step module technique significantly influenced the performance of the DT (C4.5) classifier on six Tests. Thus, the range of accuracy in DT (C4.5) was higher than SVM classifier. A decision tree is a planned like tree structure each internal node denotes a test on document, each branch represents a result of the test, and class label falls in each leaf node. It is a top-down method, whenever the decision tree was small, the best accuracy concludes. In other words, the dimension of the dataset less influences the range of accuracy in SVM classifier than DT (C4.5) and NB classifiers that due to it work better in a

ACCURACY

high dimensional environment.

100 90 80 70 60 50 40 30 20 10 0

91

86

80

87

76 64

With Without With Without With Without Preprocessing Preprocessing Preprocessing Preprocessing Preprocessing Preprocessing DT(C4.5)

NB

SVM

Figure 4.7: The Effect of Preprocessing Steps on the Experimental Results for SVM, NB and C4.5 Using Fold=10

Figure 4.8, shows the performance comparison of feature weighting TF×IDF methods in terms of accuracy on datasets. From Figure 4.8, the accuracy performance value of the two classifiers except SVM on datasets is insignificantly decreased after applying feature weighting TF×IDF method. For example, before the process, accuracy values in Original- Test1 and 77

Chapter 4

Test Results Evaluation

Preprocessing steps- Test4 for NB and DT(C4.5) classifiers were 87.17%, 64.88% and 86.42%, 80.58% respectively; however, they became 70%, 64.73% and 82.48%, 80.53% after performing feature weighting TF×IDF method, respectively. The only classifier with insignificantly increased performance was the SVM classifier. For example, the accuracy value in Original- Test1 and Preprocessing steps- Test4 datasets were increased from 87.17% to 87.72% and

87.17

91.44

70

87.72

76.89

86.42

82.48 64.88

70

64.73

80

80.58

90

80.53

100

91.48

91.44% to 91.48%.

60 50 40 30 20 10 0 DT(C4.5)

NB

SVM

Original-Test1

Original+ TF×IDF- Test6

Preprocessing steps-Test4

Preprocessing steps+TF×IDF-Test5

Figure 4.8: Experimental Accuracy Percentage Results of Classifiers on Datasets Using 10-Fold Cross Validation.

Thus, in the latter, the Kurdish Stemming-step module technique reduced the dimensionality of the dataset drastically by grouping words of the same origin together. For example, the words "‫"فێرگا‬, "‫"فێربوون‬and "‫ "فێرکار‬are grouped under the same root/stem "‫"فێر‬. While this reduction makes it easier to build a classification model (especially for classifiers suffering from the ‘curse of 78

Chapter 4

Test Results Evaluation

dimensionality’); it can be noticed that among the six Tests for the three classifiers, SVM still produces higher accuracies for speed of learning, speed of classification, tolerance to irrelevant features and noisy data. In addition, one exceptional property of SVM is that its capability to learn can be independent of the dimensionality of the feature space. The experimental results in Table 4.15 shows the accuracy results for the percentage split and the cross-validation method obtained over the two selected classifiers for six Tests.

Table 4.15: The Accuracy Experimental Results for SVM, NB and C4.5NB

percentage split 70%

10-Fold CV

Classifiers on Six Tests. Trails

Test1

Test2

Test3

Test4

Test5

Test6

SVM

87.17 %

91.39 %

87.62 %

91.44 %

91.48 %

87.72 %

NB

76.89 %

86.74 %

79.13 %

86.42 %

82.48 %

70.00 %

DT(C4.5)

64.88 %

79.81 %

64.26 %

80.58 %

80.53 %

64.73 %

SVM

86.35 %

91.09 %

87.85 %

92.01 %

92.26 %

87.60 %

NB

76.53 %

86.02 %

77.78 %

87.10 %

81.94 %

70.96 %

DT(C4.5)

62.22 %

80.61 %

65.80 %

81.11 %

81.11%

60.98 %

4.2.5 Analytical Discussions Thus, this approach would be a good choice to get correct stems to increase search precision. It can be inferred that the Kurdish Stemming-step module improved the Recall and Precision over the module with stemming. An ideal stemmer is a stemmer with low under – stemming and over – stemming errors.

79

Chapter 4

Test Results Evaluation

However, the main problem is that both errors conflict with each other; reducing one type of error can lead to an increase of the other. Heavy stemmers reduce the under-stemming errors while increasing the over-stemming errors, but light-stemmers reduce the over-stemming errors while increasing the understemming errors. Although heavy stemmers reduce the size of the corpus significantly. For a heavy morphological stemmer, Kurdish Stemming-step module is considered as a heavy morphological stemmer specifically for pre-processing steps. Thus, based on all the results put forward in above tests, it appeared to be very distinct than other information retrieval developing approaches and yielded such relevant and better results than a traditional system without stemming. Thus, the overall study led to a better, effective, efficient, reliable, relevant and excellent information system, which can be user friendly and applied anywhere on textual datasets for ease of data handling, management and access through retrieval. In the work by Singh and Gupta [84], the authors tried to explain the important aspects of text stemming and provided an extensive and useful understanding of stemming techniques. However, they kept the analysis of the current stemming techniques open to help researchers to think about the new lines for this research field in the future for languages such as English and alike that have a large number of researchers. Thus, the Kurdish language is very new in this field and needs extra attention from researchers to extensively improve the performance of a pre-processing tool, which can be used in the field of natural

language

processing,

information

classification and text clustering.

80

retrieval

applications,

text

Chapter 5

Conclusions and Future Works

Chapter Five Conclusions and Future Works 5.1 Conclusions In this thesis, the proposed preprocessing-step technique for Kurdish Sorani document classification was presented. Experiment results demonstrated that the developed technique is promising in this field. This technique which could be useful for some applications related to the Kurdish Sorani natural language processing field. For instance, text clustering, filtering messages composed in Kurdish Sorani texts (e. g. spam filtering), bolster tools (that post-process and organize the results of the Kurdish Sorani text documents), and search engines of large collections of Kurdish web pages are organized under hierarchal classes; thus, it can be quicker for a Kurdish web search engine to begin exploring the hierarchy of classifications. In the previous chapters, the concepts of preprocessing-step technique were discussed for the Kurdish Sorani texts. The robustness of the proposed scheme was tested on different datasets to determine the impacts of preprocessing steps (stemming, stop-word elimination etc.) on Kurdish Sorani text classification. In this thesis, several procedures and processes were conducted to improve the accuracy performance of the Kurdish Sorani text classification using three wellknown classifiers NB, SVM and C4.5. In this section, it is desired to point out some conclusions: 1) The experiments indicated that the SVM outperformed both NB, and C4.5 classifiers in all tests. 81

Chapter 5

Conclusions and Future Works

2) Applying normalization and Kurdish Stemming-step module on the original datasets affected the performance of the three used classifiers (SVM, NB and C4.5), as a result, these classifiers provided better classification accuracy which were (91.3%), (86.7%) and (79.8%) when using 10-Fold CV method, compared to the original data (87.1%), (76.8%) and (64.8%) respectively. Also, (91%), (86%) and (80%) when using percentage split method, compared to the original data (86.3%), (76.5%) and (62.2%) respectively. 3) The performance of the classifiers SVM, NB and C4.5 was increased marginally when the stop word filtering approach was used in the preprocessing stage. 4) Term weighting, such as TF×IDF method was performed after preprocessing steps to determine the impacts of feature weighting methods on Kurdish text classification. The experimental results indicated that: SVM increased the classification accuracy value by 0.25%. However, the classification accuracy was decreased by 5.1% using NB classifier. Besides, the classification accuracy was not affected when DT (C4.5) was used. 5) Kurdish Stemming-step module is considered as a heavy morphological stemmer, specifically, for pre-processing steps. Thus, based on the results obtained from tests in Chapter 4, this technique appeared to be very distinctive and also yielded relevant and better results than traditional systems without stemming. 6) The main advantage of the Kurdish preprocessing-step procedure is to speed up the classification time especially for DT.

82

Chapter 5

Conclusions and Future Works

5.2 Suggestions for Future Works During the discussion of the test results, the following suggestions could be taken into consideration for future works:-

1) The future work may include other well-known classifiers techniques such as using deep learning neural networks or using some advanced natured inspired algorithm for optimizing the SVM classifier. 2) Feature selection approaches which can be used for reducing features can be employed as a future work for further assessment to obtain deeper insights on KDC-4007 dataset. 3) Another future work is to construct a new bigger data set by collecting much more documents and investigating the task of text classification by utilizing a library like Hadoop MapReduce. 4) The effect of text preprocessing steps on accuracy for improving the confidence in Association Rule mining. 5) Finally, our plan is to apply natured inspired algorithm for selecting the best features of the dataset before using the text classification techniques.

83

Publication 1. Arazo M. Mustafa and Tarik A. Rashid, “Kurdish Stemmer Pre-processing Steps for Improving Information Retrieval”, published in Journal of Information Science, 1–14,sagepub.co.uk/journalsPermissions.nav, DOI: 10.1177/0165551516683617, jis.sagepub.com, Thomson Reuters, ISI indexed, Journal Citation ReportsAuthors. Impact Factor 0.878, 2016. 2. Tarik A. Rashid, Arazu M. Mustaf, Ari M. Saeed, “A Robust Categorization System for Kurdish Sorani Text Documents, published in Information Technology Journal. The journal is indexed by SCIMAGO and Elsevier (SCOPUS), Impact Factor SJR 0.20, 2017. 3. Tarik A. Rashid, Arazu M. Mustaf, Ari M. Saeed Automatic Kurdish Text Classification Using KDC 4007 Dataset, accepted in Springer book, Series Title: Lecture Notes on Data Engineering and Communications Technologies: Book title: Advances in Internetworking, Data & Web Technologies, Indexing: The books of this series are submitted to ISI Proceedings, EI, Scopus, MetaPress, Springerlink, 2017. 4. Tarik A. Rashid, Arazu M. Mustaf, Ari M. Saeed Automatic Kurdish Text Classification Using KDC 4007 Dataset, in The 5-th International Conference on Emerging Internetworking, Data & Web Technologies June 10-11, 2017, Wuhan, China

84

Appendix Appendix – A1: The basic Naïve Bayes algorithm.

Appendix – A2: The basic C4.5 algorithm

85

Appendix – B: Unique Words Pseudo – Code.

86

‫‪Appendix – C: Stop Word List for Sorani Kurdish Dialect.‬‬ ‫‪Alpha Stop Wordlist‬‬ ‫ئەوە"‪" ,‬ئەوان"‪" ,‬ئەو"‪" ,‬ئەوا"‪",‬ئەم"‪" ,‬ئەنجام"‪",‬ئێوە"‪" ,‬ئێمە"‪",‬ئەگەر"‪",‬ئێستا"‪" ,‬ئەمساڵ"‪" ,‬ئەمە"‪,‬‬ ‫‪", .‬ئەی"‪",‬ئەمان“ ”"ئەمڕۆ‬ ‫بەو"‪" ,‬بەچەند"‪",‬بەهۆ"‪" ,‬بەهیچ"‪" ,‬بە"‪" ,‬چۆن"‪" ,‬بەڵکو"‪",‬بەرەو"‪",‬بەپێی"‪",‬بەهۆ"‪",‬بەم"‪",‬بەبێ"‪" ,‬‬ ‫"بۆئەو"‪",‬بۆچی"‪" ,‬بۆيە"‪",‬بۆ"‪" ,‬بەپێی""بوون"‪",‬بوو"‪",‬بەاڵم"‪",‬بارە"‬ ‫‪",‬بەسەر"‪",‬بەڵێ"‪",‬بەوە"‪",‬بکات"‪", ,‬بۆچی"‪",‬باش"‪",‬بەباش"‪",‬بان"‪",‬بکەن"‪",‬بن"‪" ,‬بێ"‪,‬‬ ‫‪"".‬بووە"‪",‬بەو"‪",‬بەوە" ‪",‬بەوان"‪",‬بێت" ‪",‬بەدەست" "بەردەوام"‪",‬بڕی"‪",‬بدات"‪",‬بڵێ‬ ‫‪".‬پاشان"‪" ,‬پاش"‪" ,‬پاشی"‪" ,‬پێش"‪" ,‬پێنج"‪" ,‬پێشتر"‪" ,‬پێشوو"‪" ,‬پلە"‬ ‫‪" .‬ت ا"‪",‬تاکو"‪",‬تاوەکو"‪",‬تۆ"‪",‬تايبەت"‪",‬تر"‪",‬تری"‪",‬تەنها"‪",‬تەنیا"‪",‬تەواو"‪",‬تۆن"‪",‬تیدا"‪",‬تێدا"‬ ‫‪" .‬جار"‪" ,‬جگە"‪",‬جا"‪",‬جۆر"‪" ,‬جۆری"‬ ‫‪" .‬چی"‪" ,‬چوار"‪" ,‬چونکە"‪",‬چونک"‪" ,‬چۆن"‪" ,‬چەند"‪" ,‬چوارەم"‪",‬چیە"‬ ‫‪".‬حەوت"‪" ,‬حەوتەم"‬ ‫‪".‬خۆ"‪" ,‬خۆی"‪",‬خۆمان"‬ ‫دواجار"‪" ,‬دا"‪" ,‬دوو"‪" ,‬دووەم"‪" ,‬دەيەم"‪" ,‬دووشەمم"‪",‬دوای"‪" ,‬دەڵێت"‪",‬ديکە"‪" ,‬دەبێت"‪",‬دەکات"‪,‬‬ ‫‪" .‬ڕۆژ"‪",‬ڕۆژان "‬ ‫‪" .‬زۆر"‪",‬زۆرە"‪" ,‬زوو"‪" ,‬زووی"‪" ,‬زۆردەبن"‪" ,‬زياتر"‪" ,‬زۆربە"‪" . ,‬ژێر"‬ ‫‪" .‬سەد"‪" ,‬ساڵ"‪",‬ساڵە"‪" ,‬سەدا"‪" ,‬سێ"‪" ,‬سەر"‪" ,‬سەرەتا"‪" ,‬ساڵی"‪" ,‬سبەی"‪",‬سەرجەم"‬ ‫‪" .‬شوبات"‪" ,‬شەش"‪" ,‬شوێن"‪" ,‬شێوە"‪" ,‬شەشەم"‬ ‫‪" .‬قۆناغ"‬ ‫کەچی"‪" ,‬کەئەو"‪" ,‬کە"‪" ,‬کاتی"‪" ,‬کات"‪" ,‬کەبۆ"‪" ,‬کەوات"‪",‬کەم"‪",‬کەمی"‪",‬کەمە"‪",‬کەس"‪" ,‬‬ ‫‪"",‬کانون‬ ‫‪" .‬گەر"‪" ,‬گەوره"‬ ‫لەسەر"‪" ,‬لەگەڵ"‪" ,‬لە"‪" ,‬لەم"‪" ,‬لەو"‪" ,‬لەوە"‪" ,‬لەبەر"‪" ,‬لەبن"‪",‬لەوان"‪",‬لەهەر"‪",‬لەوکات"‪",‬لەدوا"‪" ,‬‬ ‫‪"".‬لەنێو"‪" ,‬لەژێر"‪" ,‬الی"‪" ,‬اليەن"‪" ,‬لەناو"‪" ,‬لێ"‪",‬لەچی‬ ‫‪", .‬مەبەست"‪" ,‬من"‪" ,‬ملیۆن"‪" ,‬مان"‪",‬مانگ"‬ ‫نەک"‪" ,‬نیە"‪" ,‬نۆ"‪" ,‬نابن"‪" ,‬نەكەن"‪" ,‬نەوەک"‪",‬نێو"‪" ,‬نێوان"‪",‬نا"‪" ,‬نەبێت"‪",‬نیشان"‪" ,‬نییە"‪" ,‬‬ ‫‪"" .‬ناو"‪" ,‬ناوبراو"‪" ,‬ناوی"‪" ,‬نوێ"‪",‬نەخێر"‪" ,‬نە"‪",‬نەچێ"‪" ,‬نەبوو‬ ‫‪".‬و"‪" ,‬وەی"‪" ,‬وە"‪" ,‬وەرز"‪",‬وا"‪" ,‬وەک"‪" ,‬وەکوو"‪",‬وەکو"‪" ,‬وەها"‪" ,‬وايە"‬ ‫هەينی"‪" ,‬هەبوو"‪" ,‬هەيە"‪" ,‬هەموو"‪" ,‬هەر"‪" ,‬هات"‪" ,‬هۆکار"‪" ,‬هۆ"‪" ,‬هەرچەند"‪" ,‬هەزار"‪" ,‬‬ ‫"هەن"‪" ,‬هەند"‪" ,‬هۆی"‪" ,‬هیچ"‪" ,‬هێند"‪" ,‬هەفت"‪" ,‬هەريەک" ‪",‬هەبێت" ‪",‬هەروەها"‪" ,‬هەمان"‪,‬‬ ‫"هێ ڵی"‪" ,‬هەردوو"‪" ,‬هەردووال"‪" ,‬هەزارها"‪" ,‬هاوکات"‪" ,‬هەشت"‪" ,‬هەشتەم"‪" ,‬هەتا"‪" ,‬هتد"‪,‬‬ ‫‪"" .‬هەردەم"‪",‬هەبێ"‪" ,‬هەمووکات" ‪".‬هیچی"‪" ,‬هاتو‬ ‫‪" .‬يان"‪" ,‬يەکەم"‪" ,‬يە"‪" ,‬يی"‪" ,‬يەک"‪" ,‬يەکەمجار"‪" ,‬يا"‬

‫ئـ‬ ‫بـ‬

‫پـ‬ ‫تـ‬ ‫ج‬ ‫چ‬ ‫ح‬ ‫خ‬ ‫د‬ ‫ڕ‬ ‫ز‬ ‫س‬ ‫ش‬ ‫ق‬ ‫ک‬ ‫گ‬ ‫ل‬ ‫م‬ ‫ن‬ ‫و‬ ‫هـ‬

‫يـ‬

‫‪The table contain nearly 240 Stop words which are arranged alphabetically‬‬ ‫‪for documentation.‬‬

‫‪87‬‬

Appendix – D: KDC-4007 Dataset Performance Evaluation Pseudo-Code.

88

References [1]

Y. Yang, S. Slattery, and R. Ghani, "A study of approaches to hypertext

categorization," Journal of Intelligent Information Systems, 2002; vol. 18, no. 2, pp: 219-241.

[2]

J. Hidalgo, M. Rodríguez, and J. Cortizo. "The Role of Word Sense Disambiguation

inAutomated Text Categorization". Proceedings in 10th International Conference on "Applications of Natural Language to Information Systems", NLDB 2005, Volume 3513, and pp: 298-309.

[3]

Cormack G V. "Email Spam Filtering : A Systematic Review". Foundations and

TrendsR in Information Retrieval, 2008; 1(4):335–455.

[4]

Dumais S., Sahami M., Heckerman D and Horvitz E. “A Bayesian Approach to

Filtering Junk E-Mail". In AAAI Workshop on Learning for Text Categorization 1996 ;( Cohen).

[5]

Mohammed FS, Zakaria L, Omar N, Albared MY. Automatic Kurdish Sorani text

categorization using N-gram based model. InComputer & Information Science (ICCIS), 2012 International Conference on 2012 Jun 12 (Vol. 1, pp. 392-395). IEEE

[6]

Al-Harbi S. ,Almuhareb A., Al-Thubaity A., Khorsheed S. and Al-Rajeh A. Automatic

Arabic Text Classification. In: 9th International Journal of Statistical Analysis of Textual Data, (2008) 77-83.

[7]

Duwairi R, Al‐Refai MN, Khasawneh N. Feature reduction techniques for Arabic text

categorization. Journal of the American society for information science and technology. 2009 Nov 1; 60(11), pp: 2347-2352.

[8]

Al-Kabi M, Al-Shawakfa E, Alsmadi I and et al.: The Effect of Stemming on Arabic

Text Classification: An Empirical Study. International Journal of Information Retrieval Research, 1(3), pp: 54-70, July-September 2011.

89

References [9]

Al-Shargabi B, Olayah F, Romimah WA. An Experimental Study for the Effect of

Stop Words Elimination for Arabic Text Classification Algorithms. International Journal of Information Technology and Web Engineering (IJITWE). 2011 Apr 1; 6(2):68-75.

[10]

A. H. Wahbeh and M. Al-kabi, “Comparative Assessment of the Performance of

Three WEKA Text Classifiers Applied to Arabic Text,” ABHATH AL-YARMOUK: "Basic Sci. & Eng."; 2012, vol. 21(1) pp: 15–28.

[11]

Zaki T, Es-saady Y, Mammass D, Ennaji A, Nicolas S. : A Hybrid Method N-Grams-

TFIDF with radial basis for indexing and classification of Arabic documents. International Journal of Software Engineering and Its Applications. 2014; 8(2), pp: 127-144.

[12]

Ababneh J, Almomani O, Hadi W, El-Omari NK, Al-Ibrahim A. Vector space models to

classify Arabic text. International Journal of Computer Trends and Technology (IJCTT). 2014 Jan; 7(4):219-23. [13]

http://diab.edublogs.org/dataset-for-arabic-document-classification/

[14]

Hmeidi I, Al-Ayyoub M, Abdulla NA, Almodawar AA, Abooraig R, Mahyoub NA.

Automatic Arabic text categorization: A comprehensive comparative study. Journal of Information Science; 2015; 41(1)pp:114–124

[15]

Mohammad AH, Alwada'n T, Al-Momani O. Arabic Text Categorization Using

Support vector machine, Naïve Bayes and Neural Network. GSTF Journal on Computing (JoC). 2016 Aug 1; 5(1): pp. 108-115.

[16]

S. Dumais, J. Platt, and D. Heckerman, “Inductive Learning Algorithms and

Representations for Text Categorization.” In Proceedings of ACM-CIKM98, Nov. 1998, pp. 148-155.

[17]

Lan M, Tan CL, Low HB, Sung SY. A comprehensive comparative study on term

weighting schemes for text categorization with support vector machines. InSpecial interest tracks and posters of the 14th international conference on World Wide Web 2005 May 10, pp: 1032-1033. ACM. 90

References [18]

Toman M, Tesar R, Jezek K. Influence of word normalization on text classification.

Proceedings of InSciT. 2006 Oct 25; 4 pp: 354-358.

[19]

Pomikálek J, Rehurek R. The Influence of preprocessing parameters on text

categorization. International Journal of Applied Science, Engineering and Technology. 2007 Sep 29; 1, pp: 430-434.

[20]

Zhang W, Yoshida T and Tang X. A comparative study of TF* IDF, LSI and multi-

words for text classification. Expert Systems with Applications. 2011 Mar 31; 38(3), pp: 27582765.

[21]

Mohsen AM, Hassan HA, Idrees AM. Documents Emotions Classification Model

Based on TF-IDF Weighting Measure. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering. 2016 Aug 1; 10(1):252-8.

[22]

Porter MF. An Algorithm for Suffix Stripping. Program.1980; 14:pp.130-137.

[23]

Porter

MF.

Snowball:

a

language

for

stemming

algorithms.

2001,

http://snowball.tartarus.org/texts/introduction.html.

[24]

Khoja S and Garside, R. Stemming Arabic text, 1999. Computing Department,

Lancaster

University,

Lancaster,

UK,http://www.comp.lancs.ac.uk/

computing/

users/khoja/stemmer.ps.

[25]

Tashakori M, Meybodi M and Oroumchian F. Bon: first Persian stemmer. In: Lecture

Notes on Information and Communication Technology (LNCS), 2002; pp. 487-494.

[26]

Naouar F, Hlaoua L, Omri MN. Possibilistic Model for Relevance Feedback in

Collaborative Information Retrieval. IJWA. 2012; 4(2):pp.78-86.

91

References [27]

Boukhari K, Omri MN. Robust Algorithm for stemming text Document. International

Journal of Computer Information Systems and Industrial Management Applications. 2016; 8:pp. 235-246. [28]

A. Hotho, A. Nurnberger, and G. PaaB, "A brief survey of text mining," in LDV

Forum-GLDV Journal for Computational Linguistics and Language Technology, vol. 20, 2005; pp:19-62.

[29]

Chen KC. "Text Mining e-Complaints Data from e-Auction Store". Journal of

Business & Economics Research, 2009; 7(5):15–24.

[30]

Mahgoub H, Ismail N, Torkey F. A Text Mining Technique Using Association Rules

Extraction. INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE, 2007; 4(1), pp: 21–8.

[31]

Montes-y-Gómez M., Gelbukh A. and López A. “Text mining at detail level using

conceptual

graphs,”

Retrieved

December

20,

2002,

from

http://citeseer.nj.nec.com/531779.html, 2002.

[32]

Sebastiani F." Machine learning in automated text categorization". ACM computing

surveys (CSUR). 2002 Mar 1; 34(1):1-47.

[33]

A. H. Wahbeh and M. Al-kabi, “Comparative Assessment of the Performance of

Three WEKA Text Classifiers Applied to Arabic Text,” ABHATH AL-YARMOUK: "Basic Sci. & Eng.", 2012, vol. 21, no. 1; pp: 15–28.

[34]

Tarik A. Rashid and Hawraz A. Ahmad, “Enhancement of Lecturer Performance

through Particle Swarm Optimization Combined Neural Network “ in Computer Applications in Engineering Education, indexed by Thomson Reuters, impact factor 0.30. Accepted for publication. Wiley publication, 2016.

[35]

K. Nalini and Dr. L. Jaba Sheela. “Survey on Text Classification,” International Journal

of Innovative Research in Advanced Engineering (IJIRAE); 2014, vol. 1, no. 6; pp: 412–417. 92

References [36]

D. Lewis. "(Naive) Bayesian Text Classification for Spam Filtering". ASA Chicago

Chapter Spring Conference, Loyola University, May 7, 2004.

[37]

Androutsopoulos I. , Koutsias J., Chandrinos K. V., and

Spyropoulos C. D., "An

experimental comparison of naive bayesian and keyword-based anti-spam filtering with personal e-mail messages," in Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval,2000; pp. 160-167.

[38]

Tzeras K, Hartmann S. Automatic indexing based on Bayesian inference networks.

InProceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval 1993 Jul 1 (pp. 22-35). ACM.

[39]

Saritha A, Naveenkumar N, Prof A. "Effective Classification of Text". International

Journal of Computer Trends and Technology (IJCTT) 2014; 11(1):1–6.

[40]

Hussien M, Olayah F, Al-dwan M, Shamsan A. Arabic text classification using SMO,

Naive Bayesian, J48 algorithms. International Journal of Research and Reviews in Applied Sciences (IJRRAS). 2011 Nov; 9(2):306-16.

[41]

Forman G. "An extensive empirical study of feature selection metrics for text

classification". Journal of machine learning research. 2003; 3(Mar):1289-305.

[42]

Fodil L, Sayoud H, Ouamour S. Theme classification of Arabic text: A statistical

approach. InTerminology and Knowledge Engineering, 2014 Jun 19 (pp. 10-p).

[43]

Ghanem OA, Ashour WM. Stemming effectiveness in clustering of Arabic documents.

International Journal of Computer Applications. 2012 Jan 1; 49(5).

[44]

Al-Kabi M, Al-Shawakfa E, Alsmadi I. The Effect of Stemming on Arabic Text

Classification: An Empirical Study. Information Retrieval Methods for Multidisciplinary Applications. 2013, pp: 207. 93

References [45]

Joachims T. A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text

Categorization (No. CMU-CS-96-118). Carnegie-mellon univ Pittsburgh pa dept of computer science; 1996 Mar.

[46]

SzymańSki J. Comparative analysis of text representation methods using

classification. Cybernetics and Systems. 2014 Feb 17; 45(2):180-99.

[47]

Salton G, Wong A, Yang CS. A vector space model for automatic indexing.

Communications of the ACM. 1975 Nov 1; 18(11):613-20.

[48]

Lan M, Sung SY, Low HB, Tan CL. A comparative study on term weighting schemes

for text categorization. InNeural Networks, 2005. IJCNN'05. Proceedings. 2005 IEEE International Joint Conference on 2005 Jul (Vol. 1, pp. 546-551). IEEE.

[49]

Zhang W, Yoshida T, Tang X. Text classification based on multi-word with support

vector machine. Knowledge-Based Systems. 2008 Dec 31; 21(8):879-86.

[50]

Caropreso MF, Matwin S, Sebastiani F. A learner-independent evaluation of the

usefulness of statistical phrases for automated text categorization. Text databases and document management: Theory and practice. 2001 Apr 1:78-102.

[51]

Patra A, Singh D. A Survey Report on Text Classification with Different Term

Weighing Methods and Comparison between Classification Algorithms. International Journal of Computer Applications. 2013 Jan 1; 75(7).

[52]

Giannakopoulos G, Mavridi P, Paliouras G, Papadakis G, Tserpes K. Representation

Models for Text Classification: a comparative analysis over three Web document types. InProceedings of the 2nd international conference on web intelligence, mining and semantics 2012 Jun 13 (p. 13). ACM.

94

References [53]

Silva C, Ribeiro B. The importance of stop word removal on recall values in text

categorization. InNeural Networks, 2003. Proceedings of the International Joint Conference on 2003 Jul 20 (Vol. 3, pp. 1661-1666). IEEE.

[54]

Robertson S. Understanding inverse document frequency: on theoretical arguments

for IDF. Journal of documentation. 2004 Oct 1; 60(5):503-20.

[55]

Rish I. An empirical study of the naive Bayes classifier. InIJCAI 2001 workshop on

empirical methods in artificial intelligence 2001 Aug 4 (Vol. 3, No. 22, pp. 41-46). IBM New York.

[56]

Domingos P, Pazzani M. On the optimality of the simple Bayesian classifier under

zero-one loss. Machine learning. 1997 Nov 1; 29(2-3):103-30.

[57]

Sharma R, Gulati N. Improving the Accuracy and Reducing the Redundancy in Data

Mining. International Journal of Engineering Science. 2016 May; 45-75.

[58]

Last, M., Markov, A., and Kandel, A., Multi-lingual Detection of Web Terrorist

Content, In: Chen, H. (Ed.), WISI, Lecture Notes in Computer Science, Springer – Verlag; 2008:16-30.

[59]

Kotsiantis SB, Zaharakis I, Pintelas P. Supervised machine learning: A review of

classification techniques; vol. 31, pp: 249-268, 2007.

[60]

HSSINA B, Merbouha A, Ezzikouri H, Erritali M. A comparative study of decision

tree ID3 and C4. 5. Int. J. Adv. Comput. Sci. Appl. 2014; 4(2); pp: 13-19.

[61]

Cortes C, Vapnik V. Support-vector networks. Machine learning. 1995 Sep 1; 20(3):

pp273-97.

95

References [62]

Trivedi M, S. Sharma, N. Soni and S. Nair. Comparison of Text Classification

Algorithms. International Journal of Engineering Research & Technology (IJERT), 2015; 4(02):334-336.

[63]

C. J. Burges, “A tutorial on support vector machines for pattern recognition,"Data

mining and knowledge discovery, vol. 2, no. 2, pp. 121-167, 1998.

[64]

C. Cortes and V. Vapnik, “Support-vector networks," Machine learning, vol. 20, no. 3,

and pp: 273-297, 1995.

[65]

Witten IH, Frank E. Data Mining: Practical machine learning tools and techniques.

Morgan Kaufmann; 2005 Jul 13.

[66]

Markov Z, Larose DT. Data mining the Web: uncovering patterns in Web content,

structure, and usage. John Wiley & Sons; 2007 Apr 6. [67]

Hasseim AA, Sudirman R, Khalid PI. Handwriting classification based on support

vector machine with cross validation. Engineering. 2013 May 31; 5(05):84.

[68]

Rodriguez JD, Perez A, Lozano JA. Sensitivity analysis of k-fold cross validation in

prediction error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2010 Mar; 32(3):569-75.

[69]

Costa E, Lorena A, Carvalho AC, Freitas A. A review of performance evaluation

measures for hierarchical classifiers. InEvaluation Methods for Machine Learning II: papers from the AAAI-2007 Workshop 2007 (pp. 1-6).

[70]

Verbraken T, Verbeke W, Baesens B. A novel profit maximizing metric for measuring

classification performance of customer churn prediction models. IEEE Transactions on Knowledge and Data Engineering. 2013 May; 25(5):961-73.

[71]

C. J. Van Rijsbergen, Information Retrieval, Butter-Worths, London, UK, 1979. 96

References [72]

Paice CD. An evaluation method for stemming algorithms. In: Proceedings of the 17th

Annual International Conference on Research and Development in Information Retrieval, Dublin: ACM, 1994; pp. 42–50.

[74] LGM

Kraaij W and Pohlmann R. Porter’s stemming algorithm for Dutch. In: Noordman and

de

Vroomen

WAM

(Eds.),

Informatiewetenschap

1994:

Wetenschappelijkebijdragenaan de derde STINFON Conferentie, 1994; pp. 167– 180.

[74]

Karaa WBA and Gribâa N. Information Retrieval with Porter Stemmer: A New

Version for English. In: Nagamalai D, Kuma A, and Annamalai A. (eds.), Heidelberg: Springer, 2013; 225: pp. 243–254.

[75]

Kurdish Academy of Language. Kurdish Language, http://www. kurdishacademy.org

/? q=node/4(accessed 15 February2012).

[76]

Thackston WM. Sorani Kurdish: A Reference Grammar with Selected Readings.

Harvard University, 2006b.

[77]

Walther G. Fitting into Morphological Structure: Accounting for Sorani Kurdish

Endoclitics. In: The Proceedings of the 8th Mediterranean Morphology Meeting, 2012; pp. 299– 321.

[78]

Samvelian P. A Lexical Account of Sorani Kurdish Prepositions. In: Proceedings of

International Conference on Head-Driven Phrase Structure Grammar, Stanford, CA: CSLI, 2007; pp. 235–249.

[79]

Jayanthi R. An Approach for Effective Text Pre-Processing Using Improved Porters

Stemming Algorithm. International Journal of Innovative Science, Engineering & Technology (IJISET). 2015; 2(7): pp. 797–807.

[80]

Dilekh T and Behloul A. Implementation of a New Hybrid Method for Stemming of

Arabic Text. International Journal of Computer Applications, 2012; 46(8): pp. 14-19. 97

References [81]

http://archive.ics.uci.edu/ml/datasets/KDC-4007+dataset+Collection.

[82]

Arazo M. Mustafa and Tarik A. Rashid, “Kurdish Stemmer Pre-processing Steps for

Improving Information Retrieval”, accepted in Journal of Information Science, 1– 14,sagepub.co.uk/journalsPermissions.nav, DOI: 10.1177/0165551510000000, jis.sagepub.com

[83]

Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann and Ian

H. Witten, “The WEKA Data Mining Software: An Update”, SIGKDD Explorations, Vol. 11, Issue 1, pp. 10-18, 2009. URL: http://www.cs.waikato.ac.nz/ml/weka/.

[84]

Singh J, Gupta V. A systematic review of text stemming techniques. Artificial

Intelligence Review, 2016; pp.1-61.

98

‫اخلالصة‬ ‫بسبب الكميات الكبرية من الوثائق والنصوص اليت يتم حتميلها على اإلنرتنت يوميا‪ .‬ولكون كمية الوثائق‬ ‫الكردية اليت ميكن وصول اليها عن طريق شبكة اإلنرتنت تزداد بشكل الكبري مع مرور الزمن‪ ،‬فاذا اخذنا بعني‬ ‫اعتبار ظهور االحداث‪ ،‬وعلى وجه التحديد‪ ،‬الوثائق اليت مت حتديدها مع فئاته‪ ،‬وعلى سبيل املثال‪ ،‬الصحة‪،‬‬ ‫السياسة‪ ،‬والرياضة تظهر لتكون يف فئة غري صحيحة أو قد تكون حمفوظة يف فئة غري حمددة تسمى اآلخرين‪.‬‬ ‫وبالتالي‪ ،‬هناك حاجة إىل تصنيف النص (واليت يتم تناوهلا بصفة عامة يف مهمة التعلم حتت املشرف)‪ .‬وتتجلى‬ ‫فائدة تصنيف النص يف خمتلف اجملاالت‪ .‬يعترب تصنيف النص عملية تلقائية لتعيني جمموعة ثابتة من الوثائق إىل‬ ‫مصنف اعتمادا على حمتويات البيانات اخلاصة به‪.‬‬ ‫بالرغم من وجود عدد كبري من الدراسات اليت أجريت على تصنيف النص يف لغات أخرى مثل اللغة‬ ‫اإلجنليزية‪ ،‬الصينية‪ ،‬اإلسبانية‪ ،‬والعربية‪ ،‬ومع ذلك‪ ،‬فان كمية الدراسات اليت أجريت يف الكردية مقتصرة للغاية‬ ‫نظرا لعدم توفر البيانات‪.‬‬ ‫مت يف هذه األطروحة‪ ،‬إقرتح اسلوب جديد هو(‪ .)pre-processing step‬وهي نهج فعال لزيادة يف دقة‬ ‫النصوص الكردية السورانية يف مهام تصنيف النص‪ .‬باإلضافة إىل ذلك‪ ،‬فقد مت إنشاء جمموعة بيانات جديدة باسـم‬ ‫( ‪.) KDC-4007‬‬ ‫( ‪ ) KDC-4007‬هي جممـــوعة بيــانات موثقة جيدا وترتيبات وثائقهــا متنـاسبة مع أدوات تعدين‬ ‫النص املعروفة‪.‬‬ ‫تستخدم ثالثة املصنفات على نطاق واسع وهي فضاء املتجهات(‪Support Vector Machine )SVM‬‬ ‫‪ ،‬شجرة القرارات املعروفة بإسم (‪ )C4.5‬و نظرية بيز املبسطة (‪ )Naïve Bayes‬يف جمال التصنيف النص‬ ‫وميزة الرتجيح ‪ TF ×IDF‬مت تقييمها على ( ‪.) KDC-4007‬‬

‫يف هذه األطروحة‪ ،‬وقد مت تنفيذ ستة جتارب لتحديد اآلثار املرتتبة على طريقة ( ‪pre-processing‬‬ ‫‪)step‬على كل املصنف‪ .‬النتائج التجريبية تشري اىل أن أفضل قيمة دقة مت اصحصول عليها كانت من خالل املصنف‬ ‫‪ ،SVM‬تتبعها نظرية بيز املبسطة )‪ )NB‬وشجرة القرار (‪ )C4.5‬يف مجيع التقييمات‪.‬‬ ‫جمموعة البيانات متوفرة (‪ )KDC-4007‬للجميع والنتائج التجريبية من هذه الدراسة ميكن أن تستخدم‬ ‫يف التجارب املقارنة من قبل باحثني آخرين‪ .‬وقد مت يف هذا العمل استخدام لغة جافا (‪ )NetBeans‬وبعض‬ ‫اجراءات أداة تعدين البيانات (‪.)WEKA‬‬

‫تأثري خطوات املعاجلة املسبقة على تصنيف النص الكردي السوراني‬

‫رسالة‬ ‫مقدمة إىل جملس كلية العلوم‬ ‫يف جامعة السليمانية كجزء من متطلبات‬ ‫نيــل شهـــادة مــــاجستري‬ ‫يف علوم اصحاسبات‬ ‫من قبل‬ ‫ئارزو حممد مصطفى‬ ‫بكلوريوس اصحاسبات ( ‪ ،) 7002‬جامعة كركوك‬ ‫بإشراف‬ ‫د‪ .‬طارق أمحد رشيد‬ ‫أســتاذ‬

‫شعبان ‪3418‬‬

‫حزيران ‪7032‬‬

‫ثـوختــــة‬ ‫بةهؤي زيادبووني ئةو دةق و دؤكومينتانةي كة رِؤذانة دةخرِينة سةر ئةنرتنيت و برِي ئةو دؤكومينتة‬ ‫كوردييانةي كة لة مالَثةرِةكان و ثةجية كوردييةكاندا هةية و رِؤذانة لة زيادبوونداية ‪.‬بةرِةضاوكردني‬ ‫دةركةوتةي هةوالَةكان‪ ،‬بةتايبةتي ئةو دؤكومينتانةي كة دياري كراون بؤ ثؤلَيَنةكةي وةكو ‪ ،‬تةندروسيت‪،‬‬ ‫سياسي و وةرزش وا دةردةكةون لة ثؤلَيَنكراوي هةلَة يان ئةوةتة ئةرشيف كراون لةشويَنيَكي دياري نةكاراو كة‬ ‫ثيَيدةوتريَت ئةوانيرت(‪ .)Others‬لةبةرئةوة ثؤليَنكردني دةق (‪ )Text Classification‬ثيَويستة (كة‬ ‫فةرماني ضارةسةر كردني دةقةكانة بةشيَوةيةكي طشيت وةك فريبوون لةذيَر ضاوديَري سةرثةرشتيار)‪.‬‬ ‫ثؤليَنكردني دةق لة بواري جياجيادا سوودي ليَوةرطرياوة‪ .‬ثؤليَنكردني دةق كرداري دياريكردني بةخؤرِةسي‬ ‫دؤكومينتةكانة بؤ ثؤلَيَنةكةي بة ثشت بةسنت بة ناوةرِؤكةكةي‪.‬‬ ‫هةرضةندة ليَكؤلَينةوةي زؤر كراوة لةسةر ثؤليَنكردني دةق لة زمانةكاني تردا وةك ئينطليزي ‪,‬سيين‬ ‫‪,‬ئيسثاني‪ ,‬عةرةبي لةطةلَ ئةوةشدا برِي ئةو ليَكؤلَينةوانةي كة كراون بؤ زماني كوردي زؤر كةمة ئةوةش‬ ‫دةطة ِريَتةوة بوَ نةبوون و دةست نةكةوتين داتاي ثيَويست‪.‬‬ ‫لةم تويَذينةوةيةدا (‪ )pre-processing - step‬شيَوازيَكي تازةي ثيَشنياركراوة‪ .‬كة ئةمةش شيَوازيَكي‬ ‫ضاالكة بؤ زيادكردني ووردي لة دةقة كوردييةكان بة زاراوي سؤراني لة ثؤليَنكرني دةقةكاندا‪ .‬وةهةروةها‬ ‫داتايةكي تازة بة ناوي (‪ )KDC-4007‬دامةزريَندراوة‪ )KDC-4007( .‬ئةم داتاية بةباشي دؤكوميَنتكراوة‬ ‫و وة طوجنيَندراوة لةطة َل ئامرازي ناسراو بة (‪.)Text Mining‬‬ ‫سيَ ث َولَيَنكةري بكارهاتوو بةشيَوةي بةرفراوان وةك ‪ )C4.5(, )SVM(:‬لةطةل (‪ )NB‬لة بواري‬ ‫ثؤليَنكردني دةق (‪ )Text Classification‬لةطةلَ )‪ (TF × IDF‬هةلَسةنطيَنران لةسةر (‪.)KDC-4007‬‬

‫لةم تويَذينةوةيةدا‪ ,‬شةش تاقيكردنةوة ئةجنامدرا بؤ تاقيكردنةوةي كاريطةري ( ‪pre-processing -‬‬ ‫‪ )step‬ثيَشنيار كراو لةاليةن ثؤلَينكةرانةوة‪ .‬وة ئةجنامةكان ئةوة دةسةمليَنن كة باشرتين ووردي بةدي هاتوو‬ ‫لةاليةن ثؤلَنكةري (‪)SVM‬ـة ‪,‬ثاشان (‪ )NB‬دواتر (‪)C4.5‬ـة لة هةر شةش تاقيكردنةوة جياوازةكةدا‪.‬‬ ‫وة داتاي(‪ )KDC-4007‬بةردةستة بؤ هةمووان وة ئةتوانريَ ئةجنامةكاني ئةم تويَذينةوةية‬ ‫بةكاربهيَنريَت لة ئةزمووني هةلَسةنطاندن لة اليةن ليَكؤلَةرةواني ترةوة‪ .‬تاقيكردنةوةكان ئةجنامدران بة زماني‬ ‫‪ Java‬بة بةكارهيَناني ثؤلَيَنكارةكاني ‪ WEKA‬كة ئامرازيَكي دةستكاريكردني زانياريية‪.‬‬

‫كاريطةري رِيَطاكاني ضاككردني لةثيَش لةسةر ثؤلَيَنكردني دةقي‬ ‫كوردي سؤراني‬

‫نامةيةكة‬ ‫ثيَشكةشكراوة بة ئةجنومةني كؤليَجي زانست‬ ‫لة زانكؤي سليَماني وةك بةشيَك لة ثيَداويستيةكاني‬ ‫بةدةستهيَناني برِوانامةي ماستةر‬ ‫لة زانسيت كوَمثيوتةر‬ ‫لةاليةن‬ ‫ئارةزوو حمةمةد مستةفا‬ ‫بةكالوريؤس لة كوَمثيوتةر (‪ ,)7002‬زانكؤي كةركووك‬

‫بةسةرثةرشيت‬ ‫د‪ .‬تاريق ئةمحةد رةشيد‬ ‫ثرؤفيسؤر‬

‫طوالن ‪7272‬‬

‫مايو ‪7072‬‬

The Impacts of Pre-processing Steps on Kurdish Sorani Text ...

The Impacts of Pre-processing Steps on Kurdish Sorani Text Classification.pdf. The Impacts of Pre-processing Steps on Kurdish Sorani Text Classification.pdf.

3MB Sizes 6 Downloads 594 Views

Recommend Documents

Impacts of drought on growth.pdf
Page 1 of 12. Author's personal copy. Agricultural and Forest Meteorology 151 (2011) 1800–1811. Contents lists available at ScienceDirect. Agricultural and ...

Kurdish Teacher's Perspectives on Play-based Learning in the Early ...
... degree within any other framework. Page 3 of 109. Kurdish Teacher's Perspectives on Play-based Learning in the Early Years - Nazeera Salih Mohammed.pdf.

The Impacts of Joint Forest Management on Forest Condition ...
Sep 1, 2010 - Figure 5.4 Asset nets, types of assets compared across wealth percentile groups ..... Has JFM created sustainable forest governance institutions at the village level ... while Chapter 7 focuses on the impact on household livelihoods and

The Regulation of Civilian Drones' Impacts on Public Safety.pdf
The Regulation of Civilian Drones' Impacts on Public Safety.pdf. The Regulation of Civilian Drones' Impacts on Public Safety.pdf. Open. Extract. Open with.

the impacts of adolescent noncompliance on career ...
results of three direct empirical tests of how the effects of noncompliance depend on ..... carrying business cards can be also seen as deviating from widely accepted norms. .... Tuma and Titma (2003) argued that teachers are more likely to reward ru

Kurdish Teacher's Perspectives on Play-based Learning in the Early ...
Master of Arts in Professional Development ... Fulfilment of the requirements for the degree of. Master of Arts in Professional Development ... Teacher's Perspectives on Play-based Learning in the Early Years - Nazeera Salih Mohammed.pdf.

The impacts of performance pay on teacher ...
Relative to these papers, our paper aims to contribute by providing a direct ..... does not rely on within-teacher variation in treatment and makes no attempt to ..... significantly less likely to assign “Worksheets” and are more likely to assign

The Impacts of Joint Forest Management on Forest Condition ...
Sep 1, 2010 - completed this thesis without the support of a number of people. .... being established in 3.6 million ha of forest land and in 1,800 villages ...... In Maseyu village, 57% of the respondents had business income (Table A5.19,.

the impacts of tourist's expenditures on malaysian ...
Department of Economics, Faculty of Economics and Business,. Universiti Malaysia Sarawak. Email: [email protected]. ABSTRACT. This study intends to measure the impacts of tourist's expenditures on Malaysian economy by using Input-Output (I-O),

Feature Preprocessing on Web Page Language ...
sequently, the problems of digital divide across the internet will get more serious. Web page ..... language in a complex multilingual business environment.

Impacts of Duty-Cycle and Radio Irregularity on HCRL ... - CiteSeerX
1School of Software, Dalian University of Technology, China. 2Department of Multimedia ... Epoch vs. Error as K changes from 1 to 7. The anchor node density.

Impacts of regional economic integration on industrial ...
standing with these short-term negative impacts, large-scale inter-bloc FDI inflows from Japan ... industries to the headquarter services in technology intensive, while the .... market entry mode under FTA can be determined through the analysis .....

Impacts of Duty-Cycle on TPGF Geographical Multipath Routing in ...
1. Impacts of Duty-Cycle on TPGF Geographical. Multipath Routing in Wireless Sensor Networks. Lei Shu. 1. , Zhuxiu Yuan. 2. , Takahiro Hara. 1. , Lei Wang. 2.

impacts of policy changes on turkish agriculture: an ...
potential effects of the application of a new WTO agreement in 2015 on. Turkish agriculture using an ...... the WTO-Agreement on Agriculture: all Harmonized System (HS) chapters from 1 to 24, excluding fish but .... Table 7 shows the agricultural imp

impacts of property tax limitations on education spending
and Saas, 2006; Temple, 1996; Fisher and Gade,. 1991). One type of property tax ..... The tax rates are county- wide real property tax rates that apply to all non-.

Impacts of Duty-Cycle and Radio Irregularity on HCRL ... - CiteSeerX
... thinking: more awake nodes can help to increase the HCRL localization accuracy. .... Education, Culture, Sports, Science and Technology, Japan and partially ...

Evaluation of COP18 outcome and impacts on Asia -
They are also incapable of coping with the climate disasters by their own ... Japan and Russia stuck to discussion on market trading structures using or ... change in the world According to world's risk report in 2012, 7 out of 10 natural disasters.

Projections of climate change impacts on potential C4 ...
ity and human management, determining mean crop productivity. ∗ Corresponding author ... els projections to drive process-based or statistical crop models. Climate change is ... cereals such as millet and sorghum, to project the large-scale impacts

Impacts of Duty-Cycle and Radio Irregularity on HCRL ... - CiteSeerX
Abstract—This paper focuses on studying the impacts of two important factors that have been once ignored in localization: 1) duty-cycle of sensor nodes and 2) DOI (degree of irregularity) in radio irregularity. It reveals the fact that a smaller DO

Impacts of Duty-Cycle and Radio Irregularity on ... - Semantic Scholar
1School of Software, Dalian University of Technology, China. 2Department of .... Aid for Scientific Research (S)(21220002) of the Ministry of. Education, Culture ...

Impacts of China's Growth on Environmental Issues
Aug 6, 2008 - manufacture cheap products fill stores in the US/EU. 08.06. ... economic growth in resp. domain & were graded in accordingly by leadership.

Impacts of regional economic integration on industrial ...
destination between countries m and l based on the comparison of expected profits from ..... technology gap is big between the member countries. ... This result is supported by the real data showing the decreased FDI inflows from ... Source: Ministry

impacts of property tax limitations on education spending
California restricts the growth of an individual's assessed. IMPACTS OF PROPERTY TAX LIMITATIONS .... changes in school spending, which are constrained.