Detecting Text Similarity Based on Discrete Wavelet Transformation

Abstract

Detecting similarities between texts is an important stage of many different applications such as text classification, plagiarism detection, fake news identification,... In this paper, we propose a new approach to detect similarities between texts based on the Discrete Wavelet Transform (DWT) method. Specifically, the available source documents are converted into a set of real numbers called DNAs (Deoxyribose Nucleic Acid) through DWT. To check the similarity of any text, we also use DWT to generate DNAs for that text and calculate the smallest Euclidean distance from these DNAs to the source DNAs. Finally, by comparing with a threshold, the distance values will indicate whether the evaluation text is similar to a certain source text or not. Experimental results demonstrate that our proposed algorithm is highly effective in detecting text similarity by testing on a standard data set at the Annual International Conference on Plagiarism Detection (Plagiarism Analysis, Authorship Identification, and Near-Duplicate detection – PAN).