|
|
Preprocessing and Transformation of Text Data Collections
Maruna, Viktor ; Burget, Radek (referee) ; Bartík, Vladimír (advisor)
This bachelor thesis deals with the issue of text-mining, mostly focused on preprocessing and transformation. In theoretical part there are contained information about development and principles of text-mining processes, text data collections and use in practice. The next part of this thesis describes in detail single steps of preprocessing and transformation of text data collections. In the final parts there are reviews of application development, testing and personal view on this thesis.
|
|
|
Stemming of Czech Words
Hellebrand, David ; Bartík, Vladimír (referee) ; Chmelař, Petr (advisor)
The goal of this master's thesis is to develop stemming algorithm for czech language based on grammatical rules. You can find a description of stemming process and a comparsion of stemming algorithms in this project. The basics of czech grammar and Snowball language are also described here. The main part of this thesis concerns the implementation of the new czech stemming algorithm.
|
|
|
Methods for Mining Association Rules from Data
Uhlíř, Martin ; Burget, Radek (referee) ; Bartík, Vladimír (advisor)
The aim of this thesis is to implement Multipass-Apriori method for mining association rules from text data. After the introduction to the field of knowledge discovery, the specific aspects of text mining are mentioned. In the mining process, preprocessing is a very important problem, use of stemming and stop words dictionary is necessary in this case. Next part of thesis deals with meaning, usage and generating of association rules. The main part is focused on the description of Multipass-Apriori method, which was implemented. On the ground of executed tests the most optimal way of dividing partitions was set and also the best way of sorting the itemsets. As a part of testing, Multipass-Apriori method was compared with Apriori method.
|
|
| |
|
|
Methods of Text Document Summarization
Pokorný, Lubomír ; Očenášek, Pavel (referee) ; Bartík, Vladimír (advisor)
This thesis deals with one-document summarization of text data. Part of it is devoted to data preparation, mainly to the normalization. Listed are some of the stemming algorithms and it contains also description of lemmatization. The main part is devoted to Luhn"s method for summarization and its extension of use WordNet dictionary. Oswald summarization method is described and applied as well. Designed and implemented application performs automatic generation of abstracts using these methods. A set of experiments where developed, which verified correct functionality of the application and of extension of Luhn"s summarization method too.
|
|
|
Stemming Methods Used in Text Mining
Adámek, Tomáš ; Chmelař, Petr (referee) ; Bartík, Vladimír (advisor)
The main theme of this master's thesis is a description of text mining. This document is specialized to English texts and their automatic data preprocessing. The main part of this thesis analyses various stemming algorithms (Lovins, Porter and Paice/Husk). Stemming is a procedure for automatic conflating semantically related terms together via the use of rule sets. Next part of this thesis describes design of an application for various types of stemming algorithms. Application is based on the Java platform with using of graphic library Swing and MVC architecture. Next chapter contains description of implementation of the application and stemming algorithms. In the last part of this master's thesis experiments with stemming algorithms and comparing the algorithm from viewpoint to the results of classification the text are described.
|
|
|
Digital Library Information Retrieval
Hochmal, Petr ; Rychlý, Marek (referee) ; Chmelař, Petr (advisor)
This thesis deals with methods of information retrieval. Firstly, it describes models of information retrieval and methods of retrieval evaluation. Then it brings closer the principles of the input text processing for IR with use of stopword list and stemmer. Furthermore, it shows the way of the query expansion with synonyms using the thesaurus, methods of handling phrases appearance in queries and introduces the idea of ranking documents by the degree of phrase occurrence similarity in documents. In the second part of this thesis is described the design of whole IR system with using vector model, query expansion with synonyms and phrases handling. This system has been implemented in C# as the application for retrieving and administration of the documents in digital libraries. The effectiveness of this system has been evaluated at the end of this thesis by several tests.
|
|
|
Determination of basic form of words
Šanda, Pavel ; Burget, Radim (referee) ; Karásek, Jan (advisor)
Lemmatization is an important preprocessing step for many applications of text mining. Lemmatization process is similar to the stemming process, with the difference that determines not only the word stem, but it´s trying to determines the basic form of the word using the methods Brute Force and Suffix Stripping. The main aim of this paper is to present methods for algorithmic improvements Czech lemmatization. The created training set of data are content of this paper and can be freely used for student and academic works dealing with similar problematics.
|
|
|
Hledání sémantické informace v textových datech s využitím latentní analýzy
Řezníček, Pavel
The first part of thesis focuses on theoretical introduction to the methods of text mining -- Information retrieval, classification and clustering. LSA method is presented as an advanced model for representing textual data. Furthermore, the work describes source data and methods for their preprocessing and preparation used to enhance the effectiveness of text mining methods. For each chosen text mining method there are defined evaluation metrics and used already existing, or newly implemented, programs are presented. The results of experiments comparing the effects of different preprocessing type and use of different models of the source data are then demonstrated and discussed in the conclusion.
|
|
|
Effect of the Czech Stemming Algorithm on the Document Retrieval
Pytelka, Petr ; Strossa, Petr (advisor) ; Pinkas, Otakar (referee)
This thesis deals with the measurement of the quality of the stemming/lemmatization algo-rithm for the Czech language in document processing systems and provides an analysis of the results. The theoretical part of the thesis describes the principles of the full-text search, the possibilities of implementation as well as the common problems which have to be solved in connection with the processing of natural language. Methods of evaluating the quality of lemmatization, using recall and precision, are discussed. In addition, the theoret-ical part covers the method of measuring the index of under-stemming and over-stemming, which can be applied for the purposes of a more detailed evaluation. An experiment for evaluating the lemmatization algorithms is proposed in the second part of the thesis. A specialized application has been developed to perform the experiment in three different systems, namely Apache Lucene, the PostgreSQL database systems and the Microsoft SQL Server. The experiment is based on the Prague Dependency Treebank cor-pus. It has been carried out both for the corpus as a whole and for selected word classes separately. Further analysis of the results for Czech stemmer in Apache Lucene leads to a proposal for several modifications of the algorithm. Such modifications result in measurable improvements. The results achieved show how metrics discussed, together with the values measured, can be used for improving the lemmatization algorithms and thus to improve the full-text search for Czech language.
|
guest ::
