|
|
Automatization of Generating Product Descriptions With Neural Language Models
Hrouda, Václav ; Kasner, Zdeněk (advisor) ; Helcl, Jindřich (referee)
Product descriptions are an important part of product presentation in e-commerce. This bachelor thesis explores the possibilities of using language models based on the Transformer architecture to generate product descrip- tions based on textual product information. Data from a real ecommerce store was used and three different approaches were tested during the work. Fine-tuning of the GPT2 small Czech model, using the Mistral model with the translation of its inputs and outputs into English and directly using Chat- GPT on the Czech data. A combination of automated metrics and human moderation was used to evaluate the generated texts. The result is a clear ranking of these approaches (ChatGPT, Mistral, GPT2 small Czech), with none proving sufficiently reliable for practical use.
|
|
|
Implicit information extraction from news stories
Kydlíček, Hynek ; Libovický, Jindřich (advisor) ; Helcl, Jindřich (referee)
This work deals with information extraction from Czech News Stories. We focus on four tasks: Publishing server, Article category, Author's textual gender and Publication day of week. Due to the absence of a suitable dataset for the tasks, we present CZEch NEws Classification dataset (CZE-NEC), one of the most extensive Czech classification datasets, composed of news articles from various sources, spanning over twenty years. Tasks are solved using Logistic Regression and pre-trained Transformer encoders. Emphasis is put on fine-tuning methods of the Transformer models, which are evaluated in detail. The models are compared to human evaluators, revealing significant superiority over humans on all tasks. Furthermore, the models are pitted against the commercial large language model GPT-3, outperforming it on half of the tasks, despite GPT-3 being significantly larger. Our work sets strong baseline results on CZE-NEC allowing for further research in the field.
|
|
|
Investigating Large Language Models' Representations Of Plurality Through Probing Interventions
Hanna, Michael ; Mareček, David (advisor) ; Helcl, Jindřich (referee)
Title: Investigating Large Language Models' Representations Of Plurality Through Probing Interventions Author: Michael Hanna Institute: Institute of Formal and Applied Linguistics Supervisor: RNDr. David Mareček, Ph.D., Institute of Formal and Applied Linguistics Abstract: Large language models (LLMs) have become ubiquitous in natural language processing, but how exactly they process their input and arrive at good downstream task performance is still poorly understood. While much work has been done using probing to examine LLM internals, or behavioral studies, to determine LLMs' linguistic capabilities, these techniques are too weak to allow us to draw conclusions how LLMs process language. In this paper, I use both probing and causal intervention methods to investigate the question of subject-verb agreement with respect to the subject's plurality. I find that while probing reveals that subject plurality information is distributed throughout a sentence, causal interventions suggest that only information stored in linguistically relevant tokens is used. Probing interventions suggest that some but not all probes capture information in a way that reflects LLMs' usage thereof. Keywords: Interpretability, Probing, Natural Language Processing, Computational Linguistics
|
|
|
Non-Autoregressive Neural Machine Translation
Helcl, Jindřich ; Hajič, Jan (advisor) ; Duh, Kevin (referee) ; Popel, Martin (referee)
In recent years, a number of mehtods for improving the decoding speed of neural machine translation systems have emerged. One of the approaches that pro- poses fundamental changes to the model architecture are non-autoregressive models. In standard autoregressive models, the output token distributions are conditioned on the previously decoded outputs. The conditional dependence al- lows the model to keep track of the state of the decoding process, which improves the fluency of the output. On the other hand, it requires the neural network computation to be run sequentially, and thus it cannot be parallelized. Non- autoregressive models impose conditional independence on the output distri- butions, which means that the decoding process is parallelizable and hence the decoding speed improves. A major drawback of this approach is lower trans- lation quality compared to the autoregressive models. The goal of the non- autoregressive translation research is to find methods that improve the trans- lation quality, while retaining high decoding speed. In this thesis, we explore the research progress so far and identify flaws in the generally accepted eval- uation methodology. We experiement with non-autoregressive models trained with connectionist temporal classification. We find that even though our models...
|
|
|
Machine Translation of Spoken English into Czech
Cífka, Ondřej ; Bojar, Ondřej (advisor) ; Helcl, Jindřich (referee)
Spoken language translation, the process of translating speech in one language into another language automatically, is in increasing demand as a means of overcoming the language barrier. In this thesis, we focus on translation of spoken English into Czech, employed as an aid for international tourists. We built a fully functional speech translation system using freely available components and used it for collecting samples of user input. We then focused on replacing the core components of the system, namely speech recognition (ASR) and machine translation (MT), with our own, domain-adapted models. We evaluated our improvements on the collected data. Powered by TCPDF (www.tcpdf.org)
|
|
|
Vícejazyčná databáze kolokací
Helcl, Jindřich ; Hajič, Jan (advisor) ; Mareček, David (referee)
Collocations are groups of words which are co-occurring more often than appearing separately. They also include phrases that give a new meaning to a group of unrelated words. This thesis is aimed to find collocations in large data and to create a database that allows their retrieval. The Pointwise Mutual Information, a value based on word frequency, is computed for finding the collocations. Words with the highest value of PMI are considered candidates for good collocations. Chosen collocations are stored in a database in a format that allows searching with Apache Lucene. A part of the thesis is to create a Web user interface as a quick and easy way to search collocations. If this service is fast enough and the collocations are good, translators will be able to use it for finding proper equivalents in the target language. Students of a foreign language will also be able to use it to extend their vocabulary. Such database will be created independently in several languages including Czech and English. Powered by TCPDF (www.tcpdf.org)
|
|
|
Extensible Instant Messenger supporting collaborative drawing
Helcl, Jindřich ; Hnětynka, Petr (advisor) ; Keznikl, Jaroslav (referee)
In the present work we describe a tool for communication between two or more users called iNetPaint. The program supports creation of a shared picture. We can also extend the program by adding new painting tools. iNetPaint runs on the XMPP protocol, which enables us to communicate with other clients using this protocol, such as Jabber, Google Talk and others. This work contains documentation for users and for programmers, a summary of similar programs and differences between them and iNetPaint. Furthermore, the work describes how to create your own new painting tools - their definition and implementation.
|
|
|
Automatic generation of images and their usage as training data
Chaloupský, Lukáš ; Rosa, Rudolf (advisor) ; Helcl, Jindřich (referee)
This thesis deals with the problem of automatic image generation based on input text in natural language. The first part deals with design and implementation of application that will generate an image composed of individual small pictures corresponding to the input description (an English sentence) based on certain rules, patterns and relations between words in the specified input sentence. The scope of objects for generating is infinite, because images are downloaded dynamically using REST API calls. The second goal is then to use this created application for training of a neural network for Image captioning task, i.e. automatic generation of captions for images, and evaluate its impact on quality of outputs from this task. The training was conducted on freely available data and it has been shown that artificial generating of images for training neural networks purposes has positive impact on the image captioning task. 1
|
|
|
Deep learning and visualization of models for image captioning and multimodal translation
Michalik, Samuel ; Helcl, Jindřich (advisor) ; Rosa, Rudolf (referee)
Title: Deep Learning and Visualization of Models for Image Captioning and Multimodal Translation Author: Samuel Michalik Institute: Institute of Formal and Applied Linguistics Supervisor: Mgr. Jindřich Helcl, Institute of Formal and Applied Linguistics Abstract: In recent years, the machine learning paradigm known as deep learning has proven to be well suited for the exploitation of modern parallel hardware and large datasets, helping to advance the frontier of research in many fields of arti- ficial intelligence and finding succesfull commercial applications. Deep learning allows end-to-end trainable systems to tackle difficult tasks by building complex hierarchical representations. However, these internal representations often avoid easy interpretation. We explore the possibilities of interpretable visualizations of attention components and beam search decoding at the task of image captioning and multimodal translation and build an application - Macaque, that can be run as an online service, to meet this end. Furthermore, we propose a novel attention function formulation, called scaled general attention. We experimentally evalu- ate scaled general attention along common attention functions on four different model architectures based on the encoder-decoder framework at the tasks of im- age captioning and...
|
|
|
Využití syntaktické informace pro identifikaci hodnocených entit
Glončák, Vladan ; Hajič, Jan (advisor) ; Helcl, Jindřich (referee)
Opinion Target Extraction (OTE) is a well-established subtask of sentiment analysis. While detecting sentiment polarity is useful in itself, the ability to extract the targets of the opinions allows for more thorough decision making. For example, an owner of a restaurant needs to know whether the guests are complaining about the food, or the ambience, or any other aspect of their establishment, etc. Despite the lexical information being crucial for the task, syntactic structures have potential in being used to correctly decide among multiple candidate entities. Rules based on such structures have been used previously for the task. The objective of this thesis is to investigate, whether syntactic information influences the behavior of the state-of-the-art models such as recurrent neural networks for the OTE task. We did not find any substantial evidence to suggest that adding the syntactic information influences the behavior of the models.
|
guest ::
RSS feed.