Original title: Kompozicionalita a disentanglement: Strukturované reprezentace v jednoduchých neuronových modelech
Translated title: Compositionality and Disentanglement: Structured Representations in Simple Neural Models
Authors: Auersperger, Michal ; Pecina, Pavel (advisor) ; Smith, Kenny (referee) ; Le, Phong (referee)
Document type: Doctoral theses
Year: 2026
Language: eng
Abstract: [eng] [cze]
Title: Compositionality and Disentanglement: Structured Representations in Simple Neural Models Author: Michal Auersperger Department: Institute of Formal and Applied Linguistics Supervisor: doc. RNDr. Pavel Pecina, Ph.D., Institute of Formal and Applied Linguistics Abstract: We study conditions that encourage the emergence of structured neural represen- tations in communication games with discrete channels and in autoencoders with continuous channels. Across both settings, we examine two pressures: compres- sion, implemented through message length penalties and bounded activations, and training pressure, introduced through agent resetting and dropout. In signaling games, we propose grammar-based compositionality metrics derived from proba- bilistic context-free grammar induction and show that structured protocols support better generalization under constrained training conditions. For continuous repre- sentations, we introduce a vocabulary-based reconstruction measure that evaluates whether latent fragments can be recombined into valid codes. Experiments in con- trolled settings suggest that compression and training pressures promote represen- tations whose parts can be reused more systematically. Keywords: compositionality, generalization, emergent languages, disentan- glement iii Název práce: K  : S   ý ý  Autor: M A Pracoviště: Ú í    Vedoucí práce: . RND. P P, P.D., Ú í    Abstrakt: Z ,     ý í  č í   ý  (í )   čí  (ý ). V  ří    ů:    ří. K  í       čí . N ří  í ů  . P ěří  í   čí     žý   ěí ý . D , ž     ší ňí,    říě, ž   čí . P   í í   í ů (),  í,   í   ý í    ě ě   ý ů. S čí, ž   í   í, ž ý  č í ý...
Keywords: compositionality|generalization|emergent languages|disentanglement; kompozicionalita|zobecňování|simulace vývoje jazyka|disentanglement

Institution: Charles University Faculties (theses) (web)
Document availability information: Available in the Charles University Digital Repository.
Original record: http://hdl.handle.net/20.500.11956/207781

Permalink: http://www.nusl.cz/ntk/nusl-697514


The record appears in these collections:
Universities and colleges > Public universities > Charles University > Charles University Faculties (theses)
Academic theses (ETDs) > Doctoral theses