Kompozicionalita a disentanglement: Strukturované reprezentace v jednoduchých neuronových modelech

Auersperger, Michal

Title: Compositionality and Disentanglement: Structured Representations in Simple Neural Models Author: Michal Auersperger Department: Institute of Formal and Applied Linguistics Supervisor: doc. RNDr. Pavel Pecina, Ph.D., Institute of Formal and Applied Linguistics Abstract: We study conditions that encourage the emergence of structured neural represen- tations in communication games with discrete channels and in autoencoders with continuous channels. Across both settings, we examine two pressures: compres- sion, implemented through message length penalties and bounded activations, and training pressure, introduced through agent resetting and dropout. In signaling games, we propose grammar-based compositionality metrics derived from proba- bilistic context-free grammar induction and show that structured protocols support better generalization under constrained training conditions. For continuous repre- sentations, we introduce a vocabulary-based reconstruction measure that evaluates whether latent fragments can be recombined into valid codes. Experiments in con- trolled settings suggest that compression and training pressures promote represen- tations whose parts can be reused more systematically. Keywords: compositionality, generalization, emergent languages, disentan- glement iii

guest :: login Digital Repository
		Search		Submit		Help		About