Název: Zkoumání reprezentace plurálu ve velkých jazykových modelech prostřednictvím sondovacích intervencí
Překlad názvu: Investigating Large Language Models' Representations Of Plurality Through Probing Interventions
Autoři: Hanna, Michael ; Mareček, David (vedoucí práce) ; Helcl, Jindřich (oponent)
Typ dokumentu: Diplomové práce
Rok: 2022
Jazyk: eng
Abstrakt: Title: Investigating Large Language Models' Representations Of Plurality Through Probing Interventions Author: Michael Hanna Institute: Institute of Formal and Applied Linguistics Supervisor: RNDr. David Mareček, Ph.D., Institute of Formal and Applied Linguistics Abstract: Large language models (LLMs) have become ubiquitous in natural language processing, but how exactly they process their input and arrive at good downstream task performance is still poorly understood. While much work has been done using probing to examine LLM internals, or behavioral studies, to determine LLMs' linguistic capabilities, these techniques are too weak to allow us to draw conclusions how LLMs process language. In this paper, I use both probing and causal intervention methods to investigate the question of subject-verb agreement with respect to the subject's plurality. I find that while probing reveals that subject plurality information is distributed throughout a sentence, causal interventions suggest that only information stored in linguistically relevant tokens is used. Probing interventions suggest that some but not all probes capture information in a way that reflects LLMs' usage thereof. Keywords: Interpretability, Probing, Natural Language Processing, Computational Linguistics
Klíčová slova: probing|interpretace|jazykový model|neuronová síť; probing|interpretation|language model|neural network

Instituce: Fakulty UK (VŠKP) (web)
Informace o dostupnosti dokumentu: Dostupné v digitálním repozitáři UK.
Původní záznam: http://hdl.handle.net/20.500.11956/175532

Trvalý odkaz NUŠL: http://www.nusl.cz/ntk/nusl-508756


Záznam je zařazen do těchto sbírek:
Školství > Veřejné vysoké školy > Univerzita Karlova > Fakulty UK (VŠKP)
Vysokoškolské kvalifikační práce > Diplomové práce