Název: Hladkost funkcí naučených neuronovými sítěmi
Překlad názvu: Smoothness of Functions Learned by Neural Networks
Autoři: Volhejn, Václav ; Musil, Tomáš (vedoucí práce) ; Straka, Milan (oponent)
Typ dokumentu: Bakalářské práce
Rok: 2020
Jazyk: eng
Abstrakt: Modern neural networks can easily fit their training set perfectly. Surprisingly, they generalize well despite being "overfit" in this way, defying the bias-variance trade-off. A prevalent explanation is that stochastic gradient descent has an implicit bias which leads it to learn functions that are simple, and these simple functions generalize well. However, the specifics of this implicit bias are not well understood. In this work, we explore the hypothesis that SGD is implicitly biased towards learning functions that are smooth. We propose several measures to formalize the intuitive notion of smoothness, and conduct experiments to determine whether these measures are implicitly being optimized for. We exclude the possibility that smoothness measures based on first derivatives (the gradient) are being implicitly optimized for. Measures based on second derivatives (the Hessian), on the other hand, show promising results. 1
Klíčová slova: hladkost; neuronové sítě; strojové učení; zobecňování; generalization; machine learning; neural networks; smoothness

Instituce: Fakulty UK (VŠKP) (web)
Informace o dostupnosti dokumentu: Dostupné v digitálním repozitáři UK.
Původní záznam: http://hdl.handle.net/20.500.11956/119446

Trvalý odkaz NUŠL: http://www.nusl.cz/ntk/nusl-415933


Záznam je zařazen do těchto sbírek:
Školství > Veřejné vysoké školy > Univerzita Karlova > Fakulty UK (VŠKP)
Vysokoškolské kvalifikační práce > Bakalářské práce