|
|
Induktivní syntéza konečně stavových kontrolérů pro decentralizované POMDP
Hranička, Vojtěch ; Lengál, Ondřej (oponent) ; Češka, Milan (vedoucí práce)
Markovovy rozhodovací procesy s částečným pozorováním~(POMDP) jsou významným stochastickým modelem pro sekvenční rozhodování s nejistotou. Decentralizované POMDP~(Dec-POMDP) tento model rozšiřují o možnost práce s více agenty. Tato práce se zaměřuje na rozšíření metody induktivní syntézy kontrolérů pro POMDP, tak aby podporovala práci s Dec-POMDP. Hlavním cílem syntézy je nalézt takové kontroléry pro každého z agentů, aby jejich společné chování nejlépe splňovalo požadované specifikace. V této práci se zaměřuji na strategie, které jsou reprezentovány pomocí konečně stavových kontrolérů~(FSC). Experimentální výsledky ukazují, že použití této metody pro návrh kontrolérů dosahuje srovnatelných výsledků se state-of-the-art přístupy. Navíc tento přístup jako první umožňuje práci v nekonečném horizontu bez použití discount faktoru. Díky tomu je tato metoda vhodnější pro řešení problémů, kde je důležité rozhodnutí provedeno až v pozdějším horizontu.
|
|
|
Advanced Methods for Synthesis of Probabilistic Programs
Stupinský, Šimon ; Holík, Lukáš (oponent) ; Češka, Milan (vedoucí práce)
Probabilistic programs play a crucial role in various engineering domains, including computer networks, embedded systems, power management policies, or software product lines. PAYNT is a tool for the automatic synthesis of probabilistic programs satisfying the given specifications. In this thesis, we extend this tool primarily to support optimal synthesis and synthesis for multi-property specifications. Further, we have proposed and implemented a new method that can efficiently synthesise continuous parameters affecting the transition probabilities alongside the synthesis of a program topology, i.e., support of both topology and parameter synthesis at the same time. We demonstrate the usefulness and performance of PAYNT on a wide range of real-world case studies from various application domains. For challenging synthesis problems, PAYNT can significantly decrease the run-time from days to minutes while ensuring the completeness of the synthesis process.
|
|
|
Improving Synthesis of Finite State Controllers for POMDPs Using Belief Space Approximation
Macák, Filip ; Holík, Lukáš (oponent) ; Češka, Milan (vedoucí práce)
This work focuses on combining two state-of-the-art controller synthesis methods for partially observable Markov decision processes (POMDPs), a prominent model in sequential decision making under uncertainty. A central issue is to find a POMDP controller that achieves a total expected reward objective. As finding optimal controllers is undecidable, we concentrate on synthesising good finite-state controllers (FSCs). We do so by tightly integrating two modern, orthogonal methods for POMDP controller synthesis: a belief-based and an inductive approach. The former method obtains an FSC from a finite fragment of the so-called belief MDP, an MDP that keeps track of the probabilities of equally observable POMDP states. The latter is an inductive search technique over a set of FSCs with a fixed memory size. The key result of this work is a symbiotic anytime algorithm that tightly integrates both approaches such that each profits from the controllers constructed by the other. Experimental results indicate a substantial improvement in the value of the controllers while significantly reducing the synthesis time and memory footprint.
|
|
|
Advanced Methods for Synthesis of Probabilistic Programs
Stupinský, Šimon ; Holík, Lukáš (oponent) ; Češka, Milan (vedoucí práce)
Probabilistic programs play a crucial role in various engineering domains, including computer networks, embedded systems, power management policies, or software product lines. PAYNT is a tool for the automatic synthesis of probabilistic programs satisfying the given specifications. In this thesis, we extend this tool primarily to support optimal synthesis and synthesis for multi-property specifications. Further, we have proposed and implemented a new method that can efficiently synthesise continuous parameters affecting the transition probabilities alongside the synthesis of a program topology, i.e., support of both topology and parameter synthesis at the same time. We demonstrate the usefulness and performance of PAYNT on a wide range of real-world case studies from various application domains. For challenging synthesis problems, PAYNT can significantly decrease the run-time from days to minutes while ensuring the completeness of the synthesis process.
|
host ::
RSS.