|
|
Central Moments and Risk-Sensitive Optimality in Markov Reward Processes
Sladký, Karel
In this note we consider discrete- and continuous-time Markov decision processes with finite state space. There is no doubt that usual optimality criteria examined in the literature on optimization of Markov reward processes, e.g. total discounted or mean reward, may be quite insufficient to select more sophisticated criteria that reflect also the variability-risk features of the problem. In this note we focus on models where the stream of rewards generated by the Markov process is evaluated by an exponential utility function with a given risk sensitivity coefficient (so-called risk-sensitive models).For the risk sensitive case, i.e. if the considered risk-sensitivity coefficient is non-zero, we establish explicit formulas for growth rate of expectation of the exponential utility function. Recall that in this case along with the total reward also it higher moments are taken into account. Using Taylor expansion of the utility function we present explicit formulas for calculating variance a higher central moments of the total reward generated by the |Markov reward process along with its asymptotic behaviour.
|
|
|
Average Reward Optimality in Semi-Markov Decision Processes with Costly Interventions
Sladký, Karel
In this note we consider semi-Markov reward decision processes evolving on finite state spaces. We focus attention on average reward models, i.e. we establish explicit formulas for the growth rate of the total expected reward. In contrast to the standard models we assume that the decision maker can also change the running process by some (costly) intervention. Recall that the result for optimality criteria for the classical Markov decision chains in discrete and continuous time setting turn out to be a very specific case of the considered model. The aim is to formulate optimality conditions for semi-Markov models with interventions and present algorithmical procedures for finding optimal solutions.
|
|
|
Bytový dům ve městě Vracově
Sladký, Karel ; Žajdlík, Tomáš (oponent) ; Šuhajdová, Eva (vedoucí práce)
Předmětem bakalářské práce je zpracování projektové dokumentace pro novostavbu bytového domu ve Vracově. Objekt bytového domu je navržen jako třípodlažní plně podsklepený objekt. V podzemním podlaží se nachází společenská místnost, vířivka a sklepní kóje. V nadzemních podlažích se nachází byty sloužící k trvalému bydlení. Bylo zde navrženo 6 bytových jednotek. Nadzemní byty disponují lodžiemi. Objekt má plochou střechu s kačírkem. Konstrukční systém objektu je především z dřevo cementových tvarovek vyplněných statickým betonem. Objekt je nad prvním nadzemním podlažím zateplen systémem ETICS. Objekt disponuje výtahovou šachtou. Vodorovné nosné konstrukce jsou navrženy železobetonové stropní desky. Celý objekt je založen na základových pasech. Fasáda je tvořena bílou barvou a imitací dřeva. Bakalářská práce obsahuje projektovou dokumentaci pro provádění stavby.
|
|
|
Markovovy řetězce vyšších řádů a jejich aplikace v ekonometrii
Straňáková, Alena ; Prášková, Zuzana (vedoucí práce) ; Sladký, Karel (oponent)
In this paper, we generalize Raftery's model of Markov chain to a higher-order multivariate Markov chain model. This model is more suitable for practical applications because of smaller number of independent parameters. We propose a method of estimation of parameters of the model and apply it to the Credit risk measuring of a portfolio. We compute Value at Risk and Expected Shortfall in this portfolio. Theoretical results are applied to real data.
|
|
| |
|
|
Stochastic Dynamic Programming Problems: Theory and Applications.
Lendel, Gabriel ; Sladký, Karel (vedoucí práce) ; Lachout, Petr (oponent)
Názov práce: Úlohy stochastického dynamického programování: teorie a aplikace Autor: Gabriel Lendel Katedra: Katedra pravděpodobnosti a matematické statistiky Vedúci diplomovej práce: Ing. Karel Sladký CSc. e-mail vedúceho: sladky@utia.cas.cz Abstrakt: V predloženej práci študujeme riadené Markovove ret'azce s ohodnotením, ktoré umožňujú modelovat' dynamické systémy, ktorých správanie je čiastočne ná- hodné a čiastočne pod kontrolou. Zaoberáme sa zostavením iteračných postupov, kto- rých cielom je nájst' riadenie systému tak, aby bolo optimálne alebo skoro optimálne vzhl'adom k zvolenému kritériu. Konkrétne v práci skúmame hlavne úlohu nájdenia riadenia, ktoré je optimálne vzhl'adom k celkovému očakávanému diskontovanému výnosu alebo priemernému očakávanému výnosu, či pre diskrétne alebo spojité sys- témy. Študujeme algoritmy iterujúce riadenie (policy iteration) a aproximatívne algo- ritmy iterujúce hodnotu (value iteration). Pre vybrané postupy uvádzame numerickú analýzu konkrétnych problémov. Kl'účové slová: Stochastické dynamické programovanie, riadené Markovove ret'azce s ohodnotením, policy iteration, value iteration
|
|
|
Risk-Sensitivity and Average Optimality in Markov and Semi-Markov Reward Processes
Sladký, Karel
This contribution is devoted to risk-sensitivity in long-run average optimality of Markov and semi-Markov reward processes. Since the traditional average optimality criteria cannot reflect the variability-risk features of the problem, we are interested in more sophisticated approaches where the stream of rewards generated by the Markov chain that is evaluated by an exponential utility function with a given risk sensitivity coefficient. Recall that for the risk sensitivity coefficient equal to zero (i.e. the so called risk-neutral case) we arrive at traditional optimality criteria, if the risk sensitivity coefficient is close to zero the Taylor expansion enables to evaluate variability of the generated total reward. Observe that the first moment of the total reward corresponds to expectation of total reward and the second central moment to the reward variance. In this note we present necessary and sufficient risk-sensitivity and risk-neutral optimality conditions for long run risk-sensitive average optimality criterion of unichain Markov and semi-Markov reward processes.
|
|
|
Second Order Optimality in Markov and Semi-Markov Decision Processes
Sladký, Karel
Semi-Markov decision processes can be considered as an extension of discrete- and continuous-time Markov reward models. Unfortunately, traditional optimality criteria as long-run average reward per time may be quite insufficient to characterize the problem from the point of a decision maker. To this end it may be preferable if not necessary to select more sophisticated criteria that also reflect variability-risk features of the problem. Perhaps the best known approaches stem from the classical work of Markowitz on mean-variance selection rules, i.e. we optimize the weighted sum of average or total reward and its variance. Such approach has been already studied for very special classes of semi-Markov decision processes, in particular, for Markov decision processes in discrete - and continuous-time setting. In this note these approaches are summarized and possible extensions to the wider class of semi-Markov decision processes is discussed. Attention is mostly restricted to uncontrolled models in which the chain is aperiodic and contains a single class of recurrent states. Considering finite time horizons, explicit formulas for the first and second moments of total reward as well as for the corresponding variance are produced.
|
|
|
Risk-sensitive and Mean Variance Optimality in Continuous-time Markov Decision Chains
Sladký, Karel
In this note we consider continuous-time Markov decision processes with finite state and actions spaces where the stream of rewards generated by the Markov processes is evaluated by an exponential utility function with a given risk sensitivitycoefficient (so-called risk-sensitive models). If the risk sensitivity coefficient equals zero (risk-neutral case) we arrive at a standard Markov decision process. Then we can easily obtain necessary and sufficient mean reward optimality conditions and the variability can be evaluated by the mean variance of total expected rewards. For the risk-sensitive case, i.e. if the risk-sensitivity coefficient is non-zero, for a given value of the risk-sensitivity coefficient we establish necessary and sufficient optimality conditions for maximal (or minimal) growth rate of expectation of the exponential utility function, along with mean value of the corresponding certainty equivalent. Recall that in this case along with the total reward also its higher moments are taken into account.
|
|
|
Risk-Sensitive Optimality in Markov Games
Sladký, Karel ; Martínez Cortés, V. M.
The article is devoted to risk-sensitive optimality in Markov games. Attention is focused on Markov games evolving on communicating Markov chains with two-players with opposite aims. Considering risk-sensitive optimality criteria means that total reward generated by the game is evaluated by exponential utility function with a given risk-sensitive coefficient. In particular, the first player (resp. the secondplayer) tries to maximize (resp. minimize) the long-run risk sensitive average reward. Observe that if the second player is dummy, the problem is reduced to finding optimal policy of the Markov decision chain with the risk-sensitive optimality. Recall that for the risk sensitivity coefficient equal to zero we arrive at traditional optimality criteria. In this article, connections between risk-sensitive and risk-neutral Markov decisionchains and Markov games models are studied using discrepancy functions. Explicit formulae for bounds on the risk-sensitive average long-run reward are reported. Policy iteration algorithm for finding suboptimal policies of both players is suggested. The obtained results are illustrated on numerical example.
|
host ::
RSS.