Markov chains

Markov chains

1. Type of STOCKHOLD PROCESS

   • The probability of an event occurring
     • MARKOV PROPERTY
       • Depends Only in
         • IMMEDIATELY PREVIOUS EVENT

2. composed by

   • Transition Probabilities
     • Go from state i to j in n units of time
     • Satisfy
       • Equation Chapman - Kolmogorov
   • Transition Matrix
     • It can be obtained in n steps
       • • In case of being homogenous
   • States
     • Variables que describen el sistema
     • Classified as
       • Accesible
         • Achievable from another state
       • Comunicatives
         • Accessible to each other
       • Transients
         • States with no possibility of return
       • Absorbent
         • Only transition to themselves
       • Recurrent
         • Probability to return to themselves
         • Positiv
           • Finite number of states
         • Null
           • Infinite number of states
       • It is said to have period d
         • Non-periodic
           • d=1
         • Periodic
       • Ergodic
         • If it's aperiodic and positive recurrent
   • Communication Classes
     • Equivalence ratio
       • Reflexive
       • Transitive
       • Symmetric
       • If all the states in the chain communicate with each other
         • Irreducible

3. Types of chains

   • Irreducible
     • From any state of E you can access any other
     • All states communicate with each other
     • C(x)=E for a x∈E
     • C(x)=E for every x∈E
   • Positive - Recurring
     • If all the states are positive-recurring
   • Regular
     • If there is any positive power of the transition matrix whose inputs are all strictly greater than zero
   • Absorbing
     • The chain has at least one absorbent state
     • From any non-absorbent state, some absorbent state is accessed
   • Markov chains in continuous time
   • Ergodica
     • If all your states are ergodic

4. Applications

   • Meteorology
     • Rain recurrence models
   • Epidemiological models
     • Process Galton-Watson.
       • Model the development of an epidemic
   • Internet
     • The pagerank of a webpage
   • Simulation
     • Tail theory the Model M/M/1
   • Random Games
     • The model of the ruin of the player
   • Economy and Finance
     • Arbitration Opportunity
     • Collapse of a stock exchange
     • Determine price volatility
   • Musik
     • Algorithms of musical composition
       • Csound
   • Operations
     • Inventaries
     • Maintenance
     • Process Flow

5. Andrei Markov

   • Russian mathematician
     • Contributions in
       • Number Theory
       • Probability Theory

Imagens e arquivos do mapa