3  Practical

3.1 Package

#pak::pak("thiyangt/stochastic")
library(stochastic)

3.1.1 n-step transition probability matrix

library(stochastic)
x <- c(0.2,0.8,0.4,0.6)
nstepmat(x, 2, 3)
[1] "The one-step transition probability matrix is:"
     [,1] [,2]
[1,]  0.2  0.8
[2,]  0.4  0.6
[1] "The 3 -step transition probability matrix is:"
      [,1]  [,2]
[1,] 0.328 0.672
[2,] 0.336 0.664
nstepmat(x, 2, 13)
[1] "The one-step transition probability matrix is:"
     [,1] [,2]
[1,]  0.2  0.8
[2,]  0.4  0.6
[1] "The 13 -step transition probability matrix is:"
          [,1]      [,2]
[1,] 0.3333333 0.6666667
[2,] 0.3333333 0.6666667

3.1.2 Compute stationary probabilities

mat <- matrix(c(0.5, 0.5, 0.7, 0.3), byrow=TRUE, ncol=2)
stationary_prob(onestep=mat)
[1] 0.5833333 0.4166667

3.1.3 Simulate a Markov Chain process

init <- c(0.1, 0.9)
mat <- matrix(c(0.5, 0.5, 0.7, 0.3), byrow=TRUE, ncol=2)
simmarkov(init, mat, 100, c("Rainy", "Sunny"))
  [1] "Sunny" "Rainy" "Sunny" "Rainy" "Rainy" "Sunny" "Rainy" "Rainy" "Sunny"
 [10] "Rainy" "Rainy" "Rainy" "Sunny" "Rainy" "Sunny" "Rainy" "Sunny" "Rainy"
 [19] "Sunny" "Rainy" "Rainy" "Sunny" "Sunny" "Rainy" "Rainy" "Rainy" "Sunny"
 [28] "Rainy" "Sunny" "Rainy" "Rainy" "Sunny" "Sunny" "Rainy" "Rainy" "Sunny"
 [37] "Sunny" "Rainy" "Sunny" "Sunny" "Sunny" "Rainy" "Sunny" "Sunny" "Rainy"
 [46] "Sunny" "Rainy" "Sunny" "Rainy" "Rainy" "Rainy" "Rainy" "Rainy" "Sunny"
 [55] "Sunny" "Rainy" "Rainy" "Sunny" "Rainy" "Rainy" "Sunny" "Rainy" "Sunny"
 [64] "Rainy" "Rainy" "Rainy" "Rainy" "Sunny" "Rainy" "Rainy" "Rainy" "Rainy"
 [73] "Rainy" "Rainy" "Rainy" "Rainy" "Rainy" "Rainy" "Sunny" "Sunny" "Sunny"
 [82] "Rainy" "Rainy" "Sunny" "Sunny" "Rainy" "Rainy" "Sunny" "Rainy" "Sunny"
 [91] "Rainy" "Sunny" "Sunny" "Rainy" "Sunny" "Rainy" "Sunny" "Rainy" "Rainy"
[100] "Sunny" "Rainy"

3.2 In-class illustration

We are given sequences of emotional states for five individuals (person1–person5). Each sequence represents how a person’s mood changed over time.

person1 <- c("happy", "calm", "neutral", "sad", "angry", "stressed", "happy")
person2 <- c("happy", "neutral", "calm", "sad", "angry", "happy", "stressed")
person3 <- c("calm", "happy", "neutral", "sad", "angry", "stressed", "happy")
person4 <- c("happy", "calm", "sad", "neutral", "angry", "stressed", "happy")
person5 <- c("happy", "calm", "sad", "angry", "neutral", "happy", "stressed")
person6 <- c("happy", "calm", "neutral", "sad")

We want to use these sequences to:

Estimate a transition probability matrix

Predict the next emotional state for person6

Visualise the Markov chain

3.3 Question 1

An insurance company offers four levels of discounts to policyholders based on their claim history:

Level Discount
1 0%
2 20%
3 30%
4 40%

The following are the rules for shifting between the four levels:

  • A policyholder at Level 1 moves to Level 2 with probability 0.5, stays at Level 1 with probability 0.5.

  • A policyholder at Level 2 moves to Level 1 with probability 0.2, moves to Level 3 with probability 0.3, and stays at Level 2 with probability 0.5.

  • A policyholder at Level 3 moves to Level 2 with probability 0.3, moves to Level 4 with probability 0.4, and stays at Level 3 with probability 0.3.

  • A policyholder at Level 4 moves to Level 3 with probability 0.2, and stays at Level 4 with probability 0.8.

Tasks

  1. Represent the system as a Markov chain using a transition probability matrix in R.

  2. Write an R function to simulate the Markov chain for 50 time periods starting from Level 1.

  3. Compute the stationary distribution of the Markov chain.

  4. Plot the simulated discount levels over time.

  5. Compute the limiting (long-run) probabilities of being in each discount level.

3.4 Question 2

Generating and Simulating a Lung Cancer Metastasis Markov Chain

Read “A Stochastic Markov Chain Model to Describe Lung Cancer Growth and Metastasis” by Paul K. Newton et al. (2012). Based on the model presented in the paper:

  1. Identify the key anatomical sites relevant for lung cancer metastasis and represent them as states in a discrete-time Markov chain.

  2. Using the descriptions and data from the paper, construct a transition probability matrix that represents the likelihood of metastasis from one site to another in one time step.

  3. Write an R function to simulate the evolution of a single patient’s metastasis over \(n\) time steps, starting from the primary lung tumor.

  4. Use your function to simulate multiple patients (e.g., 1000) over a fixed number of steps (e.g., 20) and summarize the proportion of patients in each site at the final step.

  5. Discuss how your simulation results relate to the biological progression described in the paper.