Case Study: Alive-Dead Model
Introduction and Overview of Decision Problem
This case study is designed to walk you through the process of constructing a discrete time Markov cohort model in Amua. We will begin with a simple alive-dead model for your country, and then expand into a disease-based model in our case study later today.
1. Alive-Dead Model
We will start by constructing a simple Markov model representing just two health states: “Alive” and “Dead.” We will assume that individuals can transition between these states based on the probabilities of surviving as calculated using demographic life table data from your country.
Markov Structure: Alive-Dead
The structure of the Markov model for the Alive-Dead model is shown in the figure below.
Looking at the figure, you’ll see that Amua has a special Markov node (represented by 
). The branches that lead off a Markov node designate all the Markov health states (and only Markov states).
In this example, there are two health states: (1) Alive, (2) Dead. Off each health state, you can create a subtree (also called a cycle tree) that reflects those events that can occur during a cycle. The last branch at the end of each pathway will be a state transition, which defines what health state to go to for the next cycle.
Note that in a Markov model, outcomes are defined elsewhere – NOT at the end of the branch, but at the state.
Building the Tree
Structure
After you open Amua, click Model New Markov Model.
Save your model right at the start.
Now select the decision node
, Right-click Add Markov Chain .Start by developing the structure of the Markov model using Alive and Dead for the different health states. The branches of the Markov chain should correspond to the states of the model. Label the name option to the right of the decision node as Alive-Dead.
When you reach the end of the branch, select the chance node you would like to turn into a state transition, right click, select Change to State Transition. This will give you the blue arrow
. On the right of this arrow, you can find a dropdown menu with the different health states you specified. Select the health state this part of the cohort will transition to.With this button
you can align the end nodes.
Cohort size and starting probabilities
- Go to Model Properties select the Simulation tab. The default cohort size is 1,000. Change cohort size to 1, so that all outputs will be at the individual level (i.e., per person).
- At the Markov Chain , change the initial probabilities of health states. Since all patients start in the Alive state, change the p:0 on the left of the Local state to p:1 (see blue box below).
Model Parameters
First, define the following time-constant parameters for the model in the “Parameters” panel.
This variable tells Amua that we will model a cohort of newborns. However, we could easily adapt this to model a cohort of 20 year-olds, etc.
Transition Probabilities
The underlying transition probability matrix has the following form:
However, because we are modeling a cohort of newborns until death, we need to include death transition probabilities that vary by age (i.e., cycle) in the model. In other words, there will be a different death probability in each cycle.
In Amua, time-varying inputs are operationalized as variables 
rather than parameters.
Variables can be defined to keep track of model events and dynamically update expressions as the model runs. Thus, unlike parameters 
which are fixed for a given run of the model, variables can change within a simulation. Variables may change across individuals, allowing heterogeneity to be modeled, or they may change over time.
Therefore, we will define p_die as a variable that references values in a lookup table 
, following the steps below.
A lookup table returns the value that corresponds to a particular index. The first column in a lookup table contains the table indices, and one or more columns of lookup values can be defined. Index values must be unique and in ascending order.
Depending on how your data are defined, you can use three lookup table methods in Amua:
Exact: Returns the value associated with the exact index. Use this only if the index values exactly align with the cycle length (e.g., ages 0, 1, 2, etc. with an annual cycle length). If the exact index is not found the table returns NaN (i.e., a missing value).
Truncate: Returns a value that is truncated (i.e. floor) for the index. This is a common way to use life-tables that bin ages into ranges. If the supplied index is below the minimum index the table returns NaN.
Interpolate: Returns a value that is interpolated between indices. Interpolate may be a viable option if you have age-based probabilities defined at the year level, but cycle lengths that are less than a year.
For details about these methods visit the Amua Wiki on GitHub.
We first define a lookup table tbl_p_die to read in the age-dependent background mortality constructed from country life tables.
Download Data
Download the background mortality file for your country from the workshop website. Each workbook contains the age-indexed annual mortality probability for various countries represented in our workshop.
| Bangladesh | Rwanda |
| Cambodia | South Africa |
| Cameroon | Sri Lanka |
| China | Tanzania |
| Ethiopia | Thailand |
| India | Uganda |
| Kenya | Zambia |
| Mozambique | Zimbabwe |
| Philippines |
A screenshot of the first few rows of this .csv file is shown below. Theindexcolumn is the age, whilep_die is the annual probability of death at each age.
Go to the “Tables” panel, click 
to add a table, and you should see a “Define Table” window (see screenshot below). Click the Import button and select the downloaded .csv file to import. The table will automatically be resized to fit the imported data. The first row in the file will be used as table headers.
An alternative approach to uploading the .csv is to simply copy and paste the two columns from the .csv file, and then hit the 
paste button. You can then paste the table directly into Amua.
Enter “tbl_p_die” as the name for this variable. “Lookup” is the default table type (which is what we want here). Choose “Interpolate” as the Lookup Method. Click Save to apply.
Background Mortality
Next, we can define the time-varying variable p_die (which reads from the table we just defined and operates as the background mortality probability in the current model cycle, \(t\)).
Make sure you create a variable and not a parameter. Under “Variables”, click to add a variable p_die = tbl_p_die[age_initial + t, 1].
The first column of a table is always used for indexing in tables. Therefore, the number “1” here will actually indicate the second column in your table as the corresponding value. You can also use the name of the column to read a value, e.g., tbl_p_die[age_initial + t, “p_die”].
In Amua, “t” is a built-in variable that tracks the number of cycles. It automatically updates when the model runs. For example, in the 10th cycle on the Markov model, t = 10. Therefore, in the formula “tbl_p_die[age_initial + t, 1]”, age_initial + t will equal the modeled age of the individual in cycle t, and the full formula will read the age-dependent background mortality based on the individual’s current age (instead of the initial age).
Amua defaults to a cycle time of \(t=0\). Therefore, if you hit “Evaluate” the Expected Value field should calculate the death probability for a newborn as shown in the .csv lookup table.
We now have all transition probabilities defined. Add these inputs to the branches of the model. Remember to add the complementary probabilities using “C” as well.
Termination Condition
- Specify the termination of the model by selecting [termination] and type t==110. This will allow your model to run for 110 cycles, then the model will terminate.
Rewards
The default reward in Amua is cost. We need to change this to have life-expectancy (LE) as a reward. Go to Model Properties select the Analysis tab and change the cost dimension to LE. Use LE as the symbol and set to 4 decimal places.
Add the life-expectancy “payoff” on the left of each health state after “R: (LE)”.
Apply Half-cycle correction. Go to Model Properties select the Markov tab and check the Half-cycle correction box. Click OK to apply.
Now, we have finished constructing the Alive-Dead model. Analyze the model using a cohort simulation by clicking Run Run Model.