Hierarchical forecasting of vaccine supply chain

.sticker-left[![isf](resources/stickeriff.jpeg)]
.sticker-float[![logo](resources/carbs_logo1.jpg)]

<br><br>
.center[.title[Hierarchical forecasting of vaccine supply chain]]

.bottom[
Bahman Rostami-Tabar (<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#1da1f2;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg>[@Bahman_R_T](https://twitter.com/Bahman_R_T))
<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M131.5 217.5L55.1 100.1c47.6-59.2 119-91.8 192-92.1 42.3-.3 85.5 10.5 124.8 33.2 43.4 25.2 76.4 61.4 97.4 103L264 133.4c-58.1-3.4-113.4 29.3-132.5 84.1zm32.9 38.5c0 46.2 37.4 83.6 83.6 83.6s83.6-37.4 83.6-83.6-37.4-83.6-83.6-83.6-83.6 37.3-83.6 83.6zm314.9-89.2L339.6 174c37.9 44.3 38.5 108.2 6.6 157.2L234.1 503.6c46.5 2.5 94.4-7.7 137.8-32.9 107.4-62 150.9-192 107.4-303.9zM133.7 303.6L40.4 120.1C14.9 159.1 0 205.9 0 256c0 124 90.8 226.7 209.5 244.9l63.7-124.8c-57.6 10.8-113.2-20.8-139.5-72.5z"/></svg>[www.bahmanrt.com](https://www.bahmanrt.com/)
]

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- Immunization supply chain

- Forecasting problem

- Data of vaccine consumption in  & forecasting setup

- Forecast accuracy evaluation

- Next steps
]

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- .remember[Immunization supply chain]

- .gray[Forecasting problem]

- .gray[Data of vaccine consumption in  & forecasting setup]

- .gray[Forecast accuracy evaluation]

- .gray[Conclusion & next steps]

]

---
class: center, middle
.pull-left[
<img src="img/f4sg2022_vaccine/children_immunization.png" height="350px">
]

.left[
.pull-right[
- Approximately .remember[1 in 5] African children .remember[do not receive] all basic vaccines.
- More than .remember[30 million children] under five still .remember[suffer from vaccine-preventable diseases (VPDs)] every year in Africa
- Over .remember[half a million children die] from VPDs annually
]]

.footnote[source: [**WHO**](https://www.afro.who.int/health-topics/immunization])

---

## Vaccines

.three-column[
### Vial
<img src="img/f4sg2022_vaccine/vial.jpg" height="600px">
]

.three-column[
### Dose
<img src="img/f4sg2022_vaccine/dose.jpg" height="600px">
]

.three-column[
### Administrated
<iframe src="https://giphy.com/embed/xT5LMGfQrJPpmXKUEM" height="600px" allowFullScreen></iframe><p><a href="https://giphy.com/gifs/season-16-the-simpsons-16x11-xT5LMGfQrJPpmXKUEM">via GIPHY</a></p>
]
---
.center[
.pull-left[
### Open wastage
<img src="img/f4sg2022_vaccine/open_waste.jpeg" height="600px" >
]

.pull-right[
### Close wastage
<img src="img/f4sg2022_vaccine/vaccine_wastage.jpeg" height="600px">
]
]

---
class: inverse, center, middle

.title[Doses used (consumption/needs) <br> = <br>doses administrated + wastage]

---
## The immunization supply chain

.footnote[Source: Effective Vaccine Management Assessment (EVM), 2021]

---
## What do we want to achieve?

.pull-left[
### High coverage
<img src="img/f4sg2022_vaccine/coverage.jpg" height="500px">
]

.pull-right[
✅  Lower waste

✅ Lower inventory costs

✅  Reduce missed opportunities (stock out)

✅  Lower societal costs

✅  and better coordination
]

.footnote[source:[Allan etal.(2021)](https://www.medrxiv.org/content/10.1101/2021.02.14.21251721v1.full)
]

---
class:inverse, center, middle

.title[ Forecasting accurately the needs for vaccines is one of the key elements in achieving these goals]

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- .gray[Immunization supply chain]

- .remember[Forecasting problem]

- .gray[Data of vaccine consumption in  & forecasting setup]

- .gray[Forecast accuracy evaluation]

- .gray[Next steps]

]

---
## Hierarchical structure

.center[
<img src="img/f4sg2022_vaccine/hierarchy.png" height="380px">
]

---
## Classical approaches to forecast vaccine needs

- Demographic methods 
      - expected target population
      - coverage is the expected coverage rate
      - doses/target is the number of doses per target, as per the national vaccination schedule
      - wastage factor

- Service-based forecasting method 
      - number of vials per session
      - effective number of vaccination sessions during the year
      - the vial size of the selected vaccine to be used

.alert-note[.remember[**Forecasts are annual and at the national level**]]

---
## Limitations of current forecasting method

❌  Forecasts are yearly and at the national level.

❌  Not useful to inform operational / tactical decisions.

❌  Do not capture the information available at multiple hierarchical levels.

❌  Forecasts ignore the hierarchical nature of the problem, not coherent.

❌  lead to conflicting decisions & lack of coordination.

---
## How to forecast hierarchical time series?

.pull-left[
- Bottom-Up
- Top Down
- Middle-Out
- Optimal reconciliation
]

.pull-right[
<img src="img/f4sg2022_vaccine/pyramid.jpg" width="90%" style="display: block; margin: auto;" />
]
 
---
 
## Optimal reconceiliation
 
- This approach involves first generating independent base forecast for each series in the hierarchy (i.e. Base)

- As these base forecasts are independently generated they will not be “aggregate consistent” (i.e., they will not add up according to the hierarchical structure).

- The optimal combination approach optimally combines the independent base forecasts and generates a set of revised forecasts that are as close as possible to the univariate forecasts but also aggregate consistently with the hierarchical structure.

- Unlike any other existing method, this approach uses all the information available within a hierarchy.

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- .gray[Immunization supply chain]

- .gray[Forecasting problem]

- .remember[Data of vaccine consumption in  & forecasting setup]

- .gray[Forecast accuracy evaluation]

- .gray[Next steps]

]

---
## Data

- Vaccine consumption in an African country from January 2013 until December 2021

- Four type of vaccines
    - Measles
    - Bacillus Calmette–Guérin (BCG) - for tuberculosis
    - DPT, a class of combination vaccines against three infectious diseases in humans: diphtheria-tetanus-pertussis
    - OPV, Oral poliovirus vaccines, used in the fight to eradicate polio
  
- 306 sub-county, 47 county & 9 regions
- Total of 1452 time series

---
## Doses administrated: National & regions

.center[
<img src="img/f4sg2022_vaccine/p1.png" height="600px">
]

---
## Doses administrated: trend and seasonality features

.center[
<img src="img/f4sg2022_vaccine/admin_hts_feature_cat_plot.png" height="600px">
]
---
## Doses used: total and regions

---
## Forecasting setup

- Forecasting method: ETS

- Forecast horizon: 12 months

- Time series cross-validation with re-estimation

---
## Accuracy measures

.pull-left[
### Root Mean Squared Scaled Error
`\(\text{RMSSE} = \sqrt{\text{mean}(q_{j}^2)}\)`,

where

`\begin{equation}
q^2_{j} = \frac{\displaystyle e^2_{j}}
    {\displaystyle\frac{1}{T-m}\sum_{t=m+1}^T (y_{t}-y_{t-m})^2},
\end{equation}`
]

.pull-right[
### Mean Absolute Scaled Error
`\(\text{MASE} = \text{mean}(|q_{j}|).\)`,

where

`\begin{equation}
q_{j} = \frac{\displaystyle e_{j}}
    {\displaystyle\frac{1}{T-m}\sum_{t=m+1}^T |y_{t}-y_{t-m}|}.
\end{equation}`
]

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- .gray[Immunization supply chain]

- .gray[Forecasting problem]

- .gray[Data of vaccine consumption in  & forecasting setup]

- .remember[Forecast accuracy evaluation]

- .gray[Conclusion & next steps]

]

---
## BCG - RMSSE

<table class=" lightable-classic" style="font-family: Cambria; width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Method </th>
   <th style="text-align:right;"> Total </th>
   <th style="text-align:right;"> Region </th>
   <th style="text-align:right;"> County </th>
   <th style="text-align:right;"> Sub County </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Base </td>
   <td style="text-align:right;"> 0.526 </td>
   <td style="text-align:right;"> 0.594 </td>
   <td style="text-align:right;"> 0.656 </td>
   <td style="text-align:right;"> 0.651 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Bottom Up </td>
   <td style="text-align:right;"> 0.509 </td>
   <td style="text-align:right;"> 0.594 </td>
   <td style="text-align:right;"> 0.645 </td>
   <td style="text-align:right;"> 0.649 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Top Down </td>
   <td style="text-align:right;"> 0.526 </td>
   <td style="text-align:right;"> 0.807 </td>
   <td style="text-align:right;"> 0.739 </td>
   <td style="text-align:right;"> 0.717 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Optimal Reconciliation </td>
   <td style="text-align:right;"> 0.458 </td>
   <td style="text-align:right;"> 0.570 </td>
   <td style="text-align:right;"> 0.630 </td>
   <td style="text-align:right;"> 0.649 </td>
  </tr>
</tbody>
</table>

---
## DPT - RMSSE

<table class=" lightable-classic" style="font-family: Cambria; width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Method </th>
   <th style="text-align:right;"> Total </th>
   <th style="text-align:right;"> Region </th>
   <th style="text-align:right;"> County </th>
   <th style="text-align:right;"> Sub County </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Base </td>
   <td style="text-align:right;"> 0.603 </td>
   <td style="text-align:right;"> 0.661 </td>
   <td style="text-align:right;"> 0.748 </td>
   <td style="text-align:right;"> 0.723 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Bottom Up </td>
   <td style="text-align:right;"> 0.574 </td>
   <td style="text-align:right;"> 0.635 </td>
   <td style="text-align:right;"> 0.724 </td>
   <td style="text-align:right;"> 0.723 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Top Down </td>
   <td style="text-align:right;"> 0.603 </td>
   <td style="text-align:right;"> 0.785 </td>
   <td style="text-align:right;"> 0.838 </td>
   <td style="text-align:right;"> 0.816 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Optimal Reconciliation </td>
   <td style="text-align:right;"> 0.579 </td>
   <td style="text-align:right;"> 0.650 </td>
   <td style="text-align:right;"> 0.738 </td>
   <td style="text-align:right;"> 0.734 </td>
  </tr>
</tbody>
</table>

---
## OPV - RMSSE

<table class=" lightable-classic" style="font-family: Cambria; width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Method </th>
   <th style="text-align:right;"> Total </th>
   <th style="text-align:right;"> Region </th>
   <th style="text-align:right;"> County </th>
   <th style="text-align:right;"> Sub County </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Base </td>
   <td style="text-align:right;"> 0.641 </td>
   <td style="text-align:right;"> 0.687 </td>
   <td style="text-align:right;"> 0.765 </td>
   <td style="text-align:right;"> 0.732 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Bottom Up </td>
   <td style="text-align:right;"> 0.599 </td>
   <td style="text-align:right;"> 0.648 </td>
   <td style="text-align:right;"> 0.737 </td>
   <td style="text-align:right;"> 0.732 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Top Down </td>
   <td style="text-align:right;"> 0.641 </td>
   <td style="text-align:right;"> 0.831 </td>
   <td style="text-align:right;"> 0.823 </td>
   <td style="text-align:right;"> 0.788 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Optimal Reconciliation </td>
   <td style="text-align:right;"> 0.619 </td>
   <td style="text-align:right;"> 0.670 </td>
   <td style="text-align:right;"> 0.757 </td>
   <td style="text-align:right;"> 0.744 </td>
  </tr>
</tbody>
</table>

---
## Measles - RMSSE

<table class=" lightable-classic" style="font-family: Cambria; width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Method </th>
   <th style="text-align:right;"> Total </th>
   <th style="text-align:right;"> Region </th>
   <th style="text-align:right;"> County </th>
   <th style="text-align:right;"> Sub County </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Base </td>
   <td style="text-align:right;"> 1.171 </td>
   <td style="text-align:right;"> 1.086 </td>
   <td style="text-align:right;"> 1.063 </td>
   <td style="text-align:right;"> 1.029 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Bottom Up </td>
   <td style="text-align:right;"> 1.059 </td>
   <td style="text-align:right;"> 1.052 </td>
   <td style="text-align:right;"> 1.026 </td>
   <td style="text-align:right;"> 1.029 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Top Down </td>
   <td style="text-align:right;"> 1.171 </td>
   <td style="text-align:right;"> 1.148 </td>
   <td style="text-align:right;"> 1.096 </td>
   <td style="text-align:right;"> 1.061 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Optimal Reconciliation </td>
   <td style="text-align:right;"> 1.069 </td>
   <td style="text-align:right;"> 1.053 </td>
   <td style="text-align:right;"> 1.054 </td>
   <td style="text-align:right;"> 1.081 </td>
  </tr>
</tbody>
</table>

---
background-image: url("resources/hierarchy-left.jpeg")
background-size: contain
background-position: left
class: middle

.pull-right2[
## Outline

- .gray[Immunization supply chain]

- .gray[Forecasting problem]

- .gray[Data of vaccine consumption in  & forecasting setup]

- .gray[Forecast accuracy evaluation]

- .remember[Next steps]

]

---
## Next steps

- Check and fix the data quality issues for doses used

- Report the forecast accuracy improvement aganist the current approach

- Create probabilistic forecasts and report its evaluation

- Use some useful predictors

---
## Next steps
### Using conflict data

.center[
<iframe src="https://ucdp.uu.se/country/501" width="100%" height="500px" data-external="1"></iframe>
]

---

## Acknowledgement

- John and Snow Inc team

- The representative of the National Vaccine Immunization Program for the African country

- [Developers of Tidyverts](https://tidyverts.org/) and [Tidyverse](https://www.tidyverse.org/) packages

- Harsh Charma for his support

---
## References

- Effective Vaccine Management (EVM 2.0) Assessment Report –  2021, WHO

- Vaccine Logistics, WHO training manual

- [Forecasting: Principles and Practice]([https://otexts.com/fpp3/hierarchical.html], (3rd ed, Rob J Hyndman and George Athanasopoulos]

---
.pull-left[
## About me
Bahman Rostami-Tabar <br>
Associate Professor in Business Analytics
Cardiff University, UK
<br><br>

Slides @ [www.bahmanrt.com](www.bahmanrt.com)
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:#1da1f2;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg>[@Bahman_R_T](https://twitter.com/Bahman_R_T)
<svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:blue;overflow:visible;position:relative;"><path d="M100.28 448H7.4V148.9h92.88zM53.79 108.1C24.09 108.1 0 83.5 0 53.8a53.79 53.79 0 0 1 107.58 0c0 29.7-24.1 54.3-53.79 54.3zM447.9 448h-92.68V302.4c0-34.7-.7-79.2-48.29-79.2-48.29 0-55.69 37.7-55.69 76.7V448h-92.78V148.9h89.08v40.8h1.3c12.4-23.5 42.69-48.3 87.88-48.3 94 0 111.28 61.9 111.28 142.3V448z"/></svg> Connect: [Bahman ROSTAMI-TABAR](https://www.linkedin.com/in/bahman-rostami-tabar-1046171a/)
]

.pull-right[
## Outline of my talk

- Immunization supply chain

- Forecasting problem

- Data of vaccine consumption in  & forecasting setup

- Forecast accuracy evaluation

- Next steps
]