Learning outcomes

In this activity you will learn:

• How to design a multi-cluster pfhrp2/3 deletion study.
• How to analyse and interpret the results of a pfhrp2/3 deletion study.
• How to account for intra-cluster correlation in other study designs, such as prevalence surveys and presence/absence studies.

Disclaimer: The scenarios in this document are entirely fictitious. While real place names are used, the data itself is artificial and designed for teaching purposes only. It does not necessarily represent the real epidemiological situation in these locations.

Background

You have been tasked by the Ethiopian National Malaria Control Programme (NMCP) to assist with design and implementation of a study into pfhrp2/3 deletions. Following the WHO 2020 recommendation, this study will be multi-site and conducted at a regional level. It will establish whether the prevalence of pfhrp2/3 deletions among clinical malaria cases is significantly above the 5% threshold. If so, this will trigger a nationwide switch in rapid diagnostic tests (RDTs) away from tests that rely exclusively on the HRP2 protein.

A full study of this sort would normally be run in parallel over multiple regions, however, for the purposes of this tutorial we will focus exclusively on the Amhara region.

Using sample size tables

First, we need to scope out roughly how many sites and how many samples are required. The easiest way to do this is via pre-calculated sample size tables. Go to the pfhrp2/3 Planner and navigate to the Explore tab.

You should see a sample size table with the number of sites (health facilities) in rows, and the assumed prevalence of pfhrp2/3 deletions in columns (like below). The page automatically loads with the default ICC of 0.05 and the WHO 5% prevalence threshold.

In any power analysis, we must begin by assuming a known effect size under the alternative hypothesis. For this study, that means selecting an assumed prevalence of pfhrp2/3 deletions in the population. This assumption can be challenging and may feel somewhat arbitrary. It’s important to remember that as the assumed prevalence decreases towards 5% detection threshold, the required sample size increases significantly. Conversely, assuming a very high prevalence may reduce the public health relevance of the study by focusing power on detecting only large shifts in the parasite population—changes that could already be affecting clinical diagnosis.

To balance these factors, we recommend a default assumed prevalence of 10% (highlighted in purple). This value provides a reasonable sensitivity level while keeping sample sizes manageable. For more insights on selecting this parameter, refer to the FAQ section in the web-based pfhrp2/3 Planner.

As the number of sites recruited increases, the total sample size actually decreases. This may seem counter-intuitive, as we might expect that adding more clusters would inherently make the study larger. What’s happening here is that with more clusters, the samples become closer to independent. By sampling from a greater number of distinct sites, we reduce the likelihood of repeatedly sampling from the same sub-population, thereby minimizing the impact of intra-cluster correlation. As a result, fewer samples are needed overall to achieve the same level of statistical power.

Factoring in financial considerations

The Ethiopian NMCP has provided you with rough estimates of the costs of various aspects of the planned study, including the costs of RDT testing, microscopy, staff time, transport of samples and training. They estimate it will cost $500 USD per health facility (cluster) recruited, plus an additional $6 USD per sample enrolled.

Selecting sites

Next, we have to choose how to select the 7 sites. Common choices are 1) sentinel site surveillance, where known and established sites are chosen based on feasibility, and 2) random selection, where a complete list of all health facilities is compiled and a defined number of sites is drawn at random from this list.

Some advantages of sentinel site surveillance include:

• Consistency and Comparability Over Time: Sentinel sites are typically chosen for long-term monitoring, enabling consistent data collection over time. This allows for more precise trend analysis and easier comparisons of changes in disease prevalence.

• Cost-Effectiveness and Established Infrastructure: Sentinel sites are often strategically chosen for accessibility and logistical ease, making them more cost-effective than random site selection, which may require travel to remote or difficult-to-reach locations. Additionally, these sites usually have established infrastructure, trained personnel, and local relationships, streamlining data collection and improving data quality.

• Targeting High-Burden or Priority Areas: Sentinel sites are often selected based on epidemiological significance, such as higher disease burden or strategic relevance for outbreak monitoring. This targeted approach can be advantageous in understanding and responding to disease patterns in high-risk areas, which might be overlooked in a random selection approach. In addition, numbers of positive tests are easier to obtain.

On the other hand, there are some advantages of random site selection:

• Improved Representativeness and Reduced Selection Bias: Random site selection from a complete list provides a more representative sample of the population and minimizes biases associated with known sentinel sites, which are often chosen based on specific criteria like accessibility or historical disease burden. This approach enhances the validity and generalizability of prevalence estimates, making them more applicable across various areas, including those not included in the survey.

• Greater Diversity of Environmental Conditions: Randomly selected sites are more likely to capture a range of environmental, socio-economic, and health system conditions. This variation is crucial for understanding factors that influence disease prevalence and for developing interventions that can apply across diverse settings.

In this tutorial we will go with random selection of sites. A complete list of all health facilities in the Amhara region is compiled, and 7 sites are selected from this list at random. Table 1 gives the chosen sites and the target sample size based on our initial calculations.

Refining sample sizes

With a preliminary study plan in place, we can now incorporate some real-world constraints. After discussions with site leads at each of the seven locations, we’ve learned that several sites may face challenges in reaching the target sample size. This could stem from factors like limited staffing, low malaria incidence, or other local issues. To address this, you’ve adjusted the sample sizes by increasing recruitment targets at sites where it is feasible to do so. Here’s the updated sample size table:

But now we are faced with a problem - how do we know if this new design has adequate power? We can no longer rely on sample size tables because these assume the same sample size per cluster. On the other hand, we cannot simply assume that power will be adequate. This is where the Design tab of the pfhrp2/3 Planner becomes useful.

Download a .csv file with the current study plan from here.

You can then upload this spreadsheet into the we app using the “Upload a .csv file” option. You should find that the table populates with these values.

You’ll notice that drop-out is also considered at this stage. We are assuming 10% drop-out in each site, but for a more rigorous approach you may want to factor in different drop-out levels by site. When you click “Calculate adjusted sample sizes” you should see a table that contains the adjusted sample size, buffered for drop-out.

Now we are ready to estimate power. On the same page you will find drop-down menus where you can set the assumed prevalence of pfhrp2/3 deletions and the intra-cluster correlation. When you hit the “Estimate power” button the tool will run a simulation-based power analysis (this may take a few seconds).

Because this result is simulation-based we only get an estimate of the power, and not an exact figure. If you want a more precise estimate you can try increasing the number of simulations, although this will take longer to run.

Running an estimate with 1000 simulations should show that the study’s power is close to 80%, with the 95% confidence interval likely spanning this target. For future reference, you can download a copy of your power analysis from the “Generate report” tab. Based on these results, the study plan appears statistically sound and ready for consideration. However, final approval should also take into account logistical and financial factors to ensure feasibility.

Estimating the ICC

As noted above, the DRpower method automatically accounts for ICC when estimating prevalence, so a separate ICC estimate is not strictly needed. However, obtaining an ICC estimate can still be valuable for future study planning. It’s a straightforward process and contributes to a broader understanding of ICC levels, which can benefit researchers planning similar studies in the future or in neighbouring regions.

Still on the Analysis tab, scroll down to the “Estimate ICC” button (it should look like the screenshot below). Clicking this button gives an estimate of the ICC, including a point estimate and a 95% Credible Interval (CrI).

We can navigate to the “Generate report” tab to download a report detailing our analysis.

Recap key points

Our pfhrp2/3 analysis is now complete. We began with a well-powered study design, carefully crafted to be robust and realistic. We reinforced the reliability of our approach by accounting for varying sample sizes across sites and anticipating potential drop-out. Beyond statistical considerations, we incorporated logistical and financial constraints to ensure the study’s feasibility. Importantly, we accounted for possible overdispersion at both the design and analysis stages, increasing the robustness of the results. Ultimately, this rigorous approach allows us to conclude with confidence that the prevalence of pfhrp2/3 deletions exceeds the 5% threshold — a finding with significant implications for malaria control at the national level.