Methodology

3.1 Basing recommendations on evidence

The Committee used an evidence-based medicine approach to produce these guidelines. In reality, if only the most reliable form of clinical evidence was taken into account (i.e. results of one or more randomized controlled trials with clinical endpoints), it would be impossible to formulate these guidelines. Many important aspects of clinical practice remain to be formally evaluated and very few trials with clinical endpoints are ongoing or planned. Many trials have been performed in order to obtain licensing approval for a drug. In many cases they are the only source of evidence for comparing two drug regimens. However, the designs are not ideally suited to addressing questions concerning clinical use. The most significant drawbacks of such trials are their short duration and the lack of follow-up data on patients who switch therapy. In most cases, the only available data on long-term outcomes are from routine clinical cohorts. While such cohorts are representative of routine clinical populations, the lack of randomization to different regimens means that comparisons between the outcomes of different regimens are highly susceptible to bias [78,79]. Expert opinion forms an important part of all consensus guidelines; however, this is the least valuable and robust form of evidence.

3.2 Implications for research

Unless guidelines are interpreted and applied cautiously and sensibly, valuable research initiatives that might improve standards of care will be stifled. It would be wrong to suggest that certain clinical controlled trials would be unethical if they did not conform to the guidelines, especially when these guidelines are based mainly upon expert opinion rather than more reliable evidence [80].

3.3 Use of surrogate marker data

CD4 cell counts and plasma viral load are used as markers of the effect of ART. Reduction in viral load leads to a rise in peripheral blood CD4 count, with greater rises being seen in those with greater and more sustained viral suppression [81]. Changes in these markers in response to therapy are strongly associated with clinical response [82–86]. CD4 counts measured in people on ART have been associated with a risk of AIDS defining diseases no higher than that expected in untreated individuals with similar CD4 counts [87–90]. The CD4 count is a better indicator of the immediate risk of AIDS defining diseases than the viral load in those on ART [91,92]. However, it should be remembered that CD4 count and viral load responses do not precisely reflect the expected clinical outcome and are not perfect surrogates of the clinical response [85,93,94]. This is because the drugs have other effects with clinical consequences besides those reflected in viral load and CD4 count changes. Even so, for patients with a given CD4 count and viral load, the risk of AIDS disease appears to be similar regardless of the specific antiretroviral drugs being used [95]. The relatively short length of trials designed to obtain drug approval means that, at the time of licensing, little can be known about the drugs’ long term consequences.

3.4 Issues concerning design and analysis of clinical trials

3.4. 1 Issues concerning design and analysis of clinical trials: Trial designs

As stated above, most antiretroviral drug trials are performed by pharmaceutical companies as part of their efforts to obtain licensing approval and the designs are often not ideally suited to deriving information on using the drugs in clinical practice. Besides the short duration of follow-up, their key limitation is the lack of data on outcomes in people who change from the original randomized regimen, along with a description of what those new regimens are. The results are, therefore, only clearly interpretable as long as a high proportion of participants remain on the original allocated regimens. Clinical questions about which drugs to start with or switch to require longer-term trials that continue despite changes to the original treatment. From a clinical perspective, it makes little sense to ignore what happens to patients after a specific regimen has been discontinued. Use of a given drug can affect outcomes long after it has been stopped. For example, it may select for virus resistant to drugs not yet encountered or cause toxicities which overlap with those caused by other drugs. However, interpretation of such trials is not straightforward, and account must be taken of which drugs were used subsequent to the original regimen in each arm.

The Committee generally favours entry into well-constructed trials for patients whose clinical circumstances are complex, with a number of specific instances being mentioned in these guidelines. NAM maintains a list of trials currently recruiting in the UK at www.aidsmap.com, and treatment units should work to ensure arrangements are in place to enable eligible patients to enter trials at centres within or indeed outside their clinical networks.

3.4.2 Issues concerning design and analysis of clinical trials: Viral load outcome measures

In most efficacy trials, treatments are compared in terms of viral load as defined by plasma HIV RNA. Depending on the target population, the primary outcome measure may be defined to include the achievement of viral suppression below a certain limit (usually 50 HIV RNA copies per mL) at a pre-specified time (e.g. 24 or 48 weeks after randomizations), time to viral rebound or time-weighted average change from baseline. To avoid selection bias, all enrolled patients must be included in an analysis comparing the treatments as randomized (the intention to treat principle). However, the inability to assess outcomes for some patients, leading to missing data e.g. due to patients dropout before completion of the trial, is a potential source of bias. The frequency and reasons for missing outcomes may be affected by many factors including the efficacy of treatments, toxicity and length of follow-up. Interpretation of the results of the trial is particularly problematic if a substantial number of patients drop out for reasons related to the outcome whether by design, as in many pharmaceutical industry trials where patients are withdrawn when they change their randomized treatment, or otherwise. This problem can be addressed at three levels: in the design, conduct and analysis stages of the trial. Changes in treatment during the trial must be anticipated and it is necessary to continue collecting data on all patients, even if they have switched from the original regimen, and to pre-specify the statistical methods to be used for handling missing outcomes. In general, these methods impute values for those patients who have dropped out of the trial. When the outcome is the proportion of people with viral load <50 copies/ml at a given time point, the approach almost universally adopted is to assign >50 copies/ml to all patients with missing outcome (and those who have switched from the randomized treatment, regardless of whether they remain under follow-up), known as the missing=failure (MEF) approach [91–98]. This approach to missing outcome is used in trials for drug licensing because it assigns anyone who has to stop the drug of interest as having failed and thus prevents any tendency for drugs used in a patient after the drug of interest has failed to influence the trial results. Such an approach implicitly equates failure of a regimen due to inadequate potency and/or viral drug resistance with, not only, the inability to tolerate a regimen due to pill burden, inconvenience and/or adverse effects, but also with missing assessments due to other reasons, including randomly missing visits, even though the implications of these outcomes are likely to be substantially different. This approach is often labeled conservative because it gives a minimum proportion <50 copies/mL for any given treatment group over all possible approaches. However, the primary purpose of an endpoint is to compare treatment arms and the reasons for missing outcomes may well differ between treatments. In this context, this approach is not conservative in any general sense and its indiscriminate use without consideration to its inherent limitations involves a degree of risk of bias, which could be greater than simply ignoring missing values. For these reasons, trials that are conducted for purposes of licensing a particular drug, and which treat stopping of the drug as treatment failure and ignore outcomes occurring after the drug has stopped, do not always provide the type of information that is most useful for clinical practice.

In the past, trials have generally considered whether the viral load is below 50 copies/mL or not at a given time point (e.g. 48 weeks). In recent years, the tendency has been to consider whether virologic failure (or ‘loss of virologic response’, usually defined as two consecutive values > 50 copies/mL) has occurred by a certain time point, rather than whether the viral load at the time point is < 50 copies/mL or not. In the (common) case where missing viral viral load values and switches in therapy are treated the same as values > 50 copies/mL, this approach uses a ‘time to loss of virologic response’ (TLOVR) algorithm [98]. The two approaches will give similar but not identical results, as for example patients can fulfill the definition of loss of virologic response before 48 weeks but then have a viral load value < 50 copies/mL at 48 weeks itself, without any change in regimen.

Randomization in a trial ensures balance in prognosis between the treatment arms at baseline. Inability to assess outcomes for some patients can disturb this balance and create bias in the comparison between the treatment arms. In order to avoid risk of such bias, analysis by intention to treat includes outcomes for all randomized patients. So called ‘on treatment’ analyses consider outcomes only in those still receiving the original allocated treatment. Here, the difference between assessing the proportion with viral load < 50 copies/mL at a given time point, or the proportion with viral load > 50 copies/mL by a given time point, becomes greater. In the context of an assessment of the proportion of people with viral load <50 copies/mL at a given time point, on treatment analyses make little sense because therapy has been switched in patients who experience viral load rebound during a trial. Hence, all regimens which leads to a viral load < 50 copies/mL in at least one person should lead to a value of 100%, unless there are patients who have viral load > 50 copies/mL at the time point but are yet to have their regimen switched. In contrast, an assessment of whether the viral load was > 50 copies/mL by a given time point (i.e. time to virological failure or loss of virologic response), which censors observation on patients once they have switched from the original randomized regimen may be more revealing, but is still subject to potential bias.

3.4.3 Issues concerning design and analysis of clinical trials: Non-inferiority

In contrast to superiority trials where the primary objective is to demonstrate that a new treatment regimen, or strategy, is more efficacious than a well established treatment, the aim of a non-inferiority trial is to show that there is no important loss of efficacy if the new treatment is used instead of the established reference [99]. This is particularly relevant in evaluating simplification strategies where the new treatment strategy is better than the reference treatment in aspects other than efficacy, e.g. toxicity, tolerability or cost. To demonstrate non-inferiority, large numbers of patients are usually required because of the need to exclude moderate loss of efficacy with the new treatment. The trial protocol must pre-specify a non-inferiority margin (e.g. the proportion with viral load < 50 copies per mL at 48 week, in people receiving the new treatment, is not smaller than the same proportion in the reference treatment by more than 5%). The choice of the non-inferiority margin depends on what is considered to be a clinically unimportant difference in efficacy taking into account other potential advantages of the new treatment. Stating that the response to the new treatment was not significantly different from that of the reference treatment is not evidence for non-inferiority. Graphical representations that show overlapping increased CD4 cell counts or decreased viral loads in response to therapy may hide differences in efficacy between drugs. Non-inferiority is indicated when the (95%) confidence interval for the difference between the two treatments excludes loss of efficacy greater than the pre-specified non-inferiority margin.

It is important to note that a very high standard of trial conduct (e.g. minimizing violations of entry criteria, non-adherence to allocated regimens and loss to follow-up) is more critical in non-inferiority than in superiority trials. Such deviations from the protocol would tend to bias the difference between the two treatments towards zero and thus increase the chance of erroneously concluding non-inferiority.

3.4.4 Issues concerning design and analysis of clinical trials: Cross-study comparisons: presentation of data

It is tempting to compare results of individual drug combinations assessed in different trials. Such comparisons are, however, difficult to interpret because of differences in entry criteria (particularly with respect to viral load and CD4 cell counts), methods of analysis (e.g. intention to treat versus on treatment), degrees of adherence and sensitivities of viral load assays [100].

3.4. 1 Issues concerning design and analysis of clinical trials: Trial designs

As stated above, most antiretroviral drug trials are performed by pharmaceutical companies as part of their efforts to obtain licensing approval and the designs are often not ideally suited to deriving information on using the drugs in clinical practice. Besides the short duration of follow-up, their key limitation is the lack of data on outcomes in people who change from the original randomized regimen, along with a description of what those new regimens are. The results are, therefore, only clearly interpretable as long as a high proportion of participants remain on the original allocated regimens. Clinical questions about which drugs to start with or switch to require longer-term trials that continue despite changes to the original treatment. From a clinical perspective, it makes little sense to ignore what happens to patients after a specific regimen has been discontinued. Use of a given drug can affect outcomes long after it has been stopped. For example, it may select for virus resistant to drugs not yet encountered or cause toxicities which overlap with those caused by other drugs. However, interpretation of such trials is not straightforward, and account must be taken of which drugs were used subsequent to the original regimen in each arm.

The Committee generally favours entry into well-constructed trials for patients whose clinical circumstances are complex, with a number of specific instances being mentioned in these guidelines. NAM maintains a list of trials currently recruiting in the UK at www.aidsmap.com, and treatment units should work to ensure arrangements are in place to enable eligible patients to enter trials at centres within or indeed outside their clinical networks.

3.4.2 Issues concerning design and analysis of clinical trials: Viral load outcome measures

In most efficacy trials, treatments are compared in terms of viral load as defined by plasma HIV RNA. Depending on the target population, the primary outcome measure may be defined to include the achievement of viral suppression below a certain limit (usually 50 HIV RNA copies per mL) at a pre-specified time (e.g. 24 or 48 weeks after randomizations), time to viral rebound or time-weighted average change from baseline. To avoid selection bias, all enrolled patients must be included in an analysis comparing the treatments as randomized (the intention to treat principle). However, the inability to assess outcomes for some patients, leading to missing data e.g. due to patients dropout before completion of the trial, is a potential source of bias. The frequency and reasons for missing outcomes may be affected by many factors including the efficacy of treatments, toxicity and length of follow-up. Interpretation of the results of the trial is particularly problematic if a substantial number of patients drop out for reasons related to the outcome whether by design, as in many pharmaceutical industry trials where patients are withdrawn when they change their randomized treatment, or otherwise. This problem can be addressed at three levels: in the design, conduct and analysis stages of the trial. Changes in treatment during the trial must be anticipated and it is necessary to continue collecting data on all patients, even if they have switched from the original regimen, and to pre-specify the statistical methods to be used for handling missing outcomes. In general, these methods impute values for those patients who have dropped out of the trial. When the outcome is the proportion of people with viral load <50 copies/ml at a given time point, the approach almost universally adopted is to assign >50 copies/ml to all patients with missing outcome (and those who have switched from the randomized treatment, regardless of whether they remain under follow-up), known as the missing=failure (MEF) approach [91–98]. This approach to missing outcome is used in trials for drug licensing because it assigns anyone who has to stop the drug of interest as having failed and thus prevents any tendency for drugs used in a patient after the drug of interest has failed to influence the trial results. Such an approach implicitly equates failure of a regimen due to inadequate potency and/or viral drug resistance with, not only, the inability to tolerate a regimen due to pill burden, inconvenience and/or adverse effects, but also with missing assessments due to other reasons, including randomly missing visits, even though the implications of these outcomes are likely to be substantially different. This approach is often labeled conservative because it gives a minimum proportion <50 copies/mL for any given treatment group over all possible approaches. However, the primary purpose of an endpoint is to compare treatment arms and the reasons for missing outcomes may well differ between treatments. In this context, this approach is not conservative in any general sense and its indiscriminate use without consideration to its inherent limitations involves a degree of risk of bias, which could be greater than simply ignoring missing values. For these reasons, trials that are conducted for purposes of licensing a particular drug, and which treat stopping of the drug as treatment failure and ignore outcomes occurring after the drug has stopped, do not always provide the type of information that is most useful for clinical practice.

In the past, trials have generally considered whether the viral load is below 50 copies/mL or not at a given time point (e.g. 48 weeks). In recent years, the tendency has been to consider whether virologic failure (or ‘loss of virologic response’, usually defined as two consecutive values > 50 copies/mL) has occurred by a certain time point, rather than whether the viral load at the time point is < 50 copies/mL or not. In the (common) case where missing viral viral load values and switches in therapy are treated the same as values > 50 copies/mL, this approach uses a ‘time to loss of virologic response’ (TLOVR) algorithm [98]. The two approaches will give similar but not identical results, as for example patients can fulfill the definition of loss of virologic response before 48 weeks but then have a viral load value < 50 copies/mL at 48 weeks itself, without any change in regimen.

Randomization in a trial ensures balance in prognosis between the treatment arms at baseline. Inability to assess outcomes for some patients can disturb this balance and create bias in the comparison between the treatment arms. In order to avoid risk of such bias, analysis by intention to treat includes outcomes for all randomized patients. So called ‘on treatment’ analyses consider outcomes only in those still receiving the original allocated treatment. Here, the difference between assessing the proportion with viral load < 50 copies/mL at a given time point, or the proportion with viral load > 50 copies/mL by a given time point, becomes greater. In the context of an assessment of the proportion of people with viral load <50 copies/mL at a given time point, on treatment analyses make little sense because therapy has been switched in patients who experience viral load rebound during a trial. Hence, all regimens which leads to a viral load < 50 copies/mL in at least one person should lead to a value of 100%, unless there are patients who have viral load > 50 copies/mL at the time point but are yet to have their regimen switched. In contrast, an assessment of whether the viral load was > 50 copies/mL by a given time point (i.e. time to virological failure or loss of virologic response), which censors observation on patients once they have switched from the original randomized regimen may be more revealing, but is still subject to potential bias.

3.4.3 Issues concerning design and analysis of clinical trials: Non-inferiority

In contrast to superiority trials where the primary objective is to demonstrate that a new treatment regimen, or strategy, is more efficacious than a well established treatment, the aim of a non-inferiority trial is to show that there is no important loss of efficacy if the new treatment is used instead of the established reference [99]. This is particularly relevant in evaluating simplification strategies where the new treatment strategy is better than the reference treatment in aspects other than efficacy, e.g. toxicity, tolerability or cost. To demonstrate non-inferiority, large numbers of patients are usually required because of the need to exclude moderate loss of efficacy with the new treatment. The trial protocol must pre-specify a non-inferiority margin (e.g. the proportion with viral load < 50 copies per mL at 48 week, in people receiving the new treatment, is not smaller than the same proportion in the reference treatment by more than 5%). The choice of the non-inferiority margin depends on what is considered to be a clinically unimportant difference in efficacy taking into account other potential advantages of the new treatment. Stating that the response to the new treatment was not significantly different from that of the reference treatment is not evidence for non-inferiority. Graphical representations that show overlapping increased CD4 cell counts or decreased viral loads in response to therapy may hide differences in efficacy between drugs. Non-inferiority is indicated when the (95%) confidence interval for the difference between the two treatments excludes loss of efficacy greater than the pre-specified non-inferiority margin.

It is important to note that a very high standard of trial conduct (e.g. minimizing violations of entry criteria, non-adherence to allocated regimens and loss to follow-up) is more critical in non-inferiority than in superiority trials. Such deviations from the protocol would tend to bias the difference between the two treatments towards zero and thus increase the chance of erroneously concluding non-inferiority.

3.4.4 Issues concerning design and analysis of clinical trials: Cross-study comparisons: presentation of data

It is tempting to compare results of individual drug combinations assessed in different trials. Such comparisons are, however, difficult to interpret because of differences in entry criteria (particularly with respect to viral load and CD4 cell counts), methods of analysis (e.g. intention to treat versus on treatment), degrees of adherence and sensitivities of viral load assays [100].

3.5 Adverse event reporting

Many previously unsuspected side effects of ART have been reported only after drug licensing. It is vital that prescribers report any adverse events as soon as possible so that these events are swiftly recognized. A blue-card scheme, organized by the Medicines Control Agency, the Committee for Safety of Medicines (CSM) and the Medical Researc Council (MRC), operates in the UK for reporting adverse events relating to the treatment of HIV [101].