Journal of Vaccines & Immunization
An International Peer-Reviewed Open Access Journal
ISSN 2053-1273
- Download PDF
- |
- Download Citation
- |
- Email a Colleague
- |
- Share:
-
- Tweet
-
Journal of Vaccines & Immunization
Volume 2, Issue 1, May 2014, Pages 4–6
Short report Open Access
An evaluation of the indirect cohort method to estimate the effectiveness of the pneumococcal polysaccharide vaccine
- 1 Child Health Division, Menzies School of Health Research, Charles Darwin University, Northern Territory, Australia
- 2 Menzies School of Health Research, Charles Darwin University, Northern Territory, Australia
*Corresponding author: Sarah A Moberley, Post-Doctoral Fellow, Child Health Division, Menzies School of Health Research, Charles Darwin University, Northern Territory, PO Box 41096, Casuarina, NT, 0811, Australia. Tel.: +61 414992045. E-mail: moberley@gmail.com
Received 14 January 2014 Revised 10 April 2014 Accepted 18 April 2014 Published 25 April 2014
DOI: http://dx.doi.org/10.14312/2053-1273.2014-1
Copyright: © 2014 Moberley SA, et al. Published by NobleResearch Publishers. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited
AbstractTop
We examined the validity of the indirect cohort method as a rapid assessment tool to estimate pneumococcal polysaccharide vaccine effectiveness (VE). Using evidence from published clinical trials, we reviewed the primary assumption about the appropriateness of the control group underpinning the indirect cohort method, which is that the risk of non-vaccine type invasive pneumococcal disease is equal for vaccinated and unvaccinated participants. We found an absence of evidence to support the non-differential risk assumption for non-vaccine type invasive pneumococcal disease occurring among clinical trial participants. In those instances where the design has been utilised, we also note that these typically rely on very small numbers of non-vaccine type invasive pneumococcal disease adding to the concerns as an unreliable comparator. We do not consider the indirect cohort method to be a valid tool for rapid assessment of pneumococcal polysaccharide VE.
Keywords: indirect cohort method; pneumococcal polysaccharide vaccine; vaccine effectiveness; pneumococcal disease
BackgroundTop
Post-licensure studies are a critical component of the evaluation and monitoring of vaccine programs, though the validity of such studies are often called into question due to concerns of bias and confounding. In light of the current controversies surrounding the recommendations for the use of 23-valent pneumococcal polysaccharide vaccine (PPV), it is timely to reassess the validity of the indirect cohort method [1]. Adapted from the case-control methodology, the indirect cohort method is a rapid assessment tool designed specifically to estimate the effectiveness of PPVs. We focus particularly on the appropriateness of the control group.
Features of IC method
Broome et al. [1] first described the indirect cohort method in 1980, in response
to concerns surrounding potentially reduced efficacy of the 14-valent PPV in adults
with underlying chronic illness. The premise of the method was that the vaccine
would be expected to be effective in preventing disease caused by serotypes that
are contained in the vaccine, but to have no effect on the risk of disease due to
other pneumococcal serotype (excluding serologically related serotypes).
Under Broome’s method, individuals diagnosed with invasive pneumococcal disease (IPD), that is where Streptococcus pneumoniae was isolated from a normally sterile site, were classified as cases if they were known to be due to a serotype contained in the vaccine (VT IPD) and controls if they were caused by a non-vaccine serotype (non-VT IPD). As a modified case-control methodology, the indirect cohort method compares the proportion of case subjects who are vaccinated against the proportion of control subjects who are vaccinated, that is, a comparison of the risk of vaccination between the two groups. It then follows that for an effective vaccine, the cases of VT IPD should be less likely to be vaccinated than the controls (non-VT IPD). The control group in this instance was intended to represent the population risk of vaccination.
Validity of comparator
In order for the control group to be a valid indicator of the population risk of
vaccination, the key assumption of the indirect cohort method is that the risk of
non-VT IPD is equal for both the vaccinated and unvaccinated individuals. That is,
being vaccinated does not alter an individual’s risk of acquiring non-VT disease.
If this assumption is valid and provided there were no systematic differences in
the ascertainment of vaccination status, then individuals with non-VT IPD are an
attractive choice as controls as they likely represent the same population from
which the cases of VT IPD arose (care-seeking behaviour and blood culture taking
behaviours also likely to be similar). However, if the assumption (risk of non-VT
IPD equal between vaccinated and non-vaccinated population) does not hold, then
the estimate of VE will be biased. For example, if the risk of non-VT IPD was differentially
higher in vaccinated individuals, as could occur in the presence of serotype replacement,
the indirect cohort method would exaggerate VE. In contrast, if the risk of non-VT
IPD was lower in vaccinated individuals, as could occur if vaccination afforded
any protection against non-VT disease, then the indirect cohort method would underestimate
VE.
Original evidence base
Although non-VT IPD is the comparator for the indirect cohort method, it was non-VT
pneumococcal pneumonia (a much less specific outcome) that was used by Broome et
al. [1] as the basis for comparison of risk between vaccinated and unvaccinated
individuals. The outcome data were from three early trials [2-3], where presumptive
non-vaccine type pneumonia attributed the causative serotype from a specimen collected
from a non-sterile site where the pneumococcus could have colonised without causing
disease. Although a seminal paper of the time, MacLeod et al. [3] diagnosed pneumonia
only by clinical signs. In total, the three studies considered by Broome et al.
[1] provided evidence of equal risk of presumptive non-VT pneumonia between vaccinated
and unvaccinated individuals but not evidence of equal risk of non-VT IPD. An equal
risk of presumptive non-VT pneumonia is perhaps not surprising given the most recent
meta-analyses also suggest there is evidence of equal risk of all-cause pneumonia
between vaccinated and unvaccinated individuals (i.e. no evidence of benefit against
pneumonia) [5, 6].
Re-assessment of evidence of equal risk of non-VT IPD in vaccinated and unvaccinated
individuals
We attempted to test the hypothesis that the assumed risk of non-VT IPD was equal
among vaccinated and unvaccinated individuals by utilising the data from the RCTs
included in the most recent update of the Cochrane systematic review of pneumococcal
polysaccharide vaccine in adults [7]. However, only one included study reported
data on culture confirmed non-VT IPD [4], with the remaining studies either not
reporting serotype specific data, or having no cases of non-VT IPD [8-10].
The one study contributing data on non-VT IPD was a report from three pooled studies conducted by Austrian et al. [4] amongst South African miners in the 1970s. One study used a 6-valent PPV, the other two studies used a 13-valent PPV. In total, there were 22 cases of non-VT IPD amongst 3943 vaccinated participants compared with 63 events amongst 8024 unvaccinated participants. The rate of non-VT IPD was 5.6/1000 in vaccinated participants (95%CI 3.5 to 8.4) compared with 7.9/1000 unvaccinated participants (95%CI 6.0 to 10.0, p=0.17). Given we have only one study able to contribute outcomes for non-VT IPD, we conclude that there is an absence of evidence to support the primary assumption that the risk of non-VT IPD is equal in the vaccinated and the unvaccinated. Moreover, there were no RCTs of 23vPPV that contributed data on non-VT IPD.
Studies utilising indirect cohort method
A review of studies utilising the indirect cohort method to assess the effectiveness
of the 23-valent pneumococcal polysaccharide vaccine indicates an obvious limitation
in the use of non-VT IPD as the comparator, namely, small numbers (Table 1). Few cases
of non-VT IPD results in a loss of power to detect an effect of vaccination. Broome
et al. [1] had stated that ‘the estimate does depend on a similar proportion of
vaccine type and non-vaccine type disease in unvaccinated populations’, which would
have been a realistic assumption at the time that the 14-valent vaccine formulation
was available. Although not a component of the formula for vaccine effectiveness
under the indirect cohort method, when there are few available subjects within the
comparator group this limits the power of any individual study and increases the
potential risk of bias.
IPD episodes | Excluded cases* (% cases) | non-VT IPD cases (% known serotypes) | VE (95% CI) | |
Broome [1] | 427 | Not reported | 87 (20%) | 36% (<0 to 77) |
Forrester [11] | 110 | 21 (19%) | 28 (31%) | -21% (-221 to 55) |
Butler [12] | 4624 | 1787 (39%) | 421 (15%) | 57% (45 to 66) |
Benin [13] | 606 | 294 (49%) | 34 (11%) | 35% (-33 to 69) |
Andrews [14] | 115 | 17 (15%) | 6 (6%) | 79% (-14 to 96) |
Singleton [15] | 394 | 94 (24%) | 19 (6%) | 75% (27 to 91) |
Bliss [16] | 120 | 28 (23%) | 21 (23%) | 68% (3 to 90) |
Mooney [17] | 170 | 61 (36%) | 3 (3%) | 51% (-278 to 94) |
Abbreviations: *=Reasons for exclusions include no medical records, no vaccination records, or specimen not serotyped.
Although estimates of vaccine effectiveness derived from studies utilising the indirect cohort method (Table) have been similar to estimates from other case-control studies and from some meta-analyses [6], of itself, this does not provide a theoretical basis for further use of the method.
ConclusionsTop
A novel method of its time, we do not support on going use of the indirect method for assessment of vaccine effectiveness. Our findings highlight the lack of evidence to support the methodological basis that the risk of non-VT IPD in vaccinated and unvaccinated individuals is equal. Moreover, the very small numbers of non-VT IPD available in most published studies must cast doubt on the reliability of the assumption that the comparison group adequately reflects the population from which cases arise. In our view, these are important limitations of the indirect cohort method as a rapid assessment tool, and therefore do not advocate future use.
Conflict of interest
The authors wish to express that they have no conflict of interest.
ReferencesTop
[1] Broome CV, Facklam RR, Fraser DW (1980) Pneumococcal disease after pneumococcal vaccination: an alternative method to estimate the efficacy of pneumococcal vaccine. N Engl J Med 303:549–552. Article Pubmed
[2] Ekwurzel G (1928) Studies on immunizing substances in pneumococci. VIII. Report on field tests to determine the prophylactic value of a pneumococcus antigen. Public Health Report 53:1877–1893.
[3] Macleod CM, Hodges RG, Heidelberger M, Bernhard WG (1945) prevention of pneumococcal pneumonia by immunization with specific capsular polysaccharides. J Exp Med 82:445–465. Article Pubmed
[4] Austrian R (1976) Vaccines of pneumococcal capsular polysaccharides and the prevention of pneumococcal pneumonia, in the role of immunological factors in infections, allergic and autoimmune processes, BE. Beers RF Jr, New York: Raven Press 79–89.
[5] Huss A, Scott P, Stuck AE, Trotter C, Egger M (2009) Efficacy of pneumococcal vaccination in adults: a meta-analysis. CMAJ 180:48–58. Article Pubmed
[6] Moberley S, Holden J, Tatham DP, Andrews RM (2013) Vaccines for preventing pneumococcal infection in adults. Cochrane Database of Systematic Reviews. Art. No.: CD000422. DOI: 10.1002/14651858.CD000422.pub3.
[7] Fluarix Product Information. Available on 16 November 2012. Article
[8] Ortqvist A, Hedlund J, Burman LA, Elbel E, Höfer M, et al. (1998) Randomised trial of 23-valent pneumococcal capsular polysaccharide vaccine in prevention of pneumonia in middle-aged and elderly people. Swedish Pneumococcal Vaccination Study Group. Lancet 351:399–403. Article Pubmed
[9] Riley ID, Tarr PI, Andrews M, Pfeiffer M, Howard R, et al. (1977) Immunisation with a polyvalent pneumococcal vaccine. Reduction of adult respiratory mortality in a New Guinea Highlands community. Lancet 1:1338–1341. Pubmed
[10] Simberkoff MS, Cross AP, Al-Ibrahim M, Baltch AL, Geiseler PJ, et al. (1986) Efficacy of pneumococcal vaccine in high-risk patients. Results of a Veterans Administration Cooperative Study. N Engl J Med 315:1318–1327. Article Pubmed
[11] Forrester HL, Jahnigen DW, LaForce FM (1987) Inefficacy of pneumococcal vaccine in a high-risk population. Am J Med 83:425–430. Article Pubmed
[12] Butler JC, Breiman RF, Campbell JF, Lipman HB, Broome CV, et al. (1993) Pneumococcal polysaccharide vaccine efficacy. An evaluation of current recommendations. JAMA 270:1826–1831. Article Pubmed
[13] Benin AL, O'Brien KL, Watt JP, Reid R, Zell ER, et al. (2003) Effectiveness of the 23-valent polysaccharide vaccine against invasive pneumococcal disease in Navajo adults. J Infect Dis 188:81–89. Article Pubmed
[14] Andrews RM, Counahan ML, Hogg GG, McIntyre PB (2004) Effectiveness of a publicly funded pneumococcal vaccination program against invasive pneumococcal disease among the elderly in Victoria, Australia. Vaccine 23:132–138. Article Pubmed
[15] Singleton RJ, Butler JC, Bulkow LR, Hurlburt D, O'Brien KL, et al. (2007) Invasive pneumococcal disease epidemiology and effectiveness of 23-valent pneumococcal polysaccharide vaccine in Alaska native adults. Vaccine 25:2288–2295. Article Pubmed
[16] Bliss SJ, Larzalere-Hinton F, Lacapa R, Eagle KR, Frizzell F, et al. (2008) Arch Intern Med 168:749–755. Article Pubmed
[17] Mooney JD, Weir A, McMenamin J, Ritchie LD, Macfarlane TV, et al. (2008) The impact and effectiveness of pneumococcal vaccination in Scotland for those aged 65 and over during winter 2003/2004. BMC Infect Dis 8:53. Article Pubmed