Understand how interventions are chosen

Summary of Search, Selection and Inclusion Process and Criteria

Overview

The steps used to search, select, review and annotate the content included in the Canadian Best Practices Portal are listed below. For more details about each step, click on the hyperlink at the end of each step.

Step 1: Literature and Collection search by priority topic

During the initial review stage, the search team identified interventions from documents such as collections written by health promotion non-profit organizations or government and research agencies, or from review or systematic review publications, or from those which appear in scientific review databases. To be selected to go to step 2, any intervention or collection had to be collated in the past 20 years and address health promotion and chronic disease prevention (primary and secondary). [Click here for more on Step 1.]

Step 2: Assess quality of evaluation or study design

Once collections were identified, they were then reviewed with respect to the quality of the research or evaluation design. Collections had to show their inclusion/exclusion criteria and the rigour of their study design. Interventions that may be cross-referenced in collections or identified through literature database searches were also reviewed to ensure that they were evaluated using rigorous study designs, that evidence of positive results existed, and that sufficient information was present for annotation. It was particularly important to identify which interventions had to be excluded because they did not use a community-based or population health approach, they were focused on individual behaviour change or clinical trials, they did not have an evaluation, or they were single studies or pilots. [Click here for more on Step 2.]

Step 3: Search for additional information on selected individual interventions

If an intervention appeared to have yielded significant outcomes and was based in a population health approach, the team then searched for further information on that particular approach. Using the internet and bibliographic databases such as Medline, CINAHL and ERIC, the team located the original source which described the intervention, and searched for evidence of both evaluation (process and outcome) and replication. When the original source could not be found, a combination of sources that implemented/tested/evaluated the intervention and in some cases adapted it, was used. To be considered, each intervention needed positive support from a published systematic review.

Locating French language interventions proved to be challenging as the strategy outlined above provided very few interventions. The team searched the internet and bibliographic databases in an attempt to find more French language materials. Certain topic areas also yielded little in terms of interventions using the above criteria. In those cases, key informants were consulted to identify possible candidates, and the grey literature was reviewed. [Click here for more on Step 3.]

Step 4: Expert Review using Inclusion Criteria

Interventions which met the criteria of positive outcomes, replication in a second context and inclusion in a systematic review were forwarded to a team of subject experts for additional review. Only if it met the expectations of the subject experts, was it included on the Portal.

Expert Reviewers used a set of questions to apply inclusion criteria to the interventions that met the above selection criteria, often requiring detailed review of the evaluation studies and related reports. These criteria included whether the intervention was relevant to disease prevention, health promotion or population health, whether it had positive outcomes, whether it came from a credible source and whether it was free of commercial interests. [Click here for more on Step 4.]

Step 5: Prioritize Selected Interventions for Annotation

After the expert review, members of the team selected certain interventions for annotation using the Annotation Template. This selection was based on whether the intervention was highly documented, whether it had been or could be replicated, publication within the last 10 years, and the strength of the outcomes. Once the interventions were organized in an order of priority, they were annotated and entered on the Portal in that order. [Click here for more on Step 5.]

Step 6: Select Resources

In support of the interventions, resources were also selected for the Canadian Best Practices Portal. This was a limited selection from this team because other members of PHAC were doing a more systematic identification. The team identified resources that were often mentioned by the intervention itself or in the collection of interventions. Resources were selected for the Portal if they were current (within the last 10 years), relevant to disease prevention and health promotion, a match to the Portal topics, suited to a range of practitioners and policy-makers, produced by a credible authority, clear about what is fact and opinion, usable and accessible. [Click here for more on Step 6.]