Population Services International

Company or Consultant for an Independent Evaluation of the OTSS+ Approach

Posted Date 2 months ago(3/22/2021 10:06 AM)
Job ID
# of Openings
Position Location: City
Washington, DC
Washington, DC
Position Location: Country
United States









Annex B.1 Budget Template

Annex B.2 Budget Narrative Template






Independent Evaluation of the OTSS+ Approach to Quality Assurance of Service Delivery




PSI is a leading global health organization with programs targeting malaria, child survival, HIV and reproductive health. Working in partnership within the public and private sectors, and harnessing the power of the markets, PSI provides life-saving products, clinical services and behavior change communications that empower the world's most vulnerable populations to lead healthier lives. PSI's core values are a belief in markets and market mechanisms to contribute to sustained improvements in the lives of the poor; results and a strong focus on measurement; speed and efficiency with a predisposition to action and an aversion to bureaucracy; decentralization and empowering our staff at the local level; and a long-term commitment to the people we serve. PSI has programs in 67 countries. For more information, please visit www.psi.org


The US President’s Malaria Initiative Impact Malaria Project (IM) is PMI’s flagship global malaria service delivery project, helping to improve the quality and increase the delivery of malaria prevention, diagnosis, and treatment services in up to 27 countries around the world. IM is a USAID contract hosted and led by PSI as the prime contractor. PMI Impact Malaria works with national malaria control programs to help tackle these challenges by:

  • Closing the gaps in malaria service delivery to get the right medicine, with the correct diagnosis, to patients in need, in the timeliest manner.
  • Unlocking the potential of key drug-based approaches, by helping countries to introduce, implement, and scale-up proven innovations to move countries forward in their elimination efforts, according to each country’s unique malaria situation.
  • Strengthening malaria health systems and the use of data for decision-making to link operational research and country-led dialogue with global technical leadership for the means of accelerating service delivery improvements and advancing key learnings.

The IM team is a results-focused, multidisciplinary and multicultural team, based out of Washington DC, working through PSI country offices as well as through our partners Jhpiego, MCDI and UCSF.


The Outreach Training and Supportive Supervision Plus (OTSS+) Approach


IM supports, as one of its primary interventions, the OTSS+ approach, which was originally developed through the PMI Improving Malaria Diagnosis Project (IMaD) as a quality assurance (QA) approach for strengthening laboratory diagnosis. The approach builds on lessons from adult learning, de-emphasizing classroom training and instead prioritizing regular, structured, competency-based supervision, coupled with on-the-job training, troubleshooting, and real-time feedback. The OTSS approach was expanded to provide similar QA support for outpatient malaria case management services during the follow-on MalariaCare Project. A digital tool allowing supervision data to be collected on tablets was introduced during the MalariaCare Project to streamline data entry, analysis, and feedback.


IM has built on the tools and lessons learned from its two predecessor projects, adding modules to assess the quality of severe malaria management and prevention of malaria in pregnancy (MIP). IM also has moved to expand digital data collection, using an enhanced version of the digital OTSS tool, and exploited the capacity of digital data systems to provide immediate analysis and feedback on performance to health workers in targeted facilities. Digital reporting also enables real-time uploading of data into IM’s Data Hub, enabling more rapid data compiling and analysis, real-time updating of country and project dashboards, and facilitates cross-country analyses. With these innovations, IM has renamed this strengthened approach as OTSS+.


Figure 1. IM-supported countries conducting OTSS+

Figure 1. IM-supported countries conducting OTSS+

Both the IMaD and MalariaCare Projects produced publications that documented evidence of improved competency and performance on diagnostic testing and clinical case management for malaria. IMaD commissioned an assessment of the OTSS approach on laboratory performance (Keating, et al, Trans Royal Soc Trop Med Hyg, Oct 2014), which also demonstrated good competence in malaria microscopy in supported facilities. The MalariaCare Project produced a series of reports in the American Journal of Tropical Medicine and Hygiene on various aspects of its quality assurance activities (Am J Trop Med Hyg. 2019 Apr;100(4)).


After more than a decade of support for the OTSS approach, and with a broadened package of support that includes clinical and diagnostic services and expansion to include prevention of malaria in pregnancy, an independent evaluation to validate the effectiveness of the OTSS+ approach, to document lessons learned and best practices, and to identify areas for improvement and potential streamlining would be an important contribution to global efforts to achieve universal access to high-quality malaria service delivery.


A recent series of workshops outlined a Learning Agenda for IM, including a series of questions and topics that related to the OTSS+ approach:

  • Do the fundamental elements of the OTSS+ approach achieve their intended effect (for malaria case management and MIP)?
    • What are the contributing success factors?
    • What aspects of OTSS+ provide the most benefit, to what areas?
  • Can lessons learned from countries with histories/staged approach of OTSS be applied to newer OTSS+ countries?
    • Can the approach be reviewed retrospectively, considering factors such as prioritization and geography?
    • What is the impact of multiple OTSS+ visits in countries with a history of ongoing OTSS+ and what is the impact of the initial visits in countries initiating OTSS for the first time?
  • Are there more effective or efficient approaches than the current approach to OTSS+?
    • What are the essential components of the OTSS+ package and which components can be dropped or streamlined?
    • Would a lighter OTSS+ approach achieve similar effect?
    • Can virtual or distance-based QA approaches be implemented effectively? If so, are these approaches scalable? How can this be monitored?
    • What are the implications for sustainability, continuity, and institutionalization of OTSS+?
  • Does OTSS+ improve data reporting completeness and quality at the facility level?
    • What are the factors that lead to this improvement?
  • Is OTSS+ facilitating timely data use? If so, at what level and by whom? Does electronic data collection and availability facilitate data use?
  • How has OTSS+ been affected by the introduction of digital tools, such as HNQIS?
    • Does electronic data collection and availability facilitate data use?
    • Has the digital tool facilitated obtaining information on the performance of health workers and health facilities? If so, how?
    • Has the digital tool enabled/facilitated improved allocation of resources? If so, how?
  • Is there a way to use OTSS+ data to try to attribute causes of poor quality of care?


Scope of Activities:

IM is seeking a consultant, consulting team, or organization to design and implement an independent evaluation of the OTSS+ approach. The objectives of this evaluation are:

  1. To determine whether current and past OTSS interventions improved the quality of malaria service delivery (including malaria laboratory and clinical diagnosis, management of uncomplicated and severe febrile illness, and prevention and treatment of malaria in pregnancy)
  2. To identify success factors for the OTSS+ approach, including what components of this approach have had the greatest positive effect on quality
  3. To assess whether the introduction of digital data capture and reporting, and real-time data analysis and feedback resulted in tangible improvements in quality and efficiency of service delivery, or increased utilization of data for decision-making.
  4. To identify direct or indirect evidence, and contributing determinants, of the sustainability and/or institutionalization of the OTSS+ approach by Ministries of Health
  5. To identify areas for improvement or streamlining of the OTSS+ approach


The evaluation should include at least these three components:

  1. Review of published and unpublished documents
  2. Key informant interviews
  3. Analysis of available data, including data collected during OTSS+ visits and historical routine HMIS data

The evaluation should include in-depth evaluations in four countries, which will be determined at a stakeholder’s workshop after the contract is awarded. For the proposal and budget preparation, please use the following countries: Ghana, Niger, Kenya, and Zambia. It is likely that some, but not all of these countries will be included as part of the in-depth evaluation.


Bidders may also propose additional evaluation components if they can be conducted within the available budget.


Impact Malaria will facilitate access to available documentation and OTSS+ data, and assist with identification and linkages with key informants.



The selected applicant will:

  • Develop an evaluation methodology that is responsive to the objectives and methodology outlined above, in consultation with IM and PMI
  • Implement the evaluation, including any required field work and data collection and analysis
  • Prepare a draft report of the evaluation, to be presented to IM and PMI
  • Revise and finalize the evaluation report incorporating feedback from IM and PMI
  • Prepare a manuscript for peer-review publication and a short learning brief describing the key lessons learned from the evaluation



  • Final evaluation methodology and data collection tools, approved by IM and PMI
  • Draft evaluation report
  • Facilitation of a workshop hosted by IM/PMI to review evaluation findings, to include comprehensive presentation of evaluation findings.
  • Short technical or learning brief describing the key lessons learned from the evaluation
  • Final evaluation report reflecting all feedback provided from IM and PMI
  • Final data set
  • A manuscript for publication in a peer-reviewed journal highlighting key findings from the evaluation


It is anticipated that this evaluation will take approximately six months from completion of the subcontract until submission of the draft evaluation report, plus an additional three months for finalization of the report and production of a draft manuscript for publication.


Preferred Qualifications:

  • MD or PhD in public health, epidemiology, or related field of evaluation lead
  • Demonstrated experience designing and conducting program evaluations, with experience in both qualitative and quantitative research methods (Sample of previous evaluation report should be provided with application)
  • Knowledge of malaria control in developing country settings, particularly in sub-Saharan Africa
  • A minimum of 10 years of professional experience in international public health, with a focus on sub-Saharan Africa preferred
  • Proven track record working with key stakeholders in government and non-governmental organizations
  • Proven ability to produce reports and manuscripts that convert complex findings into an easily understandable and digestible format
  • Ability to analyze and synthesize information from diverse sources and develop conclusions and recommendations that reflect the information gathered
  • Fluency in reading and writing in English required, fluency in French preferred
  • Experience with USAID/PMI-funded projects preferred
  • Preference for small businesses.



Annex B.1 Budget Template

Annex B.2 Budget Narrative Template




Sorry the Share function is not working properly at this moment. Please refresh the page and try again later.
Share on your newsfeed

Connect With Us!

Not ready to apply? Connect with us for general consideration.