White Paper

(aka ‘Project Waldo’)

High-quality COVID-19 screening/ surveillance

with extremely low-cost and maximal self-reliance

NOTE & Disclaimer: This whitepaper document is shared here as the ‘genesis thinking’ primarily in March-May 2020 for OpenCovidScreen and the effort to catalyze Frequent, Fast-turnaround, Cheap & Easy (#FFCE) testing to enable safe return to school & work, by helping catch occurrences before they become outbreaks. The document is no longer current, and is not an official document, or fully reflective of current thinking. But, it is historically significant and “interesting.” So, here it is! 

DRAFT – As of July 2020

Original formatted version at: bit.ly/waldotest // Feedback to: [email protected] 

(or “What if a high-quality COVID-19 viral test cost less than a latte?”)

The genesis for this project was a discussion with the administration of a ~300-student school, which was trying to answer a ‘simple question’ — with the current and expected state of the COVID-19 pandemic, “what do we need to safely resume school in September?” 

 

Some of the constraints to answering the question include: the need for sufficiently frequent and accurate testing to enable effective “test, trace, and isolate” (“TTI”), the need for simple and safe sample collection for testing, managing risk of shortages and supply chain issues for test reagents & supplies (e.g,. swabs, PPE, commercial lab operators), and the need for a very low cost per test. 

 

Unfortunately, there was nowhere near a simple answer to the ‘simple question’, and the question generalizes to many, many schools and universities, but also to small & medium-sized businesses (i.e., getting the economy functional again), and beyond to hot-spot sites like nursing homes, prisons, meat-packing plants, and much of the developing world without current access to complex/expensive testing infrastructure.

 

The goal of the proposed COVID-19 screening “Waldo Test” is to enable a low cost, effective, and realistically-implementable TTI strategy that takes inspiration from successful implementers like the country of New Zealand (which is currently among the lowest in the world in COVID-19 infections and deaths per capita):

 

“Elimination doesn’t mean zero cases, it means zero tolerance for cases. It means when a case emerges, and it will, we test, we contact trace, we isolate, and we do that every single time with the ambition that when we see COVID-19, we eliminate it. That is how we will keep our transmission rate under 1, and it is how we will keep succeeding.”

– Jacinda Ardern, Prime Minister of New Zealand, April 2020

 

There’s much more that needs to go into a full program, protocol, and policies for effective TTI, but this initiative started with an aspirational specification for a COVID-19 screening test that could be done frequently (at least weekly), and be simple and cheap enough that it could be done by nearly anyone with high-school level chemistry/biology understanding and with appropriate training in lab safety and “standard operating procedures” (“SOPs”) for this test. As a simplifying assumption, the focus of the test is screening (so could be “Research Use Only” / “RUO”), and would reflex (/refer) a positive result to an official clinical test with appropriate certification (e.g,. CLIA/CAP in the US) for clinical confirmation and any required follow-up medical action. Currently available COVID-19 testing options are sometimes bound by dogmatic “but that’s just the way it’s done” thinking; this is an attempt to start with first principles to innovate and achieve the simplest possible solution for these unprecedented times.

ASPIRATIONAL COVID-19 "WALDO TEST" SPECIFICATION (Preliminary)

Topline: Surveillance/screening that is Frequent, Fast Turn-Around, Cheap, and Easy (“FFCE”)

  1. Frequent: Repeatable ability to do COVID-19 viral testing of ~500 people/week (~200 people/day) with simple configuration; and scalable to thousands/week by replicating the configuration
  2. Fast Turn-Around: 12 hour maximum turn-around time (“TAT”) from sample to result, to enable “next morning” results
  3. Cheap: Target marginal cost of $5 per test or less, including consumables (may leverage pooling/multiplex; ideally wouldn’t require a commercial RNA extraction step/ kit); startup capital cost < $12K and can be done in 150 sq-ft
  4. Easy: Inobtrusive (/non-painful) self-collection – so saliva-based, buccal swab, or anterior nasal (“AN”) swab instead of nasopharyngeal swab
  5. “Research-use-only (RUO)” spec that is used for screening/ surveillance (i.e., any positive can be reflexed and referred to a certified clinical lab for a clinical result)
  6. 90%+ sensitivity in first 3 days of symptoms, and 99% specificity; positive & negative controls built-in for quality control (or comparable to results achieved by commercial clinical tests)
  7. Enables an approach with “self-reliance” that can be operationally run by high-school Biology or Chemistry teacher-level skill set, with trained volunteers (including students), which doesn’t require buying kits or expensive reagents (to avoid risk of supply chain issues)
  8. Has appropriate provisions for data privacy (e.g., results are coded for look-up by authorized person (e.g., school nurse or site administrator))
  9. Need demonstrable proof-of-concept by August, and “working” system/ protocol ready to train & deploy initial sites by September/October.

“Nice-to-Haves”

  1. reporting of aggregated data results by run, and over time
  2. sufficient automation in prep/loading and readout that it can be considered a ‘low complexity’ test by the FDA
  3. an “open source design” playbook that can be replicated easily by schools, nursing homes, and small businesses around the world that want to return to (/get to) safe operations with testing-based surveillance 
  4. publishable design and validation results for authors (researchers, grad school students, and maybe high school students)
  5. has a clever name that somehow ties into Ralph Waldo Emerson (and his ‘Essays on Self Reliance’ 😉

THE OPEN-SCIENCE ‘WALDO-TEST’ COMPETITION
update: launching July 28, 2020! Preregistration open on July 21, 2020 at http://xprize.org/testing

To drive broader participation to innovate and create screening test assays that meet the target COVID-19 “Waldo Test” Specification, OpenCovidScreen.org will run a competition culminating in judging the week of August xx.  It’s anticipated participants will range from graduate students/ PostDocs, to undergraduates, to high-school students. Outreach to proven labs and science mentors will ensure strong entries. Collaboration is strongly encouraged. Recognized deep subject matter experts from  academia & industry will evaluate and award prizes  in each category.

 

“WALDO TEST” VIRUS SCREENING ASSAY COMPETITIONS

In each category – First prize: $x00K; second prize: $x00K; honorable mention: $xxK each (depending on donor involvement). 

Individuals or Teams create/define “open science” assays and demonstrate analytical validation performance (i.e., sensitivity, specificity, and limit of detection (“LoD”)) in their choice of category:

 

  1. LAMP-based
  2. PCR-based 
  3. CRISPR-based 
  4. Sequencing-based 
  5. Open: bring new diagnostic screening technology or model that achieves Waldo Test specifications.  

 

Categories 1 through 4 are based on currently known approaches for SARS-CoV-2 virus testing (see FAQs below for more details on each). The fifth, “Open”, category is intended to encourage innovation in new modalities and technologies for SARS-CoV-2/COVID-19 detection. 

 

Judging will be based on: 

  • Cost per test – including all materials and reagents required
  • Performance – measured & validated sensitivity, specificity, and LoD
  • Turn-Around-Time (TAT) – within 12 hours to enable “next morning” results for TTI
  • Simplicity – defined “Standard Operating Procedures (“SOPs”) that enable 
  • Biosafety – approach to ensure virus neutralization and safe handling 
  • Innovation – going above & beyond to improve performance, efficiency, ease-of-use, consistency, open-source/low-cost automation, or other areas of innovation

 

“RETURN TO X” PROGRAM COMPETITION 

Up to five finalists awarded $xxK each. 

 

This competition evaluates programs proposed for implementation by individuals or Teams. Finalists will be recognized in back-to-school and back-to-work programs, and (depending on entries) may also be recognized in ‘hot spot’ categories (e.g., nursing home, prison, homeless community).

FREQUENTLY ASKED QUESTIONS (FAQs)

Why the name – ‘Project Waldo’? Is it an acronym for something? 

The name takes inspiration from Ralph Waldo Emerson’s ‘Essays on Self-Reliance’. It also coincidentally references “Where’s Waldo”, where you’re quickly screening for a positive result (in that case an image of Waldo, in this case the SARS-CoV-2 virus). And of course every bio project has to have a tortured acronym, so – “Widely Accessible Low-cost Diagnostic via Open Source” for COVID-19 screening. If you don’t like it, please propose a better one (e.g., Project Jacinda?)

What is the mission & philosophy of Project Waldo? 

Project Waldo’s mission is to enable “high-quality COVID-19 screening/ surveillance with extremely low-cost and maximal self-reliance”. 

The solution needs to be ~simple, distributed, global (/possible anywhere in the world). Project Waldo aims to provide an “open source” high quality COVID-19 screening test protocol, playbook for implementing, and essential training materials so that COVID-19 screening can be implemented successfully nearly anywhere. 

How do Waldo Tests fit in the landscape of other current test offerings? 

What kind of test is the Waldo Test – viral (/molecular), or antibody, or something else? 

Given the dynamics of COVID-19, the Waldo Test is a viral (/molecular) test detecting evidence of SARS-CoV-2 from the test subject, which is at peak in the range of -2 days to +7 days from initial symptoms. Antibody tests are also useful in TTI approaches, particularly for baselining who in the population has previously had COVID-19, but antibody signatures (IgG and/or IgM) are detectable in the range of +5 days to +20 days (and beyond) after initial symptoms. For the purposes of a TTI containment strategy to minimize COVID-19 in a population, frequent viral testing enabling early detection (with corresponding fast “trace & isolate”) is a more relevant mechanism. [TODO: links to COVID-19 viral load & antibody kinetics]

What are the leading options of how the test may be implemented?

As of late May 2020, the leading-contender platforms for the Waldo Test are LAMP, RT-PCR, CRISPR, or next-gen Sequencing:

RT-LAMP (“Reverse Transcription Loop-mediated isothermal Amplification“) – See inspiration from this pre-print on “Rapid Isothermal Detection of the COVID-19 Virus and a Framework for Scalable, Global Surveillance”. The pre-print is for swab-based collection and uses RNA extraction; remaining work would be to adapt for saliva self-collection, (ideally) remove the requirement for RNA extraction, and build basic input/output automation. Update Apr 29: another pre-print “SARS-CoV-2 On-the-Spot Virus Detection Directly from Patients” (tweet) confirms the potential for saliva + RT-LAMP + no separate RNA extraction step; needs some more work on optimizing sensitivity & specificity and scaling (e.g., tube strips or 96 well), but promising.  Update May 19: More LAMP isothermal inspiration from Christopher Mason’s Weill Cornell team: “Shotgun Transcriptome and Isothermal Profiling of SARS-CoV-2 Infection Reveals Unique Host Responses, Viral Diversification, and Drug Interactions.”  Update June 4: From Rabe & Cepko at Harvard: SARS-CoV-2 Detection Using an Isothermal Amplification Reaction and a Rapid, Inexpensive Protocol for Sample Inactivation and Purification

RT-PCR (“Reverse Transcription Polymerase Chain Reaction“) – PCR tests are the dominant form of clinically & commercially available testing for COVID-19 available today. They have the benefit of being relatively cheap (marginal cost of $10-15 per test), relatively fast (a few hours of time ‘on machine’ in the lab, typically resulting in next-day results at-best), and fairly accurate (high analytical sensitivity & specificity, somewhat lower real-world sensitivity due to viral load kinetics and collection technique & issues with nasopharyngeal swabs (“NPS”)). PCR machines are widely available and relatively inexpensive. There is currently high demand for PCR equipment and consumables, which are stretching supply chain capacity; it is expected the supply chain issues in North America & Europe will sort out over the next 12 months, but there is remaining risk for Fall 2020/Spring 2021 timeframe, particularly if there is a 2nd (or 3rd or more) wave of COVID-19 infections. Innovations in PCR like ChromaCode’s multiplexing/pooling approach can increase capacity and reduce consumables cost (by 4-10x) for COVID-19 testing. Most PCR-based COVID-19 testing protocols also require a RNA extraction step; it would be highly desirable to remove this requirement.

 

CRISPR (“Clustered Regularly Interspaced Short Palindromic Repeats“) – CRISPR-based SARS-CoV-2 tests are not yet commercially available, but multiple are in development. These have the potential of enabling low-cost (e.g., $10/test), fast (5-15 minute result), lab-based, point-of-care tests (benchtop device) or at-home (consumer-device) tests. While CRISPR consumer form-factor device tests have the potential to be a ‘sentinel test’ for applications like testing visitors to a location (e.g., school, hospital, restaurant, event, workplace, country-level customs/immigration), the fact that they’re not yet available in the market and that their unit cost is higher (perhaps 2-3 lattes!) means that they’ll be a stretch for repeated testing of a full population at a location (e.g., students & faculty full time at a school, all employees at a workplace, all residents & workers at a nursing home). Examples of current CRISPR-based R&D for COVID-19 testing with good performance that could be adapted for a simple, low-cost, lab-based test include this Mammoth/ UCSF collaboration: CRISPR–Cas12-based detection of SARS-CoV-2 or MIT/Broad’s ‘STOPCovid’ CRISPR assay. 

Next Gen Sequencing – [TODO: May be challenging for this application since capital and consumable cost is higher, sequencing equipment is relatively more rare, and turnaround time is likely longer than 12 hours/“next morning”. That said, there are some very interesting approaches for infrequent massively-scaled testing using barcoding & pooling, like “LAMP-Seq: Population-Scale COVID-19 Diagnostics Using a Compressed Barcode Space,” that combine LAMP and sequencing. Other complementary approaches summarized here: “Fast and accurate diagnostics from highly multiplexed sequencing assays.” Commercial offerings are expected from companies like Guardant, Illumina, Helix, and Color Genomics.]

[TODO: Other approaches, like antigen testing with lateral flow, e.g., E25bio]

The final determination of which approach is the winner for the Waldo Test will be based on demonstrable test accuracy/ performance (as measured by sensitivity & specificity; false-negative & false-positive rates), biosafety, complexity and consistent repeatability, wide availability in the supply chain for the required consumables, and low cost (equipment, supplies/consumables, and unit cost). 

Who else is using the OpenCovidScreen “FFCE” model for safe Return-to-Work/School?

As a reminder, “FFCE” is COVID-19 surveillance/screening that is Frequent, Fast Turn-Around, Cheap, and Easy. There are a growing number of organizations who are getting close:

  • The White House of the United States – Does Frequent (daily) and Fast Turn-Around (point of care/ 15 minute) testing based on the Abbott IDNow device. It may not be very accurate and each result is expensive (and nasopharyngeal swab based), but the White House isn’t very science/data-driven at the moment, or particularly cost sensitive (ever).
  • The Broad Institute – Announced their ‘Return to Work’ plans on May 28, which include Frequent testing (every 4 days), Fast Turn-Around (“next morning” results), (Relatively) Cheap (FDA EUA-approved LAMP assay developed by Color Genomics), and Easy (self-collect AN swab). The Broad Team always seemed smart, and this reconfirms! A complication is that the service will serve the Broad itself and a few companies in the Boston area. Commercial versions of the test will still likely be priced at $75/result.  We need their model for the rest of the world, at a price everyone can afford.  
  • Major League Baseball (MLB) – Announced on May 15, MLB announced  it will be doing saliva-based PCR testing of all personnel “several times a week”, using the FDA EUA-approved Rutgers test/protocol, which will be performed by a central lab in Utah, which promises “24 hour turnaround time”.  Given logistics of getting samples to a central lab, it’s unlikely they’ll have “next morning” results that would be ideal for a test-trace-isolate program. They’re also not likely to be price sensitive on cost per test, relative to player salaries and sports revenues.
Table 1: Summary of “FFCE” properties for announced ‘Return to X’ programs to date
Frequent  Fast Turn- Around  Cheap  Easy 
Return to X program at least weekly, w/ sub-sampling “next morning” results (or faster) “less than the price of a latte” per result self collection: saliva or AN swab
White House ✔️ ✔️
Broad Institute ✔️ ✔️ ✔️
Major League Baseball (MLB) ✔️ ✔️
How will samples for the Waldo test be safely collected?  Nasopharyngeal swabs (“NPS”) are the primary collection mechanism for commercial & clinical COVID-19 testing to date. A material contributor to the gap between lab-only ‘analytical sensitivity’ and ‘real-world sensitivity’ (where you see lower sensitivity and higher false-negatives) are from issues related to nasopharyngeal swab collection and processing. In addition NPS collection is not a pleasant process (sometimes compared to ‘tickling your brain’) that few people want to repeat with high frequency – so would present a compliance challenge and pushback for frequent testing. Nasal self-collection kits were recently approved via “EUA” (Emergency Use Authorization) by the FDA for a LabCorp COVID-19 test, but this hasn’t yet been scaled and is single-supplier. [TODO: links to china study on NPS sensitivity issues and transport medium NPS issues with Abbott point-of-care tests]. Given these issues, it would be strongly preferred to be able to do saliva self-collection based testing, with validation of comparable test performance. The FDA also recently issued an EUA for a saliva-based clinical test (developed at Rutgers), and a Yale study pre-print confirmed “Saliva is more sensitive for SARS-CoV-2 detection in COVID-19 patients than nasopharyngeal swabs.”  Depending on the saliva self-collection approach, it may also be possible to use a tube with a RNA-preserving buffer that is SARS-CoV-2 virus-neutralizing (like used by Ancestry.com’s DNA test kit), which would further minimize biosafety risk. Update 5/10: Spectrum Solutions’ SDNA-1000 FDA EUA approved tubes are FDA EUA approved.  Anterior Nasal (AN) swab or buccal swab self-collection can be acceptable alternatives or complements to saliva-based collection, per Patient-collected tongue, nasal, and mid-turbinate swabs for SARS-CoV-2 yield equivalent sensitivity to health care worker collected nasopharyngeal swabs..  What about biosafety?  For processing of the test, the lab performing Waldo tests should adhere to “BSL-2” level biosafety guidelines and procedures for ‘routine COVID-19 diagnostic testing’ (see CDC’s Laboratory Biosafety Guidelines for Handling and Processing Specimens Associated with Coronavirus Disease 2019 (COVID-19)). How frequently should participants be tested (/sampled)?   The frequency of testing depends on the specific situation’s tolerance for scope of infectious outbreak, and tradeoffs on cost and implementation process complexity. Given exposure and viral load kinetics, a case can be made for testing a given subject every ~3 days. Even at the target low unit cost of the Waldo test, this may be cost or operationally prohibitive. Weekly testing of the population may provide a balance of cost and simplicity of operational rhythm (e.g., telling a subject their ‘test day’ is every Wednesday), and could be done in a staggered approach that does portions of the population at regular intervals (e.g., ⅓ each on Monday/ Wednesday/ Friday; ⅕ each on M/T/W/Th/F; ½ each on Monday/ Thursday).  A longer testing interval (e.g., weekly, bi-weekly, monthly) substantially increases the risk of high rates of infectious outbreak between testings, which means much greater impact on number of people infected, and scope of tracing and quarantining those exposed to cases, given the high rate of infectiousness of COVID-19 (e.g., see high rates of infection in environments currently without frequent testing like nursing homes, prisons, cruise ships, and Singapore guest-worker dormitories).  Updates on frequency modeling 7/2/20:  Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance These results demonstrate that effective surveillance, including time to first detection and outbreak control, depends largely on frequency of testing and the speed of reporting, and is only marginally improved by high test sensitivity. We therefore conclude that surveillance should prioritize accessibility, frequency, and sample-to-answer time; analytical limits of detection should be secondary. Announced return-to-work/school programs with frequencies: .  Broad Institute Return to Lab program: every 4 days (tweet) .  Colby College, Bowdoin, Amherst: 2x/week (tweet) .  Harvard College: 2x/week (for students on campus; tweet) What are mechanisms to make the test cheaper and faster (while maintaining test accuracy)? Do you have to test everyone everytime, or can you do sub-sampling? [TODO: Say more on pooling/multiplexing; see above for ChromaCode PCR or LAMP-Seq examples. We will also examine more closely algorithmic pooling/sampling approaches as suggested in A Note on Double Pooling Tests. There are some examples of possible sub-sampling earlier in the whitepaper that are operationally simple and provide appropriate statistical coverage to be very effective.]  Should people that had COVID-19 and are now recovered and ~immune do the Waldo screening test as well?    While “immunity” for patients who have had COVID-19 and recovered is not yet conclusively proven, it is expected that the vast majority of people who have recovered from COVID-19 will have some amount of temporal immunity (likely months or quarters), and of sufficient length to be epidemiologically significant. Success of vaccine strategies is also dependent on this assumption being valid. For a summary of indicators that provide optimism on the case for temporal immunity, see this thread.  While ~immune patients could be excluded from testing in a population, given the low cost per test, it may be operationally simpler to just include all members of a population (including recovered patients) for operational simplicity, avoidance of complexity of tracking/managing different subpopulations, and privacy (i.e., unintentionally disclosing prior testing status by having subpopulations with different requirements/behaviors).  What about privacy? [TODO: More on result coding to protect privacy of any given test result, and low-cost/simplicity avoiding differentiated subpopulations resulting in unintentional disclosure of prior testing status.] Can Project Waldo be used for anything else?  Yes. The approach, philosophy, technology, awareness, education value, infrastructure and equipment deployed will be of benefit for other future viral conditions & pandemics – but the focus now is on crushing COVID-19. We should break the reactive cycle and build  resiliency into our system from the bottom up to prepare for the next threats to come. 2001: anthrax 2003: SARS 2009: H1N1 2014: Ebola 2015: Zika 2020: Covid-19 20xx:  ??? 20xx:  ??? 20xx:  ??? What are the underlying assumptions about the near future of the COVID-19 pandemic?
  1. An effective and widely available vaccine will not be available before September 2020 (and ~50% possibility by September 2021)
  2. An effective and widely available ‘silver bullet’ therapeutic/treatment without significant side effects/risks will not be available before September 2020
  3. Absent an effective vaccine or silver bullet therapy, there is nothing that will change the dynamics of COVID-19 infections and need for management; we are far away from ‘herd immunity’ and the societal cost of getting there unmanaged is very high
  4. Risk of need for hospitalization and severe-critical disease is heavily weighted and risk conditions (e.g., immune-compromised conditions, diabetes, obesity), so part of the goal of this is protecting at-risk groups within mixed populations (e.g., teachers & administrators at a school, 50+ age workers in a workplace, residents and healthcare workers at a nursing home)
  5. It is impractical/unrealistic to do “complete isolation” of any sizable population for an extended period of time
  6. The vast majority of recovered COVID-19 cases will have some epidemiologically significant amount of temporal immunity
  7. [TODO: more…]
Beyond Project Waldo, what else is needed for an effective COVID-19 TTI containment program?  [TODO: Probably needs to be its own separate whitepaper on combination of policy, protocol, actions, etc needed. David Brailer’s situation-specific draft paper (confidential) is a solid starting outline. Digital tracing (enabled by the Apple+Google collaboration and health certification solutions like CVKey.org (a privacy-protecting approach for health certification) will be a key part of the overall program.] What is OpenCovidScreen.org?  OpenCovidScreen is a registered non-profit organization founded to accelerate scalable, low-cost testing solutions and provide education and support for test-enabled return-to-school, return-to-work, and hot-spot protection programs. Application for 501c3 public charity status is in progress. 501c3 Fiscal Sponsor in place with Open Collective; any donations are tax-deductible.

DRAFT – As of July 22, 2020

Original formatted version at: bit.ly/waldotest // Feedback to: [email protected] 

(or “What if a high-quality COVID-19 viral test cost less than a latte?”)