Request for Information (RFI) 23-001


NESTcc Medical Device Real-World Evidence Marketplace

Summary of Information Requested


The National Evaluation System for health Technology Coordinating Center (NESTcc) is soliciting the interest of organizations to partner in generating high-quality solutions and expertise with real-world data (RWD) to support improved evidence for medical devices. Expertise is sought in five strategic areas to realize the vision of the NESTcc real-world evidence (RWE) Marketplace:


1.    Data platforms: providing a foundation for data gathering and curation, especially platforms that put the patient in the driver’s seat, to gather health data into one place and share with the device community

2.    Data sources: creating a national resource mirroring the experience of patient care in the United States and general experience of health throughout an individual’s life

3.    Data connectors: deliver continuity of care and experience between data sources (e.g., via aggregation/ linkage) while maintaining patient privacy and protections

4.    Data science and research methodology: design and analysis expertise and experience necessary to obtain the epidemiological, informatic and statistical rigor inherent in fit-for-purpose RWD analysis

5.    Analytic cores: conducting analyses of ingested data or packaging analytics for data sources to conduct analyses, ensuring rapid, valid and verified analysis suitable for RWD research


Given the complexity of working with RWD and transforming it into RWE, it is understood that many entities involved with RWD/RWE may provide services and technologies that do not fit neatly into these five areas. NESTcc is interested in learning of any relevant service or technology which may improve the ability of RWE generation.


Overview of NESTcc


NEST was established in 2016 by a cooperative agreement between the Food and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH) and the Medical Device Innovation Consortium (MDIC), a 501(c)3, as the first public-private partnership created with the sole objective of advancing regulatory science of medical devices for patient benefit. The purpose of NEST is to increase quality and efficiency in the development of RWE to inform medical device development and evaluation, as well as support clinical, patient, regulatory, and reimbursement decisions throughout the total product lifecycle (TPLC). The Coordinating Center of NEST (NESTcc) catalyzes RWE generation for medical device and health technology for all members of the device ecosystem.


The NESTcc’s direction and operations are informed by a diverse range of stakeholders including, but not limited to, patients, clinicians, device manufacturers, regulators, health systems, and payers. The Coordinating Center facilitates development of NESTcc data quality and research methodologies for RWE and coordinates an established network of NESTcc collaborating institutions who leverage health data and advanced analytics to facilitate linkages among data sources that collect, curate, and analyze data to ensure that it is fit for purpose to generate RWE. NESTcc requires the effort of the community of stakeholders, strategic partnerships, and solution providers to exemplify fit-for-purpose RWE today and to envision best practice RWE generation in the future.


As part of the FDA efforts for RWE, user fee funding is distributed to the NESTcc. The Medical Device User Fee Amendment (MDUFA) V funding of NEST will (1) support the development of RWE resources to facilitate access for studies and (2) to convene experts and develop best practices to advance innovative methodologies with respect to RWE development and analysis.


Vision for NESTcc


NESTcc is positioned to build on its existing network and expertise to assist stakeholders in navigating the medical device ecosystem and accelerate their use of RWE. NEST’s approach to increasing use of RWE is three-pronged.


(1)    Collaborative Community: The NESTcc brings together experts to consider and extend the frameworks for research methodology and data quality. NEST is maintaining a forward-looking approach and commissioning new tools to enhance utility of RWD. The frameworks and tools are being made available for use by NESTcc customers and collaborators including, but not limited to industry, FDA, and the other medical device stakeholders.

(2)    Guide: The NESTcc is providing leadership and guidance throughout the process of conducting fit-for-purpose RWE studies, including design, analysis, reporting, and supporting interactions with FDA, Centers for Medicare & Medicaid Services (CMS), and payors. NESTcc will utilize best practices which have been developed in conjunction with our diverse stakeholders in research methodology and data quality to guide the appropriate use of RWE for sponsors and provide confidence in the relevance and reliability of the data and studies conducted based on NESTcc’s neutral status as a non-profit entity.

(3)    Marketplace: NESTcc intends to create and will host a marketplace of organizations engaging in various aspects of RWE. NESTcc will understand the strengths of each organization and partner across organizations to garner the best fit solution for each RWE study using NESTcc. The goal of the marketplace is to provide the broadest range of solutions for all stakeholders.


NESTcc is a sustainable national resource that utilizes fit-for-purpose RWD to generate RWE for medical devices in a robust yet streamlined fashion. The NESTcc is guiding those wishing to engage in RWE generation throughout the process – honing a question, designing and conducting studies, reporting and dissemination, and regulatory submissions.


NESTcc provides a consistent, seamless experience to generate RWE which can support FDA premarket regulatory decisions and postmarket regulatory requirements and those of other national jurisdictions, collaborations in the Total Product Life Cycle Advisory Program (TAP), and data-generation for CMS Coverage with Evidence Development (CED) or full coverage in Medicare. By managing evidence generation needs across the diverse landscape of expertise and input necessitated by each project, NESTcc is reducing the timeframe, resources, and costs in conducting high-quality RWD studies and increasing the reliability of study results.


The Concept of the NESTcc RWE Marketplace


A medical device is “any instrument, machine, contrivance, implant, in vitro reagent that's intended to treat, cure, prevent, mitigate, diagnose disease in man.” Thus, medical devices cover a vast array of medical products, therapeutic areas, and risk profiles. Gathering and evaluating data on patient and provider real-world experience across the spectrum of devices is a major undertaking.


NESTcc created a Research Network as part of its initial work under the MDUFA IV Performance Goals, generally referred to as MDUFA Commitments. The NESTcc Research Network has been an invaluable resource for NESTcc to complete RWE test-cases which were also part of its MDUFA IV Commitments5 and to begin to work on sponsored research projects. These experiences have demonstrated that using the NESTcc Research Network alone for some projects leaves gaps that may benefit from the participation of additional strategic partners. NESTcc is currently working on methods to improve its infrastructure and capabilities and build on the strengths of its existing Research Network. As part of this work, NESTcc intends to create a RWE Marketplace which will allow a process for wider participation of the RWE ecosystem in Nestcc RWE projects.


The intent of this Request for Information (RFI) is to identify organizations who wish to partner with NESTcc in such a marketplace. It is not designed to replace the existing NESTcc Research Network, it is designed to enhance it. In addition to payment for their work, organizations selected to join the marketplace will have the opportunity to showcase their skillsets, contribute their expertise to RWD studies, and participate in collaborations which set the standard for best practice RWE generation.


Collaborating to Create the Future State


To design an ecosystem in which the right data sources are paired with best-in-class research techniques and platforms, NESTcc is soliciting interest in partnerships to access and generate high-quality RWD solutions and expertise in research with RWD in order to support improved evidence for medical devices. NESTcc has identified five strategic areas for partnership to realize the vision of the NESTcc RWE Marketplace:


1.    Data platforms: providing a foundation for data gathering and curation, especially platforms that put the patient in the driver’s seat, to gather health data into one place and share with the device community

2.    Data sources: creating a national resource mirroring the experience of patient care in the United States and general experience of health throughout an individual’s life

3.    Data connectors: deliver continuity of care and experience between data sources (e.g., via aggregation/ linkage) while maintaining patient privacy and protections

4.    Data science and research methodology: design and analysis expertise and experience necessary to obtain the epidemiological, informatic and statistical rigor inherent in fit-for-purpose RWD analysis

5.    Analytic cores: conducting analyses of ingested data or packaging analytics for data sources to conduct analyses, ensuring rapid, valid and verified analysis suitable for RWD research

6.    Other Services: any services that do not fall into the five areas above can be provided in this section


Information Requested


Organizations with expertise in any or all of the six areas defined above are invited to indicate their interest in joining the NESTCC Marketplace by submitting their response here. Please do not provide confidential information.


Terms and Conditions of this RFI


•    The issuance of this RFI does not constitute a commitment nor an obligation for NESTcc to contract services or conduct any additional follow-up to respondents.

•    Any costs associated with responding to this RFI are the sole responsibility of the respondent. NESTcc will not bear any responsibility for any such costs.


Next Steps


•    At its discretion, NESTcc will conduct follow-up with various respondents to seek clarification on information provided.

•    NESTcc will update the Medical Device ecosystem when any decisions are made regarding the NESTcc RWE Marketplace.

•    For questions, please contact nestcc@mdic.org


Submission Deadline - 5:00 p.m. ET on June 30, 2023

Select or enter value
Caret IconCaret symbol

General Information

Phone

Specific Information

Please provide the following specific information for any of the six areas of service or technology which can be used for the generation of RWE for medical devices.


Note: If you exceed the text box limit, you can attach a separate file(s) at the end of the form.


*Please follow the same question format when submitting a file.

•    Provide your current number of individuals with data available: overall, and by geographic region, sex at birth, race/Ethnicity, and socio-economic status. How many of these individuals have at least 5 years of data available?

•    With whom have you partnered to gather data and conduct research?

•    Please describe the data your platform is capable of capturing (e.g., informed consent, patient reported outcomes, diagnostic test results).

•    Are your data available for extraction outside of your platform?

•    Can analyses be conducted within your data platform?

•    What traceability capabilities exist within your platform?

•    How do you assess data integrity from initial point of capture to availability for research?

•    How do you assess continuity of patient care?

•    Describe any data adjudication policies implemented within your platform. Describe the frequency and type of any data error assessment and corrections, including a brief description of how these are documented.

a. General Information

•    Provide your current number of individuals with data available: overall, and by geographic region, sex at birth, race/Ethnicity, and socio-economic status. How many of these patients have at least 5 years of data available?

•    What proportion of patients have documentation of (any) imaging ordered? Imaging reports available? Images available?

•    What proportion of patients have documentation of (any) diagnostics ordered? Diagnostic reports available? Specific diagnostic brand/version available?

•    What number of patients have documentation of inpatient surgery? Of those, what proportion have surgical diagnoses and procedures available? What proportion have surgical notes available? What proportion have anesthesia notes available? Provide similar information for outpatient surgeries.

•    In what settings (e.g., hospital, specialist clinic, general practitioner, home) are your data captured?

  • Do you capture telehealth within your data? Psychiatric services? Oncologic services?

•    What terminologies are available for diagnoses (e.g., International Classification of Diseases, Tenth Revision, Clinical Modification [ICD-10-CM]), procedures (e.g., Current Procedural Terminology [CPT]), medications (e.g., National Drug Codes [NDC]), and labs/diagnostics (e.g., Logical Observation Identifier Names and Codes [LOINC]) in your data?

•    How are medical devices identified within your data? (Recommend providing examples for an implant, diagnostic, and single-use device.)

  • Do you utilize unique device identification (UDI) within your data? If not, do you have plans for future adoption of UDI?

•    Have you or a research partner ever converted your research data to a common data model (CDM)? Which one(s)?

•    Are your data available for extraction outside of your data source?

•    Can analyses be conducted for your data within your organization?

•    What traceability capabilities exist within your data source?

•    How do you assess data integrity from initial point of capture to availability for research? How do you assess continuity of patient care? Describe your policies and procedures to ensure completeness, consistency, and accuracy of data collection and management.

•    Describe any data adjudication policies implemented for your data source. Describe the frequency and type of any data error assessment and corrections, including a brief description of how these are documented.

•    Are patients deidentified? If needed, can researchers solicit new information from patients?

•    Have you participated in studies that aggregated your data with other data sources?

  • Have you or any partnering organizations used any tokenization technology to aggregate your data with other data sources? If so, which tokenization technologies have you used?


b. Source specific information

i. Electronic Health Record (EHR) vendors

  • Have your data been linked with claims data source(s)? Which one(s)?
  • Have your data been linked with other data source(s)? Which one(s)?

ii. Health systems with EHRs

  • Same as EHR vendors

iii. Claims Data

  • Have your data been linked with EHR data source(s)? Which one(s)?
  • Have your data been linked with other data source(s)? Which one(s)?
  • Do you have direct costs for diagnoses, procedures, or products available within your data?

iv. Other data sources: single institutions with EHRs, registries (device, disease), patient generated health data, other data sources

  • Have your data been linked with other data source(s)? Which one(s)?

•    How do you conduct aggregation/ linkage? What methodology do you use? How do ensure accuracy?

•    Do you obtain consent from patients for their data to be connected across data sources?

•   Have your services been used for aggregation/ linkage? If so, what types of organizations/ institutions have used your services (e.g., health systems, claims organizations, industry)?

• How do you ensure that patient privacy is protected?

• If you have data available in addition to aggregation/ linkage, please also complete the applicable portions of the “data sources” section above.

a.    Problem solver/guide/design/causal inference (e.g., epidemiology)

  • What expertise do you have in your organization to clarify and refine study questions? To develop protocols and analytic plans? For device studies?
  • What expertise do you have in your organization to address methodologic challenges in RWD study design? For device studies?
  • What expertise do you have in your organization to generate reports for studies using RWD which are suitable for regulatory or payor review? For device studies?

b.    Endpoint selection & determination of timeframe for follow-up

  • What expertise do you have in your organization to identify and provide rationale for endpoint selection in studies using RWD, including appropriate follow-up time?
  • What expertise do you have in your organization in designing study endpoints acceptable for regulatory review? With CDRH? Pre-market? What about for payor review?

c.    Statistics

  • What expertise do you have in your organization to address methodologic challenges in RWD study analysis? For device studies?
  • What expertise do you have in your organization to conduct analyses using RWD?
  • What expertise do you have in your organization in combining extant RWD with prospective clinical trials (e.g., using RWD as an informative prior in a Bayesian design; using RWD as an external control; deriving a performance goal from RWD for use in a single-arm prospective study)?

d.    Natural language processing / machine learning / artificial intelligence (NLP/ML/AI) design and conduct (e.g., develop algorithm/model to define specific variable)

  • What expertise do you have in your organization to determine when NLP/ML/AI algorithms are needed to define specific data elements?
  • What expertise do you have in your organization to develop and validate NLP/ML/AI algorithms/models to define specific data elements?
  • What expertise do you have in your organization to address methodologic challenges in NLP/ML/AI model development?
  • What processes have you used in the past to train and validate NLP/ML/AI models?

e.    Validation study conductor (start to finish)

  • What expertise do you have in your organization to design validation studies for RWD study outcomes? For study populations? For key covariates or other data elements?
  • What expertise do you have in your organization to manually abstract data from clinical charts and adjudicate data elements?
  • What expertise do you have in your organization for non-manual abstraction of a reference source?
  • What expertise do you have in your organization with validation using an imperfect reference standard?
  • What expertise do you have in your organization to conduct validation studies?
  • What expertise do you have in your organization to develop reports of validation studies suitable for regulatory or payor review?

f.    Operational definitions/phenotype generator

  • What expertise do you have in your organization for development of code lists for structured data (e.g., diagnoses, procedures)?
  • With which terminologies do you have experience in developing code lists (e.g., ICD-10-CM, CPT, NDC, LOINC)?
  • What expertise do you have in your organization to develop, refine, and provide justification for operational definitions of data elements within RWD?

•    Describe any data adjudication policies implemented within your analytic core. Describe the frequency and type of any data error assessment and corrections, including a brief description of how these are documented.

•    Do you have experience using a common data model (CDM)? Which one(s)?

a.    Ingest and analyze data

  • How do you verify extraction, transformation, and loading (ETL) accuracy into your system?
  • What types of data can be accommodated (e.g., claims/EHR only, any particular structured codes, unstructured data)?
  • Do you have prespecified statistical modules available for analysis of RWD?
  • What type(s) of expertise is needed (e.g., experience with specific software for analysis)?
  • Is there a user interface for data management and statistical coding?
  • What types of outputs (tables, figures, visualizations) are standard within your offering
  • What options exist specific to medical device RWD research? What options exist specific to diagnostic (IVD) RWD performance?
  • Do you have ML/AI as a standard feature in your system? What type(s) of expertise is needed to use this feature?
  • How are data explored within your system, before conducting formal analyses and without compromising the statistical integrity of the study
  • What structure(s) are in place to ensure that the exposure-outcome analysis is not conducted before development of a study plan? What structures/policies/procedures are in place to ensure that those with sufficient expertise are conducting and interpreting the study? What structures/policies/procedures are in place to document occurrence of analyses for auditing (e.g., to demonstrate the number of analyses or re-analyses)?
  • Do you have the ability to leverage data models / NLP models / source data for multiple studies? i.e., are you able to bring data on platform allowing multiple analyses to be conducted on a single data set?

b.    Other Possible Areas:

  • Data visualization UI
  • What data visualization options do you have for researchers outside of your organization to see the data, including any drill-down options for the initial visualization(s)
  • Build/deploy/house/monitor/QC data collection (e.g., for abstraction in data validation)
  • What expertise do you have in your organization to manually abstract data from clinical charts and adjudicate data elements, including images?
  • What expertise do you have in your organization for non-manual abstraction of a reference source?
  • Supervised learning with a user interface (UI) for data curation (e.g., as needed for SMEs to classify data going into AI/ML model)
  • What expertise do you have in your organization to develop a UI for SMEs outside of your organization to classify reference data?
  • Briefly provide an overview of the process for development and validation
  • Briefly provide an overview of the process for deployment and support during use

•    Detail any other services or technologies that do not fit in the above five areas.

•    Please clearly define how these services or technologies provide value in the RWE processes.

Drag and drop files here or

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.