Intertech Engineering Associates, Inc.

Medical Device Validation Series 4 of 4

Part 4, Inadequate requirements usually result in inadequate testing.

This is part four of a four-part series on medical device validation practices.

Here are some telltale signs of inadequate requirements:

  • Software developers need constant access to market and clinical specialists to understand how they want the software to work.
  • Code writing begins before requirements are written or approved.
  • Test authors need frequent access to the software developers to understand how the software should work.
  • Formalized testing finds few if any defects, while “experts” (e.g. developers) with the system find many defects in ad hoc (It could be inadequate testing too, but usually it is a combination of poor requirements and poor testing.)
  • Review of test results is time-consuming and requires “experts” with the software to determine whether tests passed or failed. Note that this does not meet the standard of objective evidence of test results if the results have to be interpreted.
  • A preponderance of tests fail by the testers but are subsequently passed because the “testers didn’t understand how the software works.”

If these red flags fly, you need to go back and rewrite the requirements, making them specific enough so that they can be tested properly. Adequate requirements are the foundation of testing and should be the foundation of software implementation too.  A little extra up-front effort can prevent the whole structure from collapsing later. While testing alone doesn’t amount to validation, proper validation demands testing, and testing depends on testable requirements.

From: Vogel, David A., Ph.D. “Validating Medical Device Software Includes and Goes Beyond Testing.” Medical Product Outsourcing Mar. 2006: n.

Pharma Companies Challenges with Combination Devices and Planning for Variation Part 1 of 3

More and more now Pharma companies are now taking on medical device development as they intend to provide their customers with combination devices. Planning the device development does not always get the attention in project planning as it needs from a project management perspective since the drug development usually takes precedence. Scheduling a drug and a device, in parallel, requires careful management to coordinate the many interactions between drug development and the device development. Planning and scheduling the lifecycle for the product, particularly the development phase, should include integration milestones for the drug and device together. The development long pole is, obviously the drug development and regulatory clearance path, but this does not mean a combination device manufacturer should leave paying attention to the device development for the end. One challenge that is frequently underestimated or overlooked is considering and planning for addressing device variation and its impact on the overall combination devices performance. Combination device developers should plan an iterative device development model as well as integration milestones as a part of the development phase of the lifecycle to also address this. This article series discusses some Pharma company project management approaches as well as some key activities frequently overlooked or underestimated.   

 

Part 1 of 3: The Combination Device Timeline  

In understanding the development lifecycle for a combination device, the overall milestones to a market release in the US is defined by the regulatory objectives. Developing a pharmaceutical drug requires much in research and testing as it relates to the science we will not cover in this article series, but a high-level explanation of the FDA expected regulatory path is where we will start.  

 

What is a combinational device? 

 

The FDA has defined a combination product as:  

 

“…a combination product is a product composed of two or more different types of medical products (i.e., a combination of a drug, device, and/or biological product with one another). The drugs, devices, and biological products included in combination products are referred to as “constituent parts” of the combination product.”

 

What are the regulatory pathways for a combination device? 

The development team needs to map out the development process and prepare to develop the product, drug and or the biologic iteratively over a development lifecycle. To do this the developer with regulatory oversite will overlay the device process with the longer drug development timeline.  The timelines for a drug are driven by the regulatory strategy considerations and may be dictated to the manufacturer based on which center has jurisdiction for the product.  The developer can file for an RFD (Request for Designation) and additionally provide a Pre-RFD to get quick feedback on how the FDA will classify the combination product as well as identify the jurisdiction. The jurisdiction is driven by the definition of the primary mode of action (PMOA) and is assigned to one or more of the three possible FDA agency centers by OCP. Additionally, the combination will have one primary jurisdiction center determined based on the PMOA. The FDA centers are: 

  • CDRH – Center for Device and Radiological Health  
  • CDER – Center for Drug Evaluation and Research 
  • CBER – Center for Biologics and Research 
  • OCP – Office of Combination Products 

 

If the Primary Mode of Action is driven by the drug the primary center of justification will be CDER, if it is the device, it would be CDRH, if it is a biologic then CBER. The FDA can help define the PMOA, as the option to request for a designation RFD is available to manufacturers.  

 

The next consideration is for the developer to plot out the regulatory submission types once the drug is far enough and ready for controlled laboratory and animal testing. Submission application types can include:  

  • an investigational device exemption application (IDE);  
  • an investigational new drug application (IND);  
  • biologics license application (BLA);  
  • new drug application (NDA); or  
  • premarket approval application (PMA).  
  • Other types of applications that might pertain to a combinational device include (premarket notification 510(k) or abbreviated new drug application ANDA). 

 

A manufacturer of a combinational device will likely be required to undergo clinical trials under a new drug application (NDA), investigational device exemption (IDE) or an investigational new drug (IND) application. Once the submission pathway is known the clinical trial strategy can be defined identifying the clinical trials as being key milestones for the development process which drives the first stages of the product lifecycle.  

 

What are important considerations that drive pharmaceutical and device development? 

 

Drug development requires preclinical evaluation, screening formulation, toxicology testing, pharmacology and biochemistry testing, just to name a few. All of these testing components represent many person-hours of effort that make up the first part of drug development. This is the basic careful science necessary before a manufacturer is even close to clearing a drug or thinking about a combinational product, to deliver the drug.  Once a drug enters the phased clinical step there is a more defined framework to schedule things around. These clinical studies are completed in three premarket clinical phases. 

 

As mentioned already, that just getting to the clinical phase of drug development is one significant effort, but getting through the three clinical phases, is also a no easy task. There are three key clinical phases and trials for a drug, each having a particular objective necessary to support FDA clearance.  The phase 1 clinical trial is intended to support primary safety. The phase 2 clinical trial will support short-term safety, but it is mainly intended to support effectiveness. Finally, the phase 3 clinical trial is intended to demonstrate safety, dosage, and effectiveness. According to a BIO report looking at clinical phase transition success and likelihood of approval (over a period from 2005 – 2015), drug manufacturers only have a 9.6% chance of successfully getting from phase 1 clinical to drug approval (through all 3 phases).  When a manufacturer decides that they wish to market the drug with a device, most frequently facilitating a delivery system, they really should decide this during the early clinical phases so the device development can occur early enough to support the clinical trials.  

 

For a combination device, the clinical submission and trials will serve to support the drug approval, but it will often support the device safety and effectiveness as well. Understanding this basic framework of the clearance establishes a timeline where some key device activities and milestones can be mapped. For a combination device, aligning the timelines between the drug and the device becomes critical. 

 

Developing a drug, from the molecule, through approval takes 10 to 15 years and can cost developers over $1 billion. Device development, from concept to approval can take 2 to 5 years and can cost developers tens of millions of dollars. Because of the cost and risks in both the device and drug development many companies developing combination devices will attempt to choose a drug and or a device that has been cleared for other similar uses and therefore proven to a degree that is possible, for the combined effort. The below example timeline is scaled to an aggressive overall timeframe and is intended to demonstrate relative time periods and some key milestones of the device development over the drug timeline. 

 

 

(Timeline  Figure 1) 

The example timeline figure above shows the drug regulatory phases with the device development lifecycle represented on the bottom. The device lifecycle represented reflects the design control elements expected in a common sequential waterfall development lifecycle. This does not suggest that a waterfall lifecycle is explicitly required but intends to represent the primary goals for the first baseline during this time period. It is expected and practical to expect that planning and requirements change, as well as the design, throughout the whole lifecycle. Good design control process should be established to ensure each of the sequential elements of design controls is applied for iterations, as the product evolves and the design output matures. 

 

At this point, we can discuss three important device development activities that are frequently overlooked or underestimated by drug manufacturers in a combination device project. These activities drive some of the milestones we should establish in our development lifecycle planning.  Three important key device activities are:  

  • Key activity 1: the sourcing or manufacture of devices to support the development lifecycle (particularly for phase 2 and 3 clinical and device verification). What is important about these devices is maintaining an understanding of the configuration of these devices, so interactions and impact on the clinical studies is understood;  
  • Key activity 2: the planning and coordination of usability studies. What is important about usability is that the results drive requirements, design as well evaluation and managing of particular safety risks; and 
  • Key activity 3: the coordination and the application of the appropriate design controls during development to address sources of variability and optimize the device design. What is important about managing the design process is that doing so adds assurance to the maturing of the actual product as well as demonstrating through documentation of the inputs and outputs the developer can establish confidence in the results testing and performance.  

 

The three keys will be further discussed in the next section of this article series.  We would be interested in your own experiences with these or based on your experiences what other areas are just as or more important.

The Essential Performance SNAFU part 3 of 4

As discussed in a previous part of this series, the family of the general standard IEC60601-1 includes collateral and particular standards that might apply to the manufacturer’s particular device. This part of the article series will highlight what collateral standards to IEC60601-1 have in them as well as what the particular standards address with regards to performance and essential performance of a medical device.

 

Examples of IEC60601-1 Collateral Standards

Collateral standards cover broad subjects, and for IEC60601-1 there are several. Below is a list of the current popular collaterals to IEC60601-1 and a brief description of what they cover:

These standards may apply to a manufacturer’s device and typically add additional requirements to the clauses of IEC60601-1 and include the testing and testing fixtures that would apply. When applying these standards, testing would require that the determined Essential Performance (EP) be evaluated during the test. After applying testing per the collateral standard there is an evaluation of the essential performance and a determination is made by the tester if EP is impacted to a point where harm can occur. Adding the Essential Performance to the test criteria was all new to certified test houses, used to evaluating only Basic Safety. Evaluation and monitoring of Essential Performance during and after testing requires a more advanced understanding of the device’s clinical impact to correctly conclude on the device’s safety. In reality, the acceptance criteria got a lot harder for testers and test houses to appropriately distinguish a passing device from a failing device.

The Essential Performance characteristics are the aspects we check for safety in standards tests; therefore there is a consequence if the EP items are not chosen correctly. If lower level, technical characteristics are identified as the Essential Performance characteristics, then we may incorrectly fail a device, where safety is not truly impacted.  If true Essential Performance characteristics go unidentified and are not addressed or tested to a point there is a potential for harm, then testing would be ineffective and the device may be released with safety inadequately addressed. This puts a lot of selecting what the Essential Performance characteristic is correct.

What Does System Engineering tell us about determining Performance measures?

Performance measurement from a system engineering perspective may include the consideration of measures of effectiveness and measures of suitability. Measures of effectiveness are measures that correspond to the accomplishment of mission objectives and achievement of the desired results. These performance measures are determined through a capabilities-based assessment of the system and its objectives. Examples of these performance measures include the characteristic of measure of a system’s performance expressed as a speed, payload, range, time-on-station, frequency, or other distinctly quantifiable performance feature. Performance measures considering suitability include a performance measure of an item’s ability to be supported in its intended operational environment. Examples of these performance characteristics typically relate to readiness or operational availability, reliability, maintainability, or the item’s support structure.

The above characteristics of performance are different from technical performance measurements. Technical Performance Measurement (TPM) involves a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development based on current assessments of products lower in the system structure (per EIA-632 “Process for Engineering a System”) 1.

Manufacturers defining there performance characteristics need to determine which performance measures that correspond to mission objectives and suitability when defining Essential Performance and not fall into the trap of cycling through all technical performance measurements made on a device to determine what the essential performance is.

Particular (dash 2 Performance) Standards

As mentioned earlier, Particular standards include a list of Essential Performance characteristics in the standard. Looking at some particular standards for IEC60601-1 will help us understand what these performance characteristics look like. The Essential Performance characteristic may include areas such as the accuracy with which the equipment controls the delivery of energy or therapeutic treatment to the patient, as well as the devices ability to process and display physiological data that will affect patient care decision management.

The IEC60601-1 Particular Standards are available for some specific medical devices. These standards are represented by the IEC 60601-2-xx series, and are often referred to as “dash 2 standards.” Some examples of particular standards for IEC60601-1 are provided below. For those devices that do not have a dash 2 standards, the determination of Essential Performance is left to the manufacturer as defined in IEC60601-.

 

Examples of Essential Performance Characteristics from the Particular Standards

In addition to providing the Essential Performance characteristics for a medical device, the particular (dash 2) standards also include additional clauses that would replace or delete requirements that are called out in the general standard (IEC60601-1). The below table provides an example of some of the Essential Performance characteristics identified in a sampling of particular standards.

Table 1: Examples of EP items for some medical devices.

The definition of Essential Performance is defined per IEC60601-1 section 3.27 as:

“performance of a clinical function, other than that related to Basic Safety, where loss or degradation beyond the limits specified by the manufacturer results in an unacceptable Risk.

Note Essential Performance is most easily understood by considering whether its absence or degradation would result in an unacceptable Risk.”

In the review of table 1 above, it is clear that the actual acceptable performance levels are not prescribed, but left to the manufacturer. Additionally, the approach for how the characteristics are to be described across a sample of particular standards is not necessarily consistent. In some cases, the clinical function of the essential performance characteristic is not part of the specified item, as in “Alarm signals for high priority alarms” for infusion pumps. But, the consideration for what the actual acceptable performance level is as well as how these essential performance characteristics are to be applied while evaluating or testing the device is left to the discretion of the manufacturer. The note in the definition of 3.27 above does provide for how manufacturers should determine the specifics as well as what the acceptable performance ranges are. Once performance degrades to a degree that the risk is unacceptable (and harm can occur), this is when an unsafe condition or state has occurred (and therefore operating out of the determined range is unacceptable).

FDA Recognized Consensus Standards

The FDA recognizes standards that are deemed appropriate for manufacturers of medical devices to declare conformance with. The table above shows which of the example particular standards are FDA recognized consensus standards. The use of consensus standards to meet premarket submission requirements can help the FDA premarket review process although this is ultimately up to the submitter. Although the FDA reviewer may request the submitter to comply with a particular standard, as well as submit test data, as is the case with ISO 10933-1 for the Biological evaluation of medical devices. Additionally, if the standard is not FDA recognized, it doesn’t mean a reference to it or compliance to will not be considered acceptable by the FDA. The submitter can determine which standards the FDA suggests or recognizes by accessing the FDA website of recognized consensus standards, or determining if an FDA guidance is applicable that may also identify applicable standards.

From clearance standpoint, the FDA may not be as focused on identifying Essential Performance or testing Essential Performance, but they will be looking for the manufacturer demonstrate safety. The FDA has moved over the years to rely on standards, as well as identifying standards they recognize, but they always reserve the right to require more, regardless of what some of the IEC standards may require. In some cases, the FDA will put out FDA guidance documents that provide expectations on manufacturers for how performance is to be indicated and tested. The FDA has also indicated that risk can be used to determine what areas of a medical device require attention, but they have yet to say they are only interested in attending to only Essential Performance characteristics.

The next part of this article series will provide some examples of what has happened to the industry when Essential Performance is not correctly identified, as well as provide some recommendations for manufacturers. We are interested in your experience with Essential Performance and hope you will contact us and share them with us.

1 –  http://www.acqnotes.com/Attachments/EIA-632%20“Processes%20for%20Engineering%20a%20System”%207%20Jan%2099.pdf

Understanding Cybersecurity Risk

Date: 1 25 2018    By Geoff Hutchins, Harold Pogue, Mohammad Raza

Understanding Cybersecurity Risk

Like other risk management perspectives, business and product, addressing cybersecurity needs to be considered throughout the development process[1].

A key part of this is understanding the types and classification of the cybersecurity risk[2] as a framework for assessment and development of control measures.

There are several viewpoints that should be considered when setting this up as follows:

  • Classification of the nature of the risk, malware, riskware, spyware etc.
  • Product lifecycle stage, premarket, post-market and legacy device
  • Risk introduced using OTS & SOUP libraries[3]
  • Product intended use, use environment, and hazard profile, life-sustaining, diagnostic, hospital or home
  • Classification by core functions, Identify, Protect, Detect, Respond and Recover
  • Classification by means of access, network connected, wirelessly, USB and medium such as CD

There are several existing frameworks that a manufacturer may consider using:

  • NIST Cybersecurity [4]
  • Common Vulnerability Scoring System (CVSS)4
  • AAMI TIR57 Principles for medical device security – Risk Management
  • UL 2900 Testability of network connected devices.

Cybersecurity Risk Management Process

The cyber risk management should be considered as part of the product lifecycle management:

  • Conduct product requirements cyber risk analysis and assessments.
  • Evaluate code, library and tool vulnerabilities and validations.
  • Plan and execute penetration testing on product configurations.
  • Document testing and re-evaluation plans for new threats
  • Plan for the deployment of updates and associated cybersecurity risks deploy cyber risk.

Some essential ways to mitigate risk include the following:

  • Require the user to update the default password of the device. Default passwords, especially for networked devices, are well known and can be exploited. Use longer passwords with a strong measurement of randomness.
  • When designing the device avoid using WEB/PSK/TKIP. These are no longer considered secure and have been depreciated. They share the same key and if that key is compromised, all devices in the network that communicates with that key are vulnerable.
  • Ensure that all data at rest is encrypted. It is also good practice to create backups prior to any change and to ensure the device user on how to recover the backup. If data is stolen the encryption should maintain privacy.
  • For software as a medical device, it is necessary to designate a patch management facility/process. Patches are common but can sometimes cause problems. Having a process in place that verifies that the patch works on the equipment before wide-scale deployment will minimize risk.
  • Have a periodic device verification plan and perform regular audits.
  • Perform security tests and provide these to the end user. These tests should include encryption, authentication, patches to the software, and version of virus protection.

[1] Content of Premarket Submissions for Management of Cybersecurity in Medical Devices, Guidance for Industry and Food and Drug Administration Staff, October 2, 2014

[2] IEC TR 80001-2-2 Edition 1.0 2012-07 Application of risk management for IT Networks incorporating medical devices.

[3] Guidance for Industry, Cybersecurity for Networked Medical Devices Containing Off-the-Shelf (OTS) Software, January 14, 2005.

[4] Postmarket Management of Cybersecurity in Medical Devices – Final Guidance, FDA presentation January 12, 2017, CDRH Webinar

 

Medical Device Validation Series 3 of 4: It all Starts with Requirements

Part 3, It All Starts with Requirements

This is part three of a four-part series on medical device validation practices.

Is writing requirements a validation activity? Of course, it is! What has been verified if verification testing is attempted without written requirements? The GPSV points out: “Success in accurately and completely documenting software requirements is a crucial factor in successful validation of the resulting software.”

There is some room for debate on what constitutes a requirement and what constitutes design detail. While much benefit derives from solid, well reviewed requirements that have nothing to do with testing, the testing effort should be based on verifying the correct implementation of requirements.

Tests should test requirements. Too often, tests are written without detailed requirements or not enough detail in requirements (exploratory testing), and the test developer is forced to refer to the software for details regarding how the software works. The result is that the test ends up documenting the way the software works, not necessarily the way it is supposed to work. The requirements for the software are embedded in the test, not in a requirements document. Many problems are associated with this, but perhaps most alarming is that potential problems can get embedded into tests as expected behavior—thus, making it difficult to identify them as problems, and guaranteeing that the behavior will be accepted forever after.

From: Vogel, David A., Ph.D. “Validating Medical Device Software Includes and Goes Beyond Testing.” Medical Product Outsourcing Mar. 2006: .

The Essential Performance SNAFU part 2 of 4

The Essential Performance SNAFU:

Defining what Essential Performance is for a medical device is a challenge, but the definition is especially challenging and often confusing when software is part of the product. This article series is intended to share what is happening in the industry once Essential Performance is defined, and provide some guidance on how to address compliance and safety avoiding the SNAFU’s we have observed in this process.

 

The compliance community has been using the Essential Performance (EP) to focus testing and evaluation of medical products since the 3rd edition of IEC60601-1 (published in 2005). Some medical device manufacturers are fortunate enough to have their Essential Performance defined for them through Particular standards if they are available for their medical device. Those less fortunate, who are left to define this for their products, are often challenged to determine what is essential.  This can often be confusing and we have found that industry test houses are not helping manufacturers make sound decisions when it comes to devices containing software.  

 

Part 2. Defining Essential Performance

The 3rd Edition of IEC60601-1 and Programmable Electrical Medical Systems (PEMS)

The changes made in the third edition of IEC60601-1 define a general approach of adopting two new main principles as described in the standards’ introduction. The first principle is the change in approach in the series of standards related to the concept of safety which has been broadened to include Basic Safety considerations and now Essential Performance matters. The second change in principle is the addition of a provision for assessing the adequacy of the process compliance when this is the only practical method of assessing the safety of certain technologies. There are two such examples that stand out in the standard, one being the application of ISO14971 risk management processes and the other is adopting software lifecycle processes to support programmable electronic medical systems (PEMS).  This PEMS consideration is clause 14 of the standard and it, when applicable, requires a software development lifecycle and software validation.  In considering these processes it is important to recognize that the ISO14971 standard requires a risk management process to support and assure safety and not just identification of characteristics of essential performance and basic safety per the IEC60601-1 standard. This article will further examine the consequences of how what is determined as essential performance might impact other supporting process standards.

 

Essential Performance (EP) and the Impact on PEMS

Examining the PEMS clauses further and clause 14.1 of  IEC60601-1 we find:

“The requirements in 14.2 to 14.12 (inclusive) shall apply to PEMS unless:

– none of the Programmable Electronic Sub-System (PESS) provides functionality necessary for Basic Safety or Essential Performance; or

– the application of Risk Management as described in 4.2 demonstrates that the failure of any PESS does not lead to an unacceptable Risk.”

 

Clause 14.1 indicates that PEMS applies if software provides functionality necessary for Basic Safety and Essential Performance or if application of clause 4.2 (ISO14971) does not lead to an unacceptable risk. An important part of note is the “or” part of the clause which might be easily ignored or missed when determining if the PEMS clauses apply. The PEMS clauses include the requirement to apply a software development lifecycle including software validation. The annex A in IEC60601-1 of the standard is intended to provide general guidance. In the annex A of the standard the guidance for clause 14.1 says this:

 

“Requirements have been minimized to those that are essential to assuring Basic Safety and Essential Performance. This has been done in recognition of the extensive and growing literature in the fields of software assurance and Risk Assessment techniques as well as the rapid evolution of this discipline.”

 

Is this suggesting that clause 14 (PEMS) is only applicable to Basic Safety and Essential Performance? There has been some indication that some manufacturers are interpreting that this is the case, and doing so with or without completing risk analysis on PEMS.

 

Interpretation of PEMS and Software Validation Driven by EP definition

There have been instances where clause 14 (PEMS) of the IEC60601-1 standard has been interpreted by medical device manufacturers that a software development lifecycle and software validation is not be required. The tendency of a manufacturer to conclude excluding additional, non-required processes and the additional resource costs and time needed for these activities is easy to understand. Unfortunately, manufacturers are already constrained by time boxed commitments, limited and dwindling budgets by the time they start looking at standard compliance. Additionally, many test houses are also concluding and complicit with manufacturers stating that if the software on a medical device can’t affect Basic Safety or Essential Performance then PEMS will not apply.  This has led to cases inside and outside the US,   where manufacturers have excluded implementing a software development lifecycle and software validations, to find out later the consequences of this.

 

Is a Software Development Lifecycle and Software Validation Important?

It is easy to agree to the premise that software validation for a medical device, where the safety of the patient is at risk, should be performed by the medical device manufacturer to provide adequate assurance the software in a device is safe. But is following a standard and applying software development activities and validation always a good idea?

 

Advances in technology has gotten us to the point where software has become a more and more significant driving force in how all devices with hardware operate. There is literature as well as our personal experience to support that the practices of defining objectives, agreeing on requirements, planning tasks and activities and analysis and testing of potential solutions lead to a higher confidence in outputs that meet intended needs. With this in mind, we would not recommend complete exclusion of the application of a software development lifecycle and development process nor some level of software validation on any product, whether it be a medical device or not.  Even companies that provide consumer products apply some software development process and testing; does it make sense that complete exclusion for a medical device is sensible?

 

Is the Process for Identifying EP the Same as Conducting Risk Analysis?

ISO14971, the Industry recognized standard for risk management has been around since 2000. This standard requires systematic use of available information to identify hazards and to estimate the risk for medical devices. Typically, risk analysis is conducted by identifying hazards including the use of an examination of potential systematic failures, such that the risks associated with the device can be evaluated. Examining systematic failures requires a bottom up analysis, where the known and previously unknown hazards and potential harms are evaluated and discovered.

 

Basic Safety refers to physical hazards; this is defined in clause 3.10 of the 60601-1 standard and was discussed in the previous section of this article. The identification of the Essential Performance aspects can be limited to those aspects that fall under what is performance as well as what is a clinical function as per section 3.27 of IEC60601-1. Do the definitions of Basic Safety and Essential Performance together include consideration of all potential Hazards? Examples of Hazards provided in Annex E of 14971 include evaluating functional operational failures for risk, providing examples such as “Incorrect or inappropriate output or functionality” or “Erroneous data transfer”.  What makes this challenging for manufacturers is whether a hazard such as these examples is something that is evaluated (or identified) by following IEC60601-1, particularly if these failures don’t fall under the definition of Essential Performance or Basic Safety. In some cases manufacturers are limiting the identification of risk controls if the function or characteristic is not defined as Essential Performance or Basic Safety and may not be supporting these conclusions through the process of risk management to assure their product is safe.

 

Is the FDA Using EP and is it the same as “essential to the proper functioning of the device”?

The FDA Design Controls refers to a process of identifying characteristics that are essential for the proper function of a medical device. This design control regulation and guidance was written and in place before the IEC60601-1 standard defined Essential Performance. The FDA Design Control Regulation was put in place in 1996 and identified in section 21 CFR 820.30(d) that:

 

“Design output procedures shall contain or make reference to acceptance criteria and shall ensure that those design outputs that are essential for the proper functioning of the device are identified.”

 

The obvious conclusion is that both Essential Performance of a clinical function and identifying what is essential to the proper functioning of the device should both be outputs of risk analysis, although this may be a result of risk analysis focused at different points of the hierarchy of the system. For example, the evaluation of risks related to clinical functionality is closer to the inputs and at the top of the system architecture versus evaluation of risks of device system or software functionality which are closer to the outputs and at a lower level of the design.

 

The FDA clears medical products through the premarket notification process, much like a notified body provides medical product manufacturers a CE mark. The FDA provides guidance for what the manufacturer should submit. For medical devices marketed in the US the regulation requires medical device software validation and risk analysis per 21 CFR 820.30(g). The FDA regulation is less prescriptive in terms of what the process is for risk analysis. The FDA Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, issued in 2005, is more prescriptive on what documents are submitted for the medical device submissions. This guidance requires submissions to contain some level of documentation for hazard analysis and software validation for ALL medical devices that contain software, for all levels of concern.  The FDA General Principles of Software Validation, issued in 2002, defines what is expected for software validation.

 

The FDA General Principles of Software Validation does provide a provision for the scale of the software validation effort in that they say that:

 

“The resultant software validation process should be commensurate with the safety risk associated with the system, device, or process.”

 

This statement in the FDA guidance is similar to what IEC60601-1 is getting at with identifying Essential Performance characteristics.  They are both derived through an evaluation of risk, although the Essential Performance characteristics are limited to the “performance of a clinical function” per its definition. The interpretation that risk analysis and applying a software development lifecycle, including software validation is obviously not something the FDA expects manufactures to exclude in their processes, or in submissions for clearance to market.

 

The FDA’s expectations of voluntary standards and performance standards will be further discussed in the next section of this article series.  We would be interested in your own experiences with essential performance and if you have had issues with CE marking process or the FDA.

 

 

Medical Device Validation Series. Part 2 of 4

Part 2, Validation: More Than Testing

This is part two of a four-part series on medical device validation practices.

A common misperception is that validation of software is synonymous with the testing of software. This is not at all accurate.

Federal regulation requires software validation, not software testing. Validation, by the FDA’s definition, is the “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”

Certainly, testing activity may be a component of validation, but note that the definition above does not use the word “test” at all. In fact, the definition mentions specifications and requirements specifically, assuming they exist and therefore creates a de facto linkage between validation and requirements.

The GPSV describes at length the definitions of, and differences between, software validation and software verification. Only a few of the related activities would be considered test activities. Similarly, verification activities, though narrower in scope, involve reviews, evaluations, and testing activities.

Keep in mind that all verification and test activities are validation activities, with other activities making up the remainder. Some testing is considered a verification activity, but there are verification activities that are not testing activities, and there is also testing that is not verification testing. Stay mindful that validation is not the same as testing.,

From: Vogel, David A., Ph.D. “Validating Medical Device Software Includes and Goes Beyond Testing.” Medical Product Outsourcing Mar. 2006: .

Build The Cybersecurity IN!

Build The Cybersecurity IN!

by: Gary Girzon

“Build the Quality In” is one of the pillars of lean manufacturing as first introduced by Dr. W.E. Deming and well perfected by Toyota Production System. The same practice should be applied to medical device cybersecurity. “Build the Cybersecurity In”, or face the consequences, as recently disclosed in FDA warning letter to St. Jude Medical (now owned by Abbott), on April 12, 20171. The letter is the latest chapter in issues concerning St. Jude implantable cardioverter defibrillators (pacemakers) and their wireless monitors. The full story continues to be well documented. Here’s a chronology of the key events: 

August 25, 2016 Muddy Waters, a hedge fund specializing in short stock positions, and MedSec, a medical device security firm, publish a report4 describing several vulnerabilities in the St. Jude devices, such as 

  • Debugging and development capabilities left on the device; 
  • Lack of encryption and authentication in the communication protocol between the implanted and external devices; 
  • Examples of ability to remotely crash the device and drain the battery 

MedSec claimed it had no choice but to go public in a joint press release with Muddy Waters since St. Jude had already known about these vulnerabilities and failed to address them.   

August – September 2016 – St. Jude refutes the Muddy Waters claims5. Muddy Waters responds to St. Jude6. University of Michigan cybersecurity researchers finds flaws in MedSec findings7. St. Jude Medical sues Muddy Waters and MedSec8. 

October 2016 FDA issues a “Safety Communication” on Premature Battery Depletion in St. Jude devices but recommends continued usage of devices9. FDA notes the investigation of cybersecurity allegations. Muddy Waters and MedSec release additional evidence (videos showing emergency shock, vibration and disabling of features) of how the devices can be compromised10, and St. Jude in turn responds11. MedSec hires an independent expert witness (Bishop Fox) to collaborate the original findings12. 

January 9, 2017 St. Jude releases a software update with cybersecurity fixes13. FDA issues another “Safety Communication” affirming the software update, with the recommendation that “health benefits to patients… outweigh the cybersecurity risks”14. ICS-CERT, a team within the NCCIC division of US Department of Homeland Security, issues an updated advisory on a “man-in-the-middle” vulnerability which has been mitigated by the software update15. 

April 12, 2017 FDA issues a “Warning Letter” to Abbott (St. Jude) describing several reliability and security problems, notably: 

  • St. Jude did not “confirm all corrective and preventative actions were completed, including full root cause investigation of actions to correct and prevent recurrence of potential cybersecurity vulnerabilities” 
  • St. Jude failed to perform a full verification of network port vulnerability – no threat testing was performed using an unauthorized interface. 
  • St. Jude did not fully address and incorporate a 3rd party security assessment commissioned in 2014 as part of its internal risk assessments. The 3rd party identified a “hardcoded universal unlock code as an exploitable hazard”.    

The FDA letter also noted several battery related issues not related to cybersecurity. Abbott’s initial response acknowledged the FDA observations – the company has 15 days to fully respond. Meanwhile, any such class III devices will not be approved for sale to the public. 

One hopes that St. Jude / Abbott, and other medical device companies, learn from this story. It may be that Abbott has addressed some of the issues noted by FDA, and either was incomplete in testing (such as not doing “negative” testing) and disclosing the changes (perhaps the “hardcoded universal unlock code” was removed). It may be that the cybersecurity fixes are more complex than originally thought, as could be the case when dealing with a communications protocol.  Clearly, St. Jude did not follow the FDA premarket guidance on cybersecurity16 and design it in at the beginning of the product creation. And by not acknowledging the full extent of issues found by Muddy Waters / MedSec early St. Jude at best created the perception of a cover-up. Even if no patients have been harmed by a lapse in security so far it’s the possibility that alarms the medical and patient communities. The FDA has recently released guidance for postmarket management of cybersecurity in medical devices17 in which methods collaboration and communication between medical device manufacturers, cybersecurity researchers, clearing houses and the public to disclose vulnerabilities is deemed integral to managing such threats. To be safe and secure, design the cybersecurity in at the start of the product lifecycle, model the threats, and be prepared for any new vulnerabilities.   

Medical Device Validation Series. Part 1 of 4

Medical Device Validation Series. This is part one of a four-part series on medical device validation practices.

Part 1, Introduction

The Food and Drug Administration (FDA) pays special attention to software because it is now embedded in a large percentage of electromedical devices, and the amount of device functionality controlled by software is continually growing. Software also controls many of a medical device manufacturer’s design, development, manufacturing, and quality processes, regardless of whether software is a part of the manufactured device.

Software failures often can be invisible and difficult to detect; thus, these failures can have disastrous consequences on the operation or quality of medical devices. For this reason, the FDA specifically requires validation of both device and quality-system automation software. Validation activities are meant to keep defects from getting into the software, as well as to detect and correct any defects that do end up in the software.

The FDA’s control over software used by medical device manufacturers is detailed in the Quality System Regulations (QSRs) found in FDA regulation 21 CFR 820. Software regulations focus on the development and use of two large categories: (1) software that is part of the device being manufactured and (2) software that is used to design, develop and manufacture the product or otherwise automate any part of the quality system.

Guidelines for complying with the FDA’s regulations are published by the agency as a “Guidance Document”. These documents are updated periodically, and new a guidance is issued as the need arises. While compliance with the guidelines is voluntary, the device manufacturer should be prepared to explain and defend any deviation from the guidances.

The most important FDA guidance available for the validation of software is the General Principles of Software Validation (GPSV), and can be obtained for free from the FDA’s website (https://www.fda.gov/downloads/MedicalDevices/…/ucm085371.pdf ). This is a “must read” for all software engineers and quality engineers working with software in the medical device industry.

From: Vogel, David A., Ph.D. “Validating Medical Device Software Includes and Goes Beyond Testing.” Medical Product Outsourcing Mar. 2006