Intertech Engineering Associates, Inc.

The Essential Performance SNAFU part 3 of 4

As discussed in a previous part of this series, the family of the general standard IEC60601-1 includes collateral and particular standards that might apply to the manufacturer’s particular device. This part of the article series will highlight what collateral standards to IEC60601-1 have in them as well as what the particular standards address with regards to performance and essential performance of a medical device.

 

Examples of IEC60601-1 Collateral Standards

Collateral standards cover broad subjects, and for IEC60601-1 there are several. Below is a list of the current popular collaterals to IEC60601-1 and a brief description of what they cover:

These standards may apply to a manufacturer’s device and typically add additional requirements to the clauses of IEC60601-1 and include the testing and testing fixtures that would apply. When applying these standards, testing would require that the determined Essential Performance (EP) be evaluated during the test. After applying testing per the collateral standard there is an evaluation of the essential performance and a determination is made by the tester if EP is impacted to a point where harm can occur. Adding the Essential Performance to the test criteria was all new to certified test houses, used to evaluating only Basic Safety. Evaluation and monitoring of Essential Performance during and after testing requires a more advanced understanding of the device’s clinical impact to correctly conclude on the device’s safety. In reality, the acceptance criteria got a lot harder for testers and test houses to appropriately distinguish a passing device from a failing device.

The Essential Performance characteristics are the aspects we check for safety in standards tests; therefore there is a consequence if the EP items are not chosen correctly. If lower level, technical characteristics are identified as the Essential Performance characteristics, then we may incorrectly fail a device, where safety is not truly impacted.  If true Essential Performance characteristics go unidentified and are not addressed or tested to a point there is a potential for harm, then testing would be ineffective and the device may be released with safety inadequately addressed. This puts a lot of selecting what the Essential Performance characteristic is correct.

What Does System Engineering tell us about determining Performance measures?

Performance measurement from a system engineering perspective may include the consideration of measures of effectiveness and measures of suitability. Measures of effectiveness are measures that correspond to the accomplishment of mission objectives and achievement of the desired results. These performance measures are determined through a capabilities-based assessment of the system and its objectives. Examples of these performance measures include the characteristic of measure of a system’s performance expressed as a speed, payload, range, time-on-station, frequency, or other distinctly quantifiable performance feature. Performance measures considering suitability include a performance measure of an item’s ability to be supported in its intended operational environment. Examples of these performance characteristics typically relate to readiness or operational availability, reliability, maintainability, or the item’s support structure.

The above characteristics of performance are different from technical performance measurements. Technical Performance Measurement (TPM) involves a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development based on current assessments of products lower in the system structure (per EIA-632 “Process for Engineering a System”) 1.

Manufacturers defining there performance characteristics need to determine which performance measures that correspond to mission objectives and suitability when defining Essential Performance and not fall into the trap of cycling through all technical performance measurements made on a device to determine what the essential performance is.

Particular (dash 2 Performance) Standards

As mentioned earlier, Particular standards include a list of Essential Performance characteristics in the standard. Looking at some particular standards for IEC60601-1 will help us understand what these performance characteristics look like. The Essential Performance characteristic may include areas such as the accuracy with which the equipment controls the delivery of energy or therapeutic treatment to the patient, as well as the devices ability to process and display physiological data that will affect patient care decision management.

The IEC60601-1 Particular Standards are available for some specific medical devices. These standards are represented by the IEC 60601-2-xx series, and are often referred to as “dash 2 standards.” Some examples of particular standards for IEC60601-1 are provided below. For those devices that do not have a dash 2 standards, the determination of Essential Performance is left to the manufacturer as defined in IEC60601-.

 

Examples of Essential Performance Characteristics from the Particular Standards

In addition to providing the Essential Performance characteristics for a medical device, the particular (dash 2) standards also include additional clauses that would replace or delete requirements that are called out in the general standard (IEC60601-1). The below table provides an example of some of the Essential Performance characteristics identified in a sampling of particular standards.

Table 1: Examples of EP items for some medical devices.

The definition of Essential Performance is defined per IEC60601-1 section 3.27 as:

“performance of a clinical function, other than that related to Basic Safety, where loss or degradation beyond the limits specified by the manufacturer results in an unacceptable Risk.

Note Essential Performance is most easily understood by considering whether its absence or degradation would result in an unacceptable Risk.”

In the review of table 1 above, it is clear that the actual acceptable performance levels are not prescribed, but left to the manufacturer. Additionally, the approach for how the characteristics are to be described across a sample of particular standards is not necessarily consistent. In some cases, the clinical function of the essential performance characteristic is not part of the specified item, as in “Alarm signals for high priority alarms” for infusion pumps. But, the consideration for what the actual acceptable performance level is as well as how these essential performance characteristics are to be applied while evaluating or testing the device is left to the discretion of the manufacturer. The note in the definition of 3.27 above does provide for how manufacturers should determine the specifics as well as what the acceptable performance ranges are. Once performance degrades to a degree that the risk is unacceptable (and harm can occur), this is when an unsafe condition or state has occurred (and therefore operating out of the determined range is unacceptable).

FDA Recognized Consensus Standards

The FDA recognizes standards that are deemed appropriate for manufacturers of medical devices to declare conformance with. The table above shows which of the example particular standards are FDA recognized consensus standards. The use of consensus standards to meet premarket submission requirements can help the FDA premarket review process although this is ultimately up to the submitter. Although the FDA reviewer may request the submitter to comply with a particular standard, as well as submit test data, as is the case with ISO 10933-1 for the Biological evaluation of medical devices. Additionally, if the standard is not FDA recognized, it doesn’t mean a reference to it or compliance to will not be considered acceptable by the FDA. The submitter can determine which standards the FDA suggests or recognizes by accessing the FDA website of recognized consensus standards, or determining if an FDA guidance is applicable that may also identify applicable standards.

From clearance standpoint, the FDA may not be as focused on identifying Essential Performance or testing Essential Performance, but they will be looking for the manufacturer demonstrate safety. The FDA has moved over the years to rely on standards, as well as identifying standards they recognize, but they always reserve the right to require more, regardless of what some of the IEC standards may require. In some cases, the FDA will put out FDA guidance documents that provide expectations on manufacturers for how performance is to be indicated and tested. The FDA has also indicated that risk can be used to determine what areas of a medical device require attention, but they have yet to say they are only interested in attending to only Essential Performance characteristics.

The next part of this article series will provide some examples of what has happened to the industry when Essential Performance is not correctly identified, as well as provide some recommendations for manufacturers. We are interested in your experience with Essential Performance and hope you will contact us and share them with us.

1 –  http://www.acqnotes.com/Attachments/EIA-632%20“Processes%20for%20Engineering%20a%20System”%207%20Jan%2099.pdf

Leave a Reply

Your email address will not be published. Required fields are marked *