ONC Guidance on Post-Certification and Surveillance
Perhaps lost amidst the waiting for the proposed ONC rule on information sharing and 21st Century Cures support was ONC releasing three very detailed guidance documents dealing with post-certification surveillance:
Overview Post-Certification Assessment
Disclosure of Material Information
Surveillance
You can find the three documents here identified as Program Policy Resource #18 dated October 5, 2018.
The three documents work together and provide clarity and insight into the more ONC regulation on post-certification surveillance from their 2015 Edition Rule and most recently their Enhanced Oversight Rule.
The Overview Post-Certification Assessment does just that and provides oversight of expectations of the developer and the ONC-ACB for activities and responsibilities occurring after initial certification and as the certified product is deployed to end-users. The other two document focus specifically on their respective areas of post-certification responsibility.
Disclosure of Material Information is the mandatory disclosures which all certified developer must display on their website and are reviewed by the ONC-ACB.
The Surveillance document speaks of the complaint process and pro-active surveillance done by ONC-ACBs for certified products deployed in the field.
In one light, these documents do not contain new information, at least to ONC-ACBs. The contain guidance ONC has shared directly or indirectly with ONC-ACBs and passed on to developers, or at least information being currently implemented. In particular, information within the disclosure of material information document has been well disseminated through the health IT community since ONC required this compliance for all 2014 Edition certified systems.
On the other hand, this level of details and clarity is very welcomed and helpful on an area that frankly is not always clear even to the ONC-ACBs. Also, while ONC-ACBs have heard this from ONC, many developers likely have not. It gives developers and ONC-ACBs something specific to work off of in implementing policy and procedures. The overview of post-certification assessment is probably the most helpful.
There are many important points of emphasis or clarifications made in these documents, but to highlight just a few…
It is more than passing the ONC test procedures
Maybe the most important takeaway for developers is that ONC explicitly recognizes that there is limitations in formal testing (i.e., testing with your ONC-ATL to achieve to your initial certification) in terms of assurance of full compliance to the criteria. ONC states:
The test procedures used in pre-certification testing are designed to reflect basic scenarios that health IT will be subjected to in the field. During testing, a product’s performance is evaluated under carefully-controlled conditions and includes only a limited range of fixed “test cases” or “scenarios,” pre-determined data and inputs, and well-defined sequences of steps, user actions, and events—all of which are described in test procedures and available to developers prior to testing. By contrast, certified products deployed in the field are ex-posed to a much greater depth and breadth of use cases, workflows, data, and other circum-stances, which are often complex and difficult to limit or control.
To put it more bluntly, you are judged not on passing the ONC test procedure but on fully complying with the ONC criteria in the various ways it was intended to be used by ONC. Thus, your product may not be compliant in the field even if it can replicate the outcomes in the ONC test procedures. Developers can no longer state “we were not tested on this” to excuse a complaint. The ONC-ACB can come back and essentially say “True, you were not tested on this by the ONC-ATL, but this non-compliance is covered by the ONC criteria which makes you responsible for it.”. Developers must think much more broadly how to implement the criteria.
Be aware of the appearance of interference
ONC really emphasized their concern on developer “interference”, such as disabling functions or capabilities in deployment or making them difficult to implement and perform due to technical or cost limitations. ACBs have wide latitude here to find a non-compliance, including as ONC states that “an ONC-ACB need not find that substantial interference with a required capability has occurred, but only that such interference is likely to occur.”
Training and usability are factors too
In the document, ONC notes an example where a developer can be at fault if they fail to provide “adequate technical or other support” so that “reasonable user” cannot preform the required capabilities or achieve the desired outcomes of the criteria. Also, usability is also factored into holding a developer responsible for a non-compliance. For example, “ONC-ACB would hold the developer responsible if the user did not populate required data fields due in part to usability or design deficiencies of the product (e.g., a user takes advantage of a shortcut to access a data entry field, but that field behaves differently depending on the page accessed immediately prior to entering the data).”
Surveillance and post-certification compliance is a huge point of emphasis with ONC, and I expect ONC-ACBs to focus on it even more in years ahead. If you want help on strategy and tactics on best dealing with these responsibilities, please reach out to Chart Lux Consulting.
