Academy Comments to AHRQ re Improving Health Care Delivery

February 21, 2016

Doris Lefkowitz
Reports Clearance Officer,
Agency for Healthcare Research and Quality
540 Gaither Road
Rockville, MD 20850

Re: Clearance for the Collection of Data Through ACTION III Field-Based Investigations to Improve Health Care Delivery

Dear Ms. Lefkowitz,

The Academy of Nutrition and Dietetics (the "Academy") appreciates the opportunity to submit comments to the Agency for Healthcare Research and Quality (AHRQ) related to its open docket regarding "Clearance for the Collection of Data Through ACTION III Field-Based Investigations to Improve Health Care Delivery," published in the Federal Register on December 20, 2016. Representing over 100,000 registered dietitian nutritionists (RDNs),1 nutrition dietetic technicians, registered (NDTRs), and advanced-degree nutritionists, the Academy is the largest association of food and nutrition professionals in the United States and is committed to improving the nation's health through food and nutrition and to providing medical nutrition therapy (MNT) and innovative research to meet the health needs of all citizens.

As both a guideline development organization and a research organization, the Academy has substantive expertise in the study of dissemination, implementation, and quality improvement in health care. The data collection proposed by AHRQ will contribute additional knowledge to this developing field which will help to guide future research and practical application by our organization and others.

Drawing from our research in the area, we would suggest to AHRQ some considerations that may improve the quality and utility of data collected in relation to stakeholder groups that may not currently be included in recruitment plans and important methodological considerations in regards to the measurement of changes in practices. These are as follows:

A. Including Gatekeeper Stakeholders

Examples of Gatekeeper Stakeholders are Institutional Review Boards and Quality Control Researchers. The Academy recommends that the current proposed data collection effort could be improved through the inclusion of interviews with IRB administrators and committee members to better understand the roles that they play in quality improvement for a variety of healthcare settings and to anticipate potential barriers that could arise from these roles.

Institutional Review Boards (IRBs) often play a perceived or actual gatekeeper role for hospital quality improvement projects. The expertise of IRB members can be an invaluable asset to clinicians, managers, and quality improvement professionals to aid in the proper conduct of non-research quality improvement. However, a poor understanding of the relationship and delineation between research and quality improvement can lead well-intentioned clinicians into conflict with regulations for the protection of human research subjects. In one AHRQ-funded project (Pronovost et al., 2006), for example, individuals who appear to have intended to conduct a quality improvement project took several actions, including application for an exempt-research review from their IRB (Borror, 2007), which led them to be—in the judgment of the Office for Human Research Protections—firmly in the category of research and at odds with the Common Rule requirements (OHRP, 2008).

In other cases, researchers aiming to create data that would aid in quality improvement efforts, such as observational health services outcomes research, have found that inconsistencies between IRBs at different sites can delay, prevent, or significantly increase the administrative burden of projects (Green, Lowery, Kowalski, & Wyszewianski, 2006). In addition, we have documented from our own AHRQ-funded research that site IRBs can be a barrier to clinician recruitment in multi-site health services research (Rosa K. Hand et al., 2015). This is the rationale for our above recommendations on improvement for data collection involving stakeholders.

B. Measuring Clinician Practice Patterns

Changes in clinician practice patterns are a difficult-to-measure, yet vital, variable in the evaluation of quality improvement programs. Our research has demonstrated that self-report of practice compliance with recommendations is not sufficiently accurate to differentiate between programs that fail to change practices and practices that fail to improve patient health (Rosa K. Hand & Abram, 2016). Easily measurable practices such as computerized provider orders can be used to measure one aspect of clinician practice, but quality improvement often requires more nuanced changes in care and in-clinical decision making.

One method that we have developed for the field of nutrition care that discretizes the clinical care process into a series of connected steps, each of which can be described by standardized language (K. L. Thompson et al., 2015) and an automated process for evaluating the similarity of documented care with expectations (Murphy, Yadrick, & Hand, 2016). We have demonstrated the ability of this method to measure changes in practices in response to clinician training in the study of Evidence Based Practice Guideline dissemination and implementation (K. Thompson, Hand, & Swan, 2016). We recommend that your proposed data collection effort may be improved with special attention given to the tools used to measure practice changes and their validity and practicality.

C. Conclusion

The Academy appreciates the opportunity to comment on the data collection related to the ACTION III Field-Based Investigations to Improve Health Care Delivery. Please contact either Jeanne Blankenship at 312/899-1730 or by email at or William Murphy at 312/899-1778 or by email at with any questions or requests for additional information.


Jeanne Blankenship, MS, RDN
Vice President, Policy Initiatives and Advocacy
Academy of Nutrition and Dietetics

William Murphy, MS, RDN
Data Scientist
Academy of Nutrition and Dietetics

1 The Academy approved the optional use of the credential "registered dietitian nutritionist (RDN)" by "registered dietitians (RDs)" to more accurately convey who they are and what they do as the nation's food and nutrition experts. The RD and RDN credentials have identical meanings and legal trademark definitions.


Borror, K. C. (2007). Human Research Protections Under Federalwide Assurances FWA-5752, FWA-287, and FWA-3834.

Green, L. A., Lowery, J. C., Kowalski, C. P., & Wyszewianski, L. (2006). Impact of institutional review board practice variation on observational health services research. Health Services Research, 41(1), 214-230.

Hand, R. K., & Abram, J. K. (2016). Sense of Competence Impedes Uptake of New Academy Evidence-Based Practice Guidelines: Results of a Survey. Journal Of The Academy Of Nutrition And Dietetics, 116(4), 695-705. doi:10.1016/j.jand.2015.12.020

Hand, R. K., Ziegler, P., Wolfram, T., Kenne, D., Naragon, P., Steiber, A. L., & Fleming, M. (2015). Challenges in Recruiting for an Inter-Disciplinary PBRN Research Initiative: Implications for Research and Healthcare. Paper presented at the North American Primary Care Research Group Practice Based Research Network Meeting, Cancun, Mexico.

Murphy, W. J., Yadrick, M. M., & Hand, R. K. (2016). Validation of an Automated Process for the Comparison of Nutrition Care with Evidence-Based Nutrition Practice Guidelines. Paper presented at the AMIA Annual Symposium, Chicago, IL.

OHRP. (2008). OHRP Statement Regarding The New York Times Op-Ed Entitled "A Lifesaving Checklist." Department of Health and Human Services News Releases Website.

Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., . . . Goeschel, C. (2006). An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine, 355(26), 2725-2804.

Thompson, K., Hand, R. K., & Swan, W. (2016). Think you know the answers? Question your thinking about Evidence Based Practice Guidelines. Paper presented at the Food & Nutrition Conference & Expo, Boston, MA.

Thompson, K. L., Davidson, P., Swan, W. I., Hand, R. K., Rising, C., Dunn, A. V., . . . Murphy, W. J. (2015). Nutrition care process chains: the "missing link" between research and evidence-based practice. Journal Of The Academy Of Nutrition And Dietetics, 115(9), 1491-1498. doi:10.1016/j.jand.2015.04.014