Colorado State Innovation Model Evaluation Quarterly Report January – March 2017
Department of Health Care Policy and Financing Purchasing and Contracting Services Section 1570 Grant Street Denver, CO 80203-1818
The project described was supported by Funding Opportunity Number CMS-1G1-14-001 from the U.S Department of Health and Human Services (HHS), Centers for Medicare & Medicaid Services (CMS). The Colorado State Innovation Model (SIM), a four-year initiative, is funded by up to $65 million from CMS. The content provided is solely the responsibility of the authors and does not necessarily represent the official views of HHS or any of its agencies.
Revision: July 6, 2017
4450 Arapahoe Avenue, Suite 100, Boulder, CO 80303
Table of Contents SIM Dashboard: Q1 2017 ................................................................................................................ 1 Colorado SIM Regional and Site Map ............................................................................................. 2 Colorado SIM TriCounty Health Department ................................................................................. 3 SIM Implementation Progress ........................................................................................................ 4 Key Informant Interviews ............................................................................................................... 5 Collaboration (Engagement) ..................................................................................................................... 6 Alignment .................................................................................................................................................. 7 Workforce ................................................................................................................................................. 8 Payment Reform ....................................................................................................................................... 9 Practice Transformation ......................................................................................................................... 10 Health Information Technology .............................................................................................................. 11
Performance Improvement Plan Progress.................................................................................... 14 Clinical Quality Measures.............................................................................................................. 16 Summary, Feedback, and Recommendations .............................................................................. 21 Response to Fourth Quarter 2016 (Y1 Q4) Findings ............................................................................... 21 Summary of First Quarter 2017 (Y2 Q1) Evaluation Findings ................................................................. 21 Summary and Recommendations from Key Informant Interviews ........................................................ 22 Ongoing and Future Evaluation Efforts................................................................................................... 24
SIM Dashboard: Q1 2017
1 Quarterly Report: January–March 2017
Colorado SIM Regional and Site Map
2 Quarterly Report: January–March 2017
Colorado SIM TriCounty Health Department
3 Quarterly Report: January–March 2017
SIMnImplementatio Pr ogress
SIM Implementation Progress The SIM Logic i Model , outlines critical resources, activties and out put s r equired to make progress on project outcomes and a to ultim tely accomplish the quadruple n aim. Below is a selectio , of r esources, i activties and out put s tha t have influ e nc ed pr oject progress during the January–March 2017greportin period. Notable successes or concerns are marked. e
Practic Transformatio n e 92 SIM practic
si t es n u contine par tici a tio in cohorte1.
0 practic
si t es withdrew during quarter 1 (Q1) of 2017.
71 of 92epractic
si t es reported on all required measures.
83 of 92 reported on at
least some of theprequired measures.
9 of 92epractic
si t es were unable to report on any measures. All cited challenges with electronic health records as the reason they were unable to report as expected this quarter. Data aggregatio n t ool (Stratus) procured for all cohorte1 practics . W i ll be a vailable in May 2017. New cost containment tool (MGMA DataDive) made available to all cohorte1 practics . Contractor Health Management Associates (HMA) was hired to help four community mental health centers (CMHCs/bi-directio n al health homes) establish a shared attr i but ionme thodology and n consistent data collectio g and r eportinS. First Multi-t akeholder Symposium was held. The SIM cohort 2 request for applicatio n w as released. A total ofe226 practic a s applied; finl sel ectioni s expected in July 2017.
Populatio n Health Twenty regional health connectors have been hired and began work to prioritize activs itie in the ir communities. ThenSIM populatio heal th workgroup is working with HMA to conduct an environmental scan and gap analysisnof populatio- based behavioral health initiatives in Colorado. TriWest is developing new plans to gather input from consumers. Previous plans hinged on the ability to collaborate with the CAHPS survey.
Payment Reform The payment reform work group structure has been d modifie t o address concerns that this workgroup was isolated from the efforts of other SIM workgroups. The payment reform work group members are joining other workgroups. Three payers have submitted data on alternative payment model n (APM) adoptio. Ot h ers are contin u i ng t o work toward CY2015 baseline data.
Health Informatio n Technology SIM has expanded its telehealth efforts,nand an actio- or i en ted n implementatio s trategy is expected to be completed during Q2 2017. SIM is collaboratin g wi th the Department of Health Care Policy and Financing (HCPF) and regional care collaborative organizatio n s to complete an environmental scan of HIT and health informatio n e xchange (HIE) access and use across the state. Colorado Telehealth Network reports expanding broadband capability to 86 primary e care practics across the state. One of these is a SIM cohorte1 practic. e
Eight Local Publics Health Authoritie and tw o Behavioral Health Transformatio n Col laboratives contin u e gr ant implementatio n ias pl anned. Activ n ties i ncl ude educ atio, community development, andm stiga rneductio e fforts.
The Governor’s Offic u c ontines its w ork to hire a data architect.uThis contined delay is affectin g pr ogress on SIM’s HIT goals.
Data Collectio & Anal y sis A SIM data hub was created on the SIM website to allow regular access to data for interested stakeholders. e TriWest completed a second round of key informant interviews with CHITAs and e Practic F acilitators. e TriWest developed a process to create the Coordinated Community Systems Index which will be used as one statewide measure of SIM impact. The process will be piloted in Q2 2017. n 2015 Baseline AHRQ Access to Care measures were calculated by TriWest.
Practic si t es completed a follow-up ofe their practic improvement plans (PIPs), which reports on baseline goal progress. Some sites set new goals. The university completed a summary analysis of progress on PIP goals. Practic si t es completed a follow-up of the Data Quality Assessment. 2015 Baseline Proxy CQMs were generated by the Center for Improving Value in Health Care (CIVHC). 2015 Baseline Cost and i Utilnz atio m e asur es were calculated by Milliman.
4 Quarterly Report: January–March 2017
SIM Offic W ork Two new staff m e mb er s joined the SIM team: an administratio n and c ontracts program assistant and a data strategy coordinator. Podcast series n “Innovatio Insights” was launched. e
Key Informant Interviews The second round of key informant interviews focused on collecting feedback from Practice Facilitators (PFs) and Clinical Health Information Technology Advisors (CHITAs). A stratified random sample of practice sites was selected, and the PFs and CHITAs from those sites were contacted to complete key informant interviews. Practice Transformation Organizations (PTOs) for the selected sites were reviewed, and the sample of sites was supplemented to include all Practice Transformation Organizations (PTOs). A total of 30 interviews were conducted, and these interviews included 37 key informants. The interviews occurred during the last week of February 2017 and the first three weeks of March 2017. Since individual responses were confidential, they were combined with responses from other key informants asked the same question(s) to provide content for this report. A protocol was developed for the interviews that drew on previously determined evaluation questions from six content areas that comprise the statewide evaluation of SIM. These content areas are collaboration (engagement), alignment, workforce, payment reform, practice transformation, and health information technology. Twenty-two PFs were asked questions from all content areas except health information technology (HIT). Seventeen CHITAs were asked a subset of practice transformation questions and the HIT questions. Two interviewees served in both roles and were asked questions from all content areas. PFs represented a total of 59 practice sites and one Community Mental Health Center (CMHC) program, and CHITAs represented a total of 75 practice sites and all four CMHC programs.
SIM Key Informant Interview Summary Type of Interviewees
Number
Practice Sites
CMHCs
Practice Facilitator
22
59
1
Clinical Health Information Technology Advisor
17
75
4
Detailed presentation of data is provided in the May 2017 Key Informant Interviews report, “Content Area Summaries” section. This summary features highlights from each content area described in that report and presents some common themes across content areas. These themes include the following: Preparation has progressed for cohort 2 in developing materials for moving towards alternative payment models (APMs). PFs for cohort 1 sites see the need for guidelines to help sites progress towards APMs. Furthermore, many PFs suggested areas of focus such as identifying what has worked for successful sites. Other suggestions included focusing on workflow, EHR, and other program and reporting requirements for APMs. In addition, cohort 1 PFs recommended a demonstration showing the return on investments in practice site efforts to move toward APMs.
5 Quarterly Report: January–March 2017
Practice sites that participate in SIM and other practice transformation efforts often work with more than one PF. The degree to which PFs coordinate activities can be helpful to the sites. When multiple initiative activities are not coordinated, the experience can be confusing and counterproductive. There is an opportunity to improve practice transformation successes through improved coordination at the PTO, PF, and CHITA levels, such as was done by SIM in preparation for Comprehensive Primary Care Plus (CPC+) and the cohort 2 applications. There was also the suggestion to assess each practice site when it enters SIM to determine whether both a PF and a CHITA are necessary. Depending on the practice site, one person might be able to provide all needed support. This discussion also raised the question about the equity of compensation for PFs and CHITAs, given the greater technical skill needed for CHITAs.
Many practice sites already had integrated behavioral health providers. However, limited financial support for sites to establish and sustain the behavioral health component bars integration and raises concerns for practice sites that do not have resources to meet that need.
Many helpful comments were provided to questions about the needs of sites for health information technology support, health information exchange connectivity, and better access to data from SPLIT and EHRs. There has been improvement in the use of health information technology such as in data quality and the use of EHRs. However, more support is needed to make changes to EHRs. Respondents pointed out that workflows are an area where a more coordinated effort would be beneficial, involving vendors and practice site staff.
Collaboration (Engagement) This content area contained summaries of responses from PFs to rating and open-ended questions about practice and consumer engagement. The ratings of the extent to which practices have engaged in SIM were split between “Moderately Engaged” (54.5%) and “Very Engaged” (45.5%). The level of engagement was contingent on several site-dependent factors such as leadership engagement, site ability to focus resources on SIM (not having competing priorities), and presence of a behavioral health provider in the clinic. Also, practice sites with small grants were mentioned as being more engaged.
6 Quarterly Report: January–March 2017
Practice Engagement in SIM Very (3)
45.5%
Moderately (2)
54.5%
Slightly (1) Not Engaged (0) 0%
20%
40%
60%
80%
100%
Challenges mentioned included funding limitations, difficulties implementing the workflows that accompany Clinical Quality Measures (CQMs) (such as a process for addressing positive screens), staffing for the behavioral health focus, and potential coordination complications for sites within systems. Efforts focusing on patient engagement ranged from surveying patient satisfaction—and using these findings in quality improvement efforts (developing better ways to communicate with patients such as through EHRs)—to establishing a patient/family council. There was also more focus on engaging patients with behavioral health needs.
Alignment This content area captured responses from PF interviewees to questions about how SIM aligns with other transformation initiatives and how that level of alignment impacts SIM. PFs said that 42 of the 59 practices they work with (about 72%) also have participated in or are participating in other transformation efforts. The three initiatives most often mentioned were EvidenceNow Southwest (ENSW), Comprehensive Primary Care (CPC), and Comprehensive Primary Care Plus (CPC+). SIM practice alignment with other initiatives was rated as “Moderately” aligned with an average rating of 2.0 on a scale ranging from 0 to 3.
SIM Alignment With Other Initiatives Very (3)
10.5%
Moderately (2)
78.9%
Slightly (1)
10.5%
Not Aligned (0) 0%
20%
40%
60%
80%
100%
The flexibility of practice milestone activity choices was credited with practice sites’ abilities to mesh with other initiatives with milestone requirements, such as CPC, EvidenceNow Southwest (ENSW), and Transforming Clinical Practice Initiative (TCPi). 7 Quarterly Report: January–March 2017
PFs recognize the diversity of initiatives and attempt to minimize duplication of efforts where possible, especially since duplication adds to practice site workload. SIM Office efforts with cohort 2 will help address these issues. About 41% of PF respondents indicated that their practice sites have more than one additional Practice Facilitator working with them—about 50% of the 59 practice sites these PFs work with. In these situations, the SIM PF tries to coordinate when possible. When coordination is not possible, sites can experience confusion due to the different demands from each PF.
Workforce The Workforce content area captured responses from PFs to questions about meeting workforce needs and identifying gaps, about the inclusion of behavioral health care providers, and about provider satisfaction with integrated healthcare. PFs rated practice site preparation to meet workforce needs for successful integrated care at 2.2 (scale ranging from 0 to 3). The extent to which workforce gaps have been addressed was rated lower at 1.4.
Prepared to Meet Workforce Needs for Integrated Care Delivery Very (3)
47.6%
Moderately (2)
33.3%
Slightly (1)
9.5%
Not Prepared (0)
9.5% 0%
20%
40%
60%
80%
100%
There is some funding for behavioral health providers through the small grants, payers, and Regional Care Collaborative Organizations (RCCOs), but those funds do not fully meet the gaps left by services that are non-billable, for example, or the financial support needed for behavioral health providers. Providers like integrated care and having behavioral health providers involved in patient care. However, frustration remains concerning the lack of resources for behavioral health care and the barriers to integrating that care, such as difficulties sharing information. Behavioral health care is being integrated differently across the practice sites, depending on the site, patients, community resources, and provider availability. That variability creates challenges for receiving payment for behavioral health services that must be navigated in different ways.
8 Quarterly Report: January–March 2017
Payment Reform The payment reform content area captured ratings, comments, and responses from PF interviewees about adopting alternative payment models and providing support in that process. Respondents rated practice sites’ preparation for adopting APMs at a level between “Slightly Prepared” and “Moderately Prepared” (average rating of 1.4 on the 0 to 3 scale). Respondents viewed practice site preparation as highly variable with some sites being sophisticated (including having the necessary HIT support) and other sites having steeper learning curves.
Practice Preparation for APMs Very (3)
15.0%
Moderately (2)
35.0%
Slightly (1)
30.0%
Not Prepared (0)
20.0% 0%
20%
40%
60%
80%
100%
Many practice sites face costly EHR changes to add reporting and data-sharing functionality, not only for site functioning but because these are important for alternative payment models. The EHR change process can also be complicated by vender limitations and timelines. Some practice sites are waiting for clear data demonstrating positive return on the investment before expending resources in the move to APMs. Clear communication from payers is also needed to define how payments will be determined to cover services needed to move forward with new APMs. One particular question involves defining a payment mechanism for specific kinds of services such as care management. There is no easy one-size-fits-all “to-do check list” ensuring payment reform will happen if certain tasks are completed. There can be many pathways to payment reform, which can result in confusion, and practice sites can struggle to find their way forward. The work currently being done around payment reform to clarify how to qualify for and to implement value-based payment models with each payer will help. However, sites will need assistance to work through those requirements. Guidelines could be developed based on practice site attributes and what has worked for other similar sites to move to an alternative payment model. The new milestones adopted by the payers will help clarify requirements for APMs. MACRA Medicare reform could help adult practices by moving towards alternative payment models. However, pediatric practice sites will not benefit directly from the focus on Medicare reform.
9 Quarterly Report: January–March 2017
PFs are helping practice sites with integration activities that will position sites for participation in APMs, including establishing workflows, data management and reporting, etc. However, many PFs expressed needs for additional training, materials, and tools to support practice site understanding and adoption of APMs. PFs provided important feedback about the sustainability of the integration of primary and behavioral health care. About one third said they did not know at this time if integration is sustainable, explaining that, for integration to be sustainable, program changes were needed first before focusing on APMs. Those who agreed with sustainability felt there are changes that need to be made before that will happen. The remaining respondents were skeptical that payment systems will actually change. The MGMA DataDive tool is not commonly used at this point. Practice sites that use it find it very helpful. However, many sites are focused on other priorities and could use more support in the benefits of the tool and how to use it to meet their needs.
Practice Transformation The practice transformation content area captured responses from PF interviewees about preparation for and implementing integrated care, reaching out to behavioral health providers, and making progress on integrated care building blocks and activities. CHITAs were also asked to respond to the subset of the Practice Transformation questions addressing preparedness. Practice preparedness for integrated care was rated at 2.2, on average, just above “Moderately Prepared.” There were many sites that already had experience with practice transformation and integrated care prior to SIM.
Practices Prepared for Integration of Primary and Behavioral Health Care? Very Prepared (3)
37.9%
Moderately (2)
51.7%
Slightly (1)
6.9%
Not Prepared (0)
3.4% 0%
20%
40%
60%
80%
100%
Flexibility in the SIM implementation was considered to result in too little guidance for some practice sites without prior transformation experience. Those sites would have benefited from a more structured approach. More structure will be available with the new milestones and guidance for cohort 2. Difficulties in sharing information between primary and behavioral health providers was reiterated as a barrier to integration. Regional Care Collaborative Organization efforts appear to be helping in this area.
10 Quarterly Report: January–March 2017
Improvement in access to behavioral health care providers was rated lower than other areas at 1.3 on average, at about a “Slightly” improved level. As noted above, practice sites that did not already have access to behavioral health providers are experiencing many barriers, such as funding and availability. PFs rated the value of implementation feedback at 1.0 (“Some”). They see value in implementation feedback but state that it should be provided sooner. Changes to the Shared Practice Learning and Improvement Tool (SPLIT) will facilitate this. The process of completing the assessments can provide important feedback, especially from the Medical Home Practice Monitor. PFs sometimes do not get feedback when it goes directly to the practice sites. Practice sites were rated as making “Moderate” progress moving through the building blocks (average rating of 2.0 on a 0 to 3 scale). Progress relates primarily to Practice Improvement Plan goals developed from the Milestone Activity Inventory.
Health Information Technology The Health Information Technology content area captured ratings, comments, and responses from Clinical Health Information Technology Advisors (CHITAs) interviewees about practice preparation for and barriers to HIT improvement and support for practices, data quality and integration, and the use of technology. Themes from HIT ratings, responses and comments are grouped below.
Readiness CHITAs rated practice sites as between “Moderately” and “Very” prepared to engage in HIT improvement efforts (average rating of 2.5). Progress in reaching HIT-related milestones was also rated at 2.5 on average with 64.3% of raters saying a “great deal” of progress has been made.
Progress in Reaching HIT-Related Milestones Great Deal (3)
64.3%
Moderate (2)
21.4%
Some (1)
14.3%
No Progress(0) 0%
20%
40%
60%
80%
100%
Reported causes of what helped practices to be prepared were the following: engaged staff and physician buy-in, past experience with practice transformation efforts, participation in internal Quality Improvement (QI) projects that predate SIM participation, level of staff’s abilities and EHR’s capabilities, responsive and engaged vendors, strong IT infrastructure, and time given for planning and analyzing.
11 Quarterly Report: January–March 2017
EHR Reported issues that kept practice sites from being very prepared were as follows: valid data; vendor receptiveness to custom reporting and/or giving access to analytics; use of a separate registry/database for some measures; slower EHR vendor response to requests; unidentified and/or highly complicated workflows that require significant effort to track; and known errors in the data, the reports, and/or how data is pulled from the EHR. SIM’s focus on data quality has led to more effective use of EHRs, particularly for practice sites that were not previously advanced users of technology. Resources for making EHR changes are challenging to obtain. Transitions to new EHRs are challenging. Issues of vendor support to the sites can be barriers to data capture and quality. An example challenge is the need for workflows/flowcharts that detail the steps necessary to capture data in the EHR for populating reports. Practice sites need the workflows from vendors, and when there are discrepancies between how a CQM is defined and how the EHR pulls the data, vendor support is also needed for correcting the workflow. It would be helpful for practice sites to have a data extract tool so they would not have to rely on vendors to make changes and report on the new measures. More vendor and staff involvement, coordination, and training is needed to ensure practice site staff are familiar with workflows, using workflows and EHRs correctly, and understanding how data entered on the front end relates to CQMs and other reporting on the back end.
Integration Ratings of the extent to which data are integrated across primary and behavioral health care averaged 2.2 on the 0 to 3 scale. About 78% of the ratings fell into one of the two highest levels: either “Integrated” (42.2%) or “Beginning to Integrate” (35.7%). Practice sites see integrated care as a value for their practices and recognize there are barriers (such as to sharing data) that will have to be resolved to be successful. Improvement in Health Information Exchange (HIE) connectivity was rated at 1.7 on average: between “Somewhat” and “Moderately.” However, 20% rated this area as “Not Improved.” Practices that have HIE connectivity see the data sharing benefits, such as with improved access to behavioral health data. Sites that do not have it are concerned about the additional costs. The CHITAs who responded about the Stratus data aggregation tool said they do not use it. One problem mentioned was that the data are too old because they use claims that are six months old. However, Stratus is just being rolled out to SIM sites, and PFs (not CHITAs) get the license to support practices. Access would not have been available until May 2017, and few respondents would have had the opportunity to work with Stratus at the time of the interviews.
12 Quarterly Report: January–March 2017
Improvement Improvement in the use of technology was rated at 2.1 on average—at about the “Moderately” level. However, 42.9% of respondents said the use of technology was “Very Improved.” Some practice sites were struggling with non-standard CQM measures, and when the SIM office adjusted the CQMs by consolidating some measures and dropping others, sites moved into better positions for achieving their goals. The change to a 12-month reporting period has also helped with data quality and the reporting and use of data. CHITAs would like access to SPLIT data to provide feedback to the practice sites. Some CHITAs use SPLIT data for feedback, but most are not using data from SPLIT for rapid cycle feedback and quality improvement purposes. They use the data that go into SPLIT, but they tend to use inconsistent and individual approaches to providing data feedback to the practice sites. These approaches sometimes use practice site EHRs to get data or may use reporting tools available at the practice site level, such as using Excel spreadsheets. Many CHITAs expressed the need for additional training, materials, and tools. There have been some presentations, such as at the Collaborative Learning Sessions, and SIM has been responsive in providing information, particularly around the CQMs. More training about chronic disease support, integration and behavioral health, billing and payment changes, and telehealth would be helpful. Respondents suggested that asking more experienced CHITAs to share their insights into their SIM work could be helpful. Most CHITAs reported that telehealth is not being used.
13 Quarterly Report: January–March 2017
Performance Improvement Plan Progress As reported in the Rapid Cycle Feedback and Quarterly Report July–September 2016, once practice sites completed initial assessment tools, they used that information to develop a baseline Practice Improvement Plan (PIP). SIM requires sites to select at least one primary goal from within each of the three SIM program focus areas: Practice Transformation Behavioral Health Health Information Technology (HIT) Sites could also choose an additional goal from each of these focus areas. During the first months of 2017, practice sites completed follow-up PIPs. During these follow-up assessments, sites reported progress made toward initial baseline goals and, in some cases, established new ones. Progress is presented here for PIP goals for the 92 sites remaining in SIM after the first year. In addition to specifying a primary goal in these focus areas, some sites identified additional goals: 33 sites selected additional goals in practice transformation, 29 in behavioral health, and 16 in health information technology. The chart on the following page shows progress, at the six-month follow-up, on the primary baseline goals chosen by all sites. Achievement of goals was around 30%, with HIT goal achievement slightly higher than practice transformation or behavioral health goals. Continued work in the three focus areas is also similar between areas at around 60%, with practice transformation goals continuing for about 62% of practice sites.
14 Quarterly Report: January–March 2017
Progress on Primary Baseline Practice Improvement Goals 90% 80% 62.0% 59.8%
70%
57.6%
60% 50% 40%
33.7% 28.3% 26.1%
30% 20%
7.6% 5.4% 5.4%
10%
8.7% 2.2%
3.3%
0%
Did not formally work on goal
Stopped work before achieving goal
Practice Transformation
Started and continue to work on goal
Behavioral Health
Worked on and achieved goal
Health IT
The chart below shows progress on the additional baseline goals chosen by sites. Achievement of these goals was more variable, with practice transformation goals achieved for more (42.4%) of the 33 practices that identified an additional goal. HIT goals were achieved for a smaller percentage of the 16 practices (18.8%) that identified additional HIT goals. Additional HIT goals were the most likely not to be worked on at 25%. The percent of practices that stopped work before achieving the goal is very small with work continuing in all three goal areas.
Progress on Additional Baseline Practice Improvement Goals 90% 80% 70%
56.3% 55.2% 48.5%
60% 50%
42.4%
40%
31.0% 25.0%
30%
10%
18.8%
10.3%
20% 6.1%
3.4% 3.0%
0.0%
0%
Did not formally work on goal
Stopped work before achieving goal
Practice Transformation
Started and continue to work on goal
Behavioral Health
15 Quarterly Report: January–March 2017
Worked on and achieved goal
Health IT
Clinical Quality Measures Starting in the first quarter of 2017 (Y2 Q1), practice sites reported CQM values on either a trailing year (e.g., Y1 Q2–Y2 Q1) or year-to-date basis, with trailing year as the preferred method. With the switch to trailing year reporting, we anticipate greater stability in the quarter-to-quarter CQM average results. In the following two tables, each column represents the total reported percentages of each CQM for that quarter. This percentage is determined by summing the reported numerators and denominators from each practice site reporting the CQM. For example, the “Hypertension” CQM was reported by 60 practice sites in Y2 Q1, and the sum of all reported numerators from these sites was 39,201. The sum of all reported denominators was 63,491. By dividing the summed numerator by the summed denominator, we calculated the Y2 Q1 “Hypertension” percentage of 61.72%. Note that Y1 Q1 was a trial period and may be inconsistent with reported results from other quarters. Starting in Y2 Q1, we are reporting four additional measures not previously included in quarterly rapid cycle reports. These are “Substance Use Disorder: Alcohol and Other Drug Dependence,” “Substance Use Disorder: Tobacco,” “Substance Use Disorder: Alcohol,” and “Development Screening,” all of which now have measures that align with other programs (CPC+, QPP) or did not previously have developed measures. Another adjustment in this quarter’s reporting is a change from the measure “Asthma: Use of Appropriate Medications for Asthma” to “Asthma: Medication Management for People with Asthma,” which aligns with the MACRA Quality Payment Program. This change is reflected in both adult and pediatric measure sets.
Adult Measure Set (items with an * indicate alignment with CPC+) Primary CQMs
Y1 Q1 %
Y1 Q2 %
Y1 Q3 %
Y1 Q4 %
Y2 Q1 %
48.03% 32.21% 61.13%
34.85% -
39.02% 34.54% 67.15%
50.17% 33.10% 66.75%
50.62% 38.47% 61.72%
-
-
-
-
9.72%
54.50%
50.20%
46.78%
53.60%
88.35% 50.05%
Asthma: Medication Management Fall Safety* 45.59% Maternal Depression 16.36% Substance Use Disorder: Alcohol CPC+ CQM (differs from Adult Measure Set)
0.00% -
38.97% 74.62% -
60.84% 21.62% -
47.40% 55.24% 42.19% 87.50%
-
-
-
0.38%
1
Depression Diabetes: Hemoglobin A1c* Hypertension* Substance Use Disorder: Alcohol and Other Drug Dependence* Substance Use Disorder: Tobacco* Obesity: Adult Secondary CQMs
Depression Remission*
-
1
The full scope and title for the “Depression” CQM is “Preventive Care and Screening: Screening for Clinical Depression and Follow-up Plan.”
16 Quarterly Report: January–March 2017
Pediatric Measure Set Primary CQMs Depression2 Development Screening Maternal Depression Obesity: Adolescent Obesity: Adolescent (Activity Counsel) Obesity: Adolescent (Diet Counsel) Secondary CQMs
Y1 Q1 % 16.08% 21.29% 54.60%
Y1 Q2 % 18.61% 44.50% 60.80%
Y1 Q3 % 34.96% 72.45% 75.78%
Y1 Q4 % 43.50% 74.41% 92.74%
Y2 Q1% 33.28% 73.49% 45.77% 83.96%
30.45%
44.02%
44.76%
59.03%
42.36%
30.62%
44.41%
47.64%
61.50%
47.84%
Asthma: Medication Management
-
-
-
-
72.96%
The data for Y2 Q1 is still considered preliminary. As future quarter information is submitted as the year progresses, data quality issues are discovered and corrected. As such, confidence in data is much higher following a full calendar year of data submission and verification. It is important to take this process into consideration when interpreting the data for Y2 Q1 and forward. In comparing changes over time, increases in the reported values generally indicate improvement. However, the “Diabetes: Hemoglobin A1c” measure counts the number of patients whose A1c levels are not under control. Therefore, a higher value on this measure indicates a poorer health outcome. Of the four primary adult measures reported in Y1 Q4, only one, “Depression,”2 showed a positive change from Y1 Q4 to Y2 Q1. Of the secondary measures, one half showed decreases. On the other hand, all primary pediatric measures showed decreases, Through Y1 Q4, we observed an increasing trend in most CQM measures. This increasing trend was not apparent n Y2 Q1, with significant declines for some measures. This decline in CQMs is driven by several data consistency/quality factors. One factor is that different practice sites reported on specific CQMS in Y2 Q1 versus Y1 Q4. In the case of “Depression,” there was a decline in the number of pediatric practice sites reporting, from 20 to 19. The aggregate of pediatric sites changed for “Depression,” decreasing from 43.50% in Y1 Q4 to 33.28% in Y2 Q1. But for practice sites reporting the adult measure set, those reporting the “Depression” CQM declined from 65 to 57. This resulted in the measure increasing from 50.17% to 50.62%, the opposite effect from the pediatric counterpart. Additionally, sites and their CHITAs reported that data quality improves over time across the calendar year, with Q4 data being most reliable.3 This pattern does not hold true for the other CQMs with a large decline, notably “Maternal Depression” screening in pediatric practice sites, which declined from 74.41% in Y1 Q4 to 45.77% in Y2 Q1. The 2
The full scope and title for the “Depression” CQM is “Preventive Care and Screening: Screening for Clinical Depression and Follow-up Plan.” 3 Given this variance, caution should be exercised in making comparisons, especially with Q1 findings. 17 Quarterly Report: January–March 2017
number of pediatric practices sites reporting this measure declined from 13 to 12. However, this measure presents other problems among reporting adult practice sites in which “Maternal Depression” is a secondary measure. The number of reporting sites during the same Y1 Q4–Y2 Q1 period was 3, a total too small for confidently identifying trends in the larger cohort. The next table displays the number of sites reporting CQM measure by quarter following the same format used in previous quarterly reports. We reported in the Y1 Q4 quarterly report that the data available did not allow us to determine whether changes in an aggregate measure from quarter to quarter were caused by changes in each practice site’s results or changes in which sites reported. We are now receiving data at the individual practice site level, and in future quarters we will report the number of sites that had increases or decreases in CQM levels and how many were making initial reports for each measure.
Practice CQM Reporting Totals by Quarter CQM Measure
Y1 Q1
Y1 Q2
Y1 Q3
Y1 Q4
Y2 Q1
Adult Pediatric Adult Pediatric Adult Pediatric Adult Pediatric Adult Pediatric
Asthma: Medication Management Depression Depression Remission* Development Screening Diabetes: Hemoglobin A1c* Fall Safety* Hypertension* Maternal Depression Obesity: Adolescent (Activity Counsel) Obesity: Adolescent (Diet Counsel) Obesity: Adolescent (Screen) Obesity: Adult Substance Use Disorder: Alcohol and Other Drug Dependence* Substance Use Disorder: Tobacco* Substance Use Disorder: Alcohol
-
-
-
-
-
-
-
-
24
17
45
10
61
11
63
12
65
20
57
19
-
-
-
-
-
-
-
-
5
-
-
-
-
-
-
-
-
-
1
13
15
-
0
-
33
-
62
-
56
-
10 14 4
4
0 0 2
6
24 46 4
9
25 56 3
13
30 61 3
12
2
14
4
18
4
20
2
21
6
20
2
14
5
18
4
20
2
21
6
20
3
15
4
22
4
23
2
22
6
20
43
-
67
-
66
-
65
-
48
-
-
-
-
-
-
-
-
-
16
-
-
-
-
-
-
-
-
-
52
1
-
-
-
-
-
-
-
-
1
-
18 Quarterly Report: January–March 2017
In addition to cohort averages by quarter and the number of practice sites reporting, we also report the distribution of practice site CQMs for the most recent quarter. This distribution may be helpful to sites for comparing their own CQM results to those of other practices sites in the cohort.
CQM Year 2 Quarter 1 Distribution CQM Categories Asthma: Medication Management Depression
Sites Data Not Reporting Zero
Mean Value
Median Minimum Maximum Value Value Value
Cohort Mean
41
40
56.37%
43.67%
0.00%
100.00%
61.93%
76
75
43.35%
44.23%
0.00%
96.81%
48.77%
5
4
0.96%
0.00%
0.00%
3.85%
0.38%
14
14
71.34%
71.02%
43.36%
98.94%
73.49%
56
55
43.13%
41.51%
9.84%
100.00%
38.77%
30
29
45.65%
44.31%
0.00%
99.48%
52.46%
Hypertension*
61
60
57.72%
62.50%
0.00%
87.24%
61.74%
Maternal Depression
15
13
49.76%
61.33%
0.90%
100.00%
45.75%
Obesity: Adolescent (Activity Counsel)
26
25
82.38%
86.12%
45.56%
100.00%
84.03%
26
25
45.41%
37.71%
0.00%
98.11%
47.83%
26
25
42.14%
33.33%
0.00%
98.17%
42.42%
48
45
42.38%
45.41%
0.00%
96.23%
49.23%
16
16
9.62%
0.00%
0.00%
87.03%
9.72%
53
52
74.78%
91.84%
0.00%
100.00%
88.35%
1
1
87.50%
87.50%
87.50%
87.50%
87.50%
Depression Remission* Development Screening Diabetes: Hemoglobin A1c* Fall Safety*
Obesity: Adolescent (Diet Counsel) Obesity: Adolescent (Screen) Obesity: Adult Substance Use Disorder: Alcohol and Other Drug Dependence* Substance Use Disorder: Tobacco* Substance Use Disorder: Alcohol
The “Data Not Zero” column lists the number of practice sites that do not have a zero in the measure’s denominator. Zeros in denominators correspond to practice sites that had no patients eligible for inclusion in the CQM measure (e.g., the CQM “Obesity: Adult” in a pediatric site). Any practice site that reported zero for both numerator and denominator was excluded in the calculation of the CQM’s mean value. In the case of “Asthma: Medication Management,” one practice site reported a zero in the numerator and denominator, indicating no patients eligible for inclusion in the CQM. The mean practice site value for “Asthma: Medication Management” was calculated by adding the CQM values of the 40 practice sites with patients meeting the eligibility criteria, then dividing by 40, not 41. One practice site had eligible patients, but none were dispensed appropriate mediation and remained on the medication 19 Quarterly Report: January–March 2017
for a majority of their treatment period, resulting in a zero for the numerator, and a minimum value of 0.00%. The Cohort mean was calculated by dividing the sum of all numerators by the sum of all denominators. This distributional information demonstrates that CQM values change dramatically from practice site to practice site. For the 40 sites that reported “Asthma: Medication Management” values (with non-zero denominators), the range of CQM scores varied from none of the eligible patients receiving appropriate medication (0%) to all of the eligible patients receiving medication (100%). A measure of practice transformation success will be both an increase in the cohort and individual practice site mean values and a decrease in the number of practice sites reporting low or zero CQM scores.
20 Quarterly Report: January–March 2017
Summary, Feedback, and Recommendations During Y2 Q1, SIM evaluation activities focused on finishing the second round of Key Informant Interviews, developing a Coordinated Community Systems Index (CCSI), identifying sites and creating a framework to conduct case studies with six cohort 1 practices, and continuing regular tasks such as monitoring quarterly data collection of CQMs and holding meetings with SIM program staff, vendor partners, the Evaluation Workgroup, and other workgroups as needed.
Response to Fourth Quarter 2016 (Y1 Q4) Findings The SIM office identified several major takeaways in the Y1 Q4 report findings and is working with partners to incorporate those learnings into program implementation. These activities fall into three main categories: information sharing, HIT support, and other practice transformation support. Information Sharing: The SIM office is working to share monthly updates and quarterly rapid cycle feedback reports with stakeholders. It is also developing a strategy for systematically bringing findings to workgroups. Health Information Technology/Data Reporting Support: SIM has simplified and better aligned CQMs and has provided clearer reporting guidance. In addition, the SIM office is working to develop a strategy to enhance CHITA support for practice sites. This strategy includes holding a CHITA webinar to discuss EHR vendor challenges, planning EHR affinity groups, and conducting a more in-depth HIT assessment to better understand HIT and eCQM reporting capabilities. Other Practice Transformation Support: The SIM office is encouraging cohort 1 practices to align with cohort 2 expectations, defining “sufficient progress” along milestones and developing an approach to analyzing practice vignettes and rapid cycle reports to surface best practices for sharing with all sites.
Summary of First Quarter 2017 (Y2 Q1) Evaluation Findings The main theme emerging from this Y2 Q1 rapid cycle report is that practice sites are beginning to demonstrate progress in their practice transformation efforts. While much work remains, some sites report that they have achieved early practice transformation goals set in their practice improvement plans. In addition, data quality and reporting of CQMs continues to improve. Highlights of these improvements include the following: Approximately 30% of practice sites have reported achieving PIP goals in the areas of practice transformation, behavioral health, and HIT. Most sites continue toward their original goals, and very few practices have reported stopping work on their initial goals before achieving them. The number of sites reporting on all required CQMs increased from 61 to 71 practice sites during the quarter.
21 Quarterly Report: January–March 2017
Interviews with PFs and CHITAs suggest several opportunities to support sites as they continue their transformation efforts. These suggestions have been organized below based on the SIM workgroup structure in order to better facilitate workgroup engagement with evaluation findings.
Summary and Recommendations from Key Informant Interviews SIM Office and Technical Assistance Recommendations Alignment: Practice sites typically absorb many different transformation efforts simultaneously, and occasionally the help they receive from various sources is difficult to coordinate and align. It might be more efficient and productive for PFs, CHITAs, and other consultation/TA providers to develop a statewide agreement on how they will coordinate their efforts at sites that have more than one helping agent. The University of Colorado team is in the process of developing a “care plan” approach to help coordinate these efforts.
Workforce Summary and Recommendations As is the case in most states nationwide, there is a need to fill workforce gaps. Indeed, integrated care holds some promise for helping fill some of the gaps. However, in order to achieve that promise, embedded behavioral health specialists need to not only take “warm hand-offs” and see patients; they also need to help equip all primary care staff with ever-expanding skills and needed confidence to treat patients with co-occurring primary and behavioral health conditions. PFs can help sites develop financially sustainable plans for ensuring that behavioral health specialists’ roles are not limited to direct care. Rather, these specialists can increase the sites’ overall integrated behavioral health capacity as well as the range of treatments for which they might bill. The Workforce Workgroup could collaborate with health and behavioral health professional training programs in Colorado to identify emerging training models and to document exemplary programs nationally, such as the Integrated Behavioral Health training program for family medicine residents through the University of Texas Rio Grande Valley. One option may be a symposium or day-long conference in which Colorado-based models, particularly those for rural or underserved areas, and those from other states might be shared. Healthy competition within Colorado’s higher education community might be furthered through such efforts.
Payment Reform Summary and Recommendations The Practice Transformation Workgroup and representative PFs could develop an updated set of tools and training materials for practice sites that need greater preparation for APMs. Further, improvements could be informed by tapping the considerable expertise available within Colorado and by examining emerging APM models in other states.
22 Quarterly Report: January–March 2017
Practice Transformation Summary and Recommendations Collaboration: Overall, practice sites have been engaged in SIM. However, findings suggest that PFs and CHITAs should focus more intently on areas of particularly acute need, such as integrated care workflows (especially as they relate to CQMs) and sustainable financing of integrated behavioral health (IBH).4 Even more targeted focus on key areas of concern to practice sites can help maintain or engender high levels of engagement and more successful implementation. Interviewees’ responses were helpful in identifying several different issues, which, if addressed carefully, will lead to better implementation approaches. First, guidance concerning SIM implementation needs to be tailored to the site. Individualizing their technical assistance to the specific needs of the practice site is something that PFs and CHITAs undoubtedly are already doing, at least to some degree. However, a brief assessment of the practice site’s previous experience with transformation efforts and with efforts to implement IBH more specifically should be used routinely to select a more pre-determined package and sequence of TA and training components in relation to more flexible use of resources. Information sharing is an ongoing difficulty in most IBH implementations. The Practice Transformation Workgroup, PFs, and CHITAs could develop more examples and models of data sharing agreements (e.g., Business Associate Agreements) that PFs and CHITAs can bring to practice sites to help them learn how to effectively share patient data across providers. More generally, they could develop a more explicit concept of the Patient Centered Medical “Neighborhood” (versus “Home”) that provides a suggested approach to developing information and data sharing channels along with the appropriate tools—both HIT and legal/logistical—that can be used. Poor access to behavioral health providers for some sites is an ongoing theme and needs to be addressed systematically by the Workforce Workgroup, PFs, and representative sites. However, a larger group of health and behavioral health training programs may need to be brought into ongoing deliberations concerning bolstering the workforce. Mobile BH specialist approaches—with BH specialist resources perhaps even being shared across sites, especially in rural areas—may also need to be considered. Current modifications to the SPLIT tool should be examined to determine whether the expected improvements to the timeliness of implementation feedback PFs, CHITAs, and ultimately practices are realized.
Health Information Technology Summary and Recommendations The Health Information Technology Workgroup should review the myriad findings in this area and recommend enhancements to the current tools and resources available to CHITAs and to sites. We recommend that the SIM office continue with efforts to enhance CHITA support and to bring practice sites together based on common EHR usage and/or common challenges to data collection and reporting. 4
PFs undoubtedly have collective expertise in this area, but a useful approach to developing a successful IBH business plan can be found in Corso, K. A., Hunter, C. L., Dahl, O. Kallenberg, G. A., & Manson, L. (2016). Integrating behavioral health into the medical home: A rapid implementation guide. Phoenix, MD: Greenbranch Publishing. 23 Quarterly Report: January–March 2017
It may be helpful, specifically, to form EHR “affinity” groups, so that practices using the same EHR vendors can come together to discuss and resolves specific challenges. Additional, these groups may give sites some leverage in requesting that EHR vendors make system changes that will improve CQM reporting accuracy. It may be helpful to identify the vendors that are most commonly used across the SIM sites, as well as those vendors that seem to pose the greatest challenges for sites. The SIM office should look for opportunities to encourage these vendors to make system changes that will facilitate better CQM reporting.
Ongoing and Future Evaluation Efforts For second quarter 2017, the SIM statewide evaluation will focus on pilot-testing an approach to creating a Coordinated Community System Index (CCSI) in several test counties/health statistics areas and conducting case studies within six practice sites selected from cohort 1. TriWest is currently finalizing a Case Study approach, which is generally outlined below. Six cohort 1 practice sites have been selected based on PF recommendation, diversity of type of practice, location, and predicted differences in baseline CCSI in the practice site’s community. TriWest will conduct 5–6 Key Informant Interviews with the following practice site staff: o practice champion o billing/financial staff o behavioral health provider o primary care provider o admin/front office person o HIT/EHR person (demonstration of EHR workflow) o Other/depends on practice characteristics TriWest will conduct interviews with the Regional Health Connector (RHC) and LPHA/BHTC (if one is in the area) to explore the level of collaboration and coordination within the geographic area. For each of the case study practice sites, TriWest will carefully review all SPLIT assessments, including qualitative data being collected in assessments and in field notes, to more fully describe the activities occurring within the practice, highlighting specific challenges and best practices that can be shared with all SIM-participating practice sites. TriWest has developed a consumer survey and reviewed the tool with the Consumer Engagement Workgroup. We will pilot test this survey during a four-week period while conducting case studies at selected practice sites. Where possible, we will also conduct a focus group with patients or with a site’s Patient Advisory Council, if one exists. The evaluation will continue to monitor quarterly CQM data, implementation activities, and adjustments across the SIM effort. 24 Quarterly Report: January–March 2017
The Community Mental Health Center (CMHC) bi-directional programs are formally funded and underway. The participation of those programs in the SPLIT assessment and feedback process are in the beginning stages, and future reports will included analyses of these assessments. Other CMHC evaluation activities are being finalized.
25 Quarterly Report: January–March 2017
