Matthew DeHart, MPH
Research, Statistics & Design Engineer Fusion5 Executive Summary
Through analysis of Medicare claims data derived from the Centers for Medicare and Medicaid Services (CMS) Limited Data Set (LDS) Standard Analytical Files from 2012-2016 in concert with iterative analysis of Fusion5 risk assessment data we are able to conclude with high confidence how any specific candidate factor (biopsychosocial characteristic) contributes to a patient’s risk of an adverse outcome. We are also able to develop complete and individual risk profiles for patients when we combine multiple candidate factors and determine exactly that patients’ potential risk of several negative outcomes.
We accomplish this through a multivariate mathematical modeling approach to generate a risk level for each initiated episode of care. The resulting risk profile can direct our clinical practitioners to optimization protocols and prevention measures that would serve to reduce a patient’s risk for many potential undesirable outcomes that may include poor clinical quality performance and low patient satisfaction.
The process that we elucidate in this paper will describe how we at Fusion5 collect patient data through our risk assessment process and analyze that data to create individualized risk profiles. These risk profiles will ultimately predict a patient’s risk for a ‘failed’ episode of care and direct providers towards interventions that will avoid those ‘failed’ episodes.
Within the episode of care environment (comprised of pre-acute, acute, and ninety-day post-discharge time frames) there exist unique challenges to patient safety. Neglecting to develop a comprehensive risk management process can compromise patient care, increase liability, and produce excessive financial burden.
Fusion5’s Risk Assessment is designed to give clinicians an understanding of a patient’s risk of health complication based on their social, medical, and psychological health. We are able to evaluate how these health risks will affect the patient’s recovery following an acute hospitalization or surgical intervention. This tool not only allows clinicians the ability to identify a patient’s risk level, it will also assist them in identifying interventions for high-risk characteristics. By identifying and addressing these characteristics, clinicians can assess the opportunity for optimization, thus increasing the probability of improved health outcomes. Our risk assessment will also guide clinicians to examine the social needs of patients and utilize individualized post-acute care resources.
As practices and hospitals integrate Fusion5’s risk assessment into their patient care workflows, we will iteratively improve the quality of care being provided, overall health, and acute and post-acute outcomes of this patient population. Our risk assessment process includes a scientifically rigorous assessment of patient risk through comprehensive data collection and analysis. Several previous studies, including research from the Center for Disease Control1 have identified modifiable and non-modifiable risk factors that can lead to failure. However, each organization faces unique challenges, making a universal risk assessment model inaccurate.
The reliability of any universal risk assessment model is substantially low. We at Fusion5 have developed an evidence-based rigorous methodology that is continually evolving to improve our predictive model and to help ascertain and mitigate the potential risk factors before they can affect an episode.
Several analyses contributed to the development of the Fusion5 risk assessment survey. We completed literature reviews of several validated biopsychosocial risk assessments from institutions such as Dartmouth, Grey Bruce Health Network, Saint-Jacques Hospital, and Johns Hopkins. This meta-analysis of ordinal and nominal scoring methodology informed our interpretations and decisions when scoring questions within our risk assessment and applying appropriate weight to specific responses.
Determination of the impact of comorbid conditions on an unsuccessful episode of care is a result of our ability to analyze and interpret CMS claims data. Using the CMS LDS Standard Analytical Files from 2012-2016, we were able to examine approximately 15 million claims for 30 different comorbid conditions (Table 1).
All analyzed data were normally distributed and met the assumptions of parametric methodology. The dependent variables in our analyses, as aforementioned, were target price and readmissions. A univariate analysis was performed using the Cochran-Mantel-Haenszel2 test producing an estimate of the common odds ratios. These odds ratios were pooled across different repeats of the experimental analyses.
We also performed a univariate analysis between dependent and independent variables to attribute model of fit using an Akaike Information Criterion (AIC)3. These AIC scores informed candidate model selection for a multivariate regression analysis which ultimately produced the risk ratios that were assigned to the comorbid conditions within the risk assessment. The relative likelihood of the model gives us the ability to have less restriction in multivariate model selection because we are maximizing entropy in the sample. Primarily, we are identifying which multivariate model most accurately predicts new and incoming patient data when determining which candidate factors (biopsychosocial characteristics) contribute to detrimental outcomes for patients (readmissions, falls, infection, etc.) (Fig. 1). All the analyses performed to produce the risk assessment survey and model maintained acceptable statistical power and statistical significance. The results are externally valid at the (alpha) 99% confidence level. The risk assessment survey and scoring metrics are under constant scrutiny by our clinical care management team, incorporating continuous feedback integration from our clients and our team members. The mathematical modeling and conclusions will become increasingly refined moving forward, as well as the information that our clients can glean from the completed assessments.
Using information theory methodology and AIC model selection we developed several multivariate models. These models inform which factors contribute to adverse outcomes and the relative impact that each factor has on episode outcome. As a result, we are able to conclude with high confidence, the overall impact that each candidate factor (biopsychosocial characteristic) contributes to a patient’s risk of the unfavorable outcome by itself, and then when confounded with other candidate factors in a complete patient profile. This approach maintains superior effectiveness for several reasons. Primarily, the AIC methodology is one of the best contemporary methods to predict new data based on an existing sample. In this risk assessment endeavor, we are not trying to discover why something happened in the past, but rather we are focused on predicting what will happen within an episode of care at the time that episode is initiated.
Armed with this forecasting knowledge clinical practitioners can make use of evidence-based optimization protocols to minimize the influence of factors known to lead to poor outcomes and excess cost. Secondly, the evidence that we are basing our conclusions on with this methodology is vast. The CMS LDS is one of the largest available databases of its kind in the world. This fact alone gives great statistical power to our method. In concert with the data that we collect from our partner groups as one of the largest Bundled Payments for Care Improvement Advanced (BPCI-A) conveners in the nation only furthers the accuracy of that methodology.
These inherent methodological strengths in combination with effective care navigation strategies are essential parts of a successful episode of care. It is also important to note that we can only develop risk models for those outcomes on which we have data. Therefore, it is essential for us to work closely with our BPCI-A partners to continuously improve our process through data collection, research, education, and communication. The current iteration of the modeling process will be continually improved through diligence and cooperation. More outcome models will be developed, and more candidate factors will be analyzed to develop more robust future models that will provide the even greater predictive ability for reducing adverse outcomes within episodes of care.
As we continue to collect data, we will become better informed on the specific aspects of our unique population that contribute to the success or failure of the episodes of care. Assessing and addressing risk is an iterative and evolving process that requires constant evaluation and re-evaluation. It is paramount to understand exactly what information the Fusion5 risk assessment is designed to produce. It is not meant to replace a clinician’s judgment; however, it will serve to enhance evidence-based decision-making as to how our clinical team and our partners can create and modify patient-specific care plans to reduce the likelihood of injury to patients and excessive episode costs, and ultimately improve the quality of life for those we serve.
1. Healthcare-associated Infections. (2017, July 19). https://www.cdc.gov/hai/ca_uti/uti.html
2. Mantel, N. (1963). Chi-Square Tests with One Degree of Freedom; Extensions of the Mantel- Haenszel Procedure. Journal of the American Statistical Association, 58(303), 690. doi:10.2307/2282717
3. Akaike, H. (2011). Akaike’s Information Criterion. International Encyclopedia of Statistical Science, 25-25. doi:10.1007/978-3-642-04898-2_110