IN their commentaries upon our article, Romano as well as Ash and Ellis provide their perspective as to the differences and similarities between regression and categorical models. For Romano, the distinction is itself a "false dichotomy." We acknowledge that in comparison with many regression models, such as those routinely touted for predictive analytics, the hierarchical condition category (HCC) and DxCG models with which Ash and Ellis are widely associated, undergo significant clinical input in model construction, making the distinction less stark.
In their comments, each of the authors highlight (correctly) that the simple example we provide to help understand differences in modeling approach can be adapted such that when similar adjustments are made to a more complex model such as HCCs that "By adding selected interactions as needed, the model can accurately characterize the importance of potentially more than 2394 combinations of CCs, more than the number of atoms in the universe!" (Ash & Ellis, 2016).
That this can happen is not contested. What is far less clear is the determination of when "as needed" applies or for Romano that "the developer of the model must specify the predictor variables to be considered, the shape of the relationship between each predictor and the outcome, and the hypothesized interactions among predictor variables" (Romano, 2016). In the HCC model for 2014, for example, the community model included 6 disease interaction terms (Health First Network Inc, n.d. drawn from the universe of 604 sextillion possible interactions: 279 (there are 79 HCCs in the model) possible interactions. To determine when interaction terms are needed requires significant work such as that conducted in the building of categorical models.
The example of MS-DRGs used by each commenter model and described under regression by Ash and Ellis is a useful demonstration of this principle. The MS-DRGs can be seen as "a class of particularly simple regression models" (Ash & Ellis, 2016), but only in their function of translating the MS-DRG "atoms" into weighted predictions of outcomes (eg, costs). However, the simple structure of MS-DRGs they outline is built upon detailed clinical logic governing how the atoms of the individual MS-DRGs are formed with intentional separation of the clinical model from its subsequent application to payment. In fact, by failing to separate the description of a patient (or enrollee) from the outcome of interest, we can vary prediction over time with updated risk scores but fail to see how (if at all) we have impacted the underlying populations. Consistent descriptions of patients/enrollees across time, setting, geography, and even socioeconomic divides generate an invaluable baseline for understanding differences, not just identifying that they exist.
How the MS-DRG model is built, as with other categorical models, is available to inspection through detailed definitions manuals. The clinical hierarchy and the use of disease interaction terms contained within are a product of both logic and data. There is no requirement of extensive data analysis for clinicians to comment on the appropriateness or deficiencies of assumptions, enabling greater transparency and comment. In fact, the effects that various interactions might have, including the example provided of "cases of nervous system neoplasms with different kinds of serious complications or comorbidities," can be assessed by anyone with access to the clinical description. If a suggested modification provides an enhancement to the existing model, then the MS-DRGs can be readily revised in response, as they have been over time. As echoed by Romano, neither approach holds a monopoly on flexibility. It is important, however, to remember that both approaches need to address the practical complexity of disease interaction rather than merely supporting the theoretical bandwidth to do so.
This description of how the logic is viewed and accessed by users highlights the core difference in approach. Regression models tend to assume that "users are typically not interested in model 'mechanics'" (Ash & Ellis, 2016); they get to see "what the model predicts for any individual or group of people." This point is also raised by Romano when he describes calculating Framingham and CHADS2 risk scores on mobile devices. Categorical model design assumes that users will want to understand the mechanics, the how and why of classifying complex patients and disease, and to engage with that description based upon their knowledge and experience. The major benefit of a categorical model is in the interaction with the end user. The clinical logic is intended for review so that all stakeholders can ensure that they are treated fairly-meaning that judgments about quality of care are based on how well they performed their duties rather than whether their patients are sicker. To focus solely on the prediction of a risk without a clear and understandable description of the underlying population loses much value in identifying where problems exist and how they might be corrected when performance is judged (fairly) to be substandard. After all, behavioral change is ultimately the focus of nearly all applications in which risk adjustment is deployed, whether it is for capitation or comparison of in-hospital mortality rates.
We would therefore submit that the most important criterion for evaluating the success of a risk adjustment model is whether it produces information that actually results in substantial and sustainable behavior changes that lower cost or improve quality of care. Because the categorical structure of DRGs was effective in linking the clinical and financial aspects of care, a language was created that revolutionized hospital management, resulting in dramatic reductions in the cost of hospital care (Russell & Manning, 1989). The idea that users are not interested in the clinical detail of a risk adjustment model ("model mechanics") is completely contrary to the experience with DRGs. Indeed, in the Federal Register each year, the Centers for Medicare & Medicaid Services summarizes the extensive user requests for changes to the structural logic of the DRGs, which, as noted by Romano, result in their ongoing evolution. Understandable and useable "model mechanics" are essential for creating a language that facilitates real behavior change. In contrast, despite the strong financial incentive for efficiency created by HCC risk-adjusted capitated payments, there appears to have been no revolution in the management the Medicare Advantage plans. Indeed, after 15 years of implementation, Medicare Advantage costs are on average the same as care delivered under Medicare fee-for-service (Medicare Payment Advisory Commission, 2014). At a minimum, this raises the question of how much the failure to have a unit of payment that can be used as a language for change contributed to the cost performance of Medicare Advantage plans. Failure to fully consider the importance of "model mechanics" to end users results in a model that is not as useful for producing real-world behavior changes. To achieve substantive and sustainable improvements in efficiency and quality having a unit of payment that communicates the incentive for change to all stakeholders, whether actuary, patient, or clinician, in an understandable and clinically credible way is more important to the psychology of change than mathematical properties and performance. This is the fundamental lesson from the DRG Prospective Payment System.
REFERENCES