Background and Study Design

This report details a randomized evaluation of a quality improvement initiative conducted in 19 of 21 Kaiser Permanente Northern California (KPNC) hospitals. The focus of this evaluation was the targeted expansion of the Transitions Program, a post-discharge care pathway tailored for hospitalized patients identified as having low predicted readmission risk. The rationale behind this expansion was grounded in a causal machine learning model designed to predict patient response to the Transitions Program intervention.

The Transitions Program

Originally, the Transitions Program represented a care coordination initiative that provided specialized support to eligible hospitalized patients immediately following discharge. Eligibility was determined by a Transitions Support Level (TSL) score, which estimates the risk of non-elective re-hospitalization and mortality within a 30-day window after discharge. For context, prior to this study, only high-risk patients—those with TSL scores exceeding 25%—were typically referred to the program.

With demonstrable success in reducing readmissions among high-risk patients, there arose a compelling argument for extending the program to lower-risk patients, those with TSL scores below 25%. Given that these lower-risk populations represented a significantly larger segment of the hospitalized patients, the strategy was refined. The focus transitioned from merely assessing risk to emphasizing individualized predicted benefits—essentially, estimating the treatment effect of the Transitions Program on these patients.

To operationalize this, a new statistical model termed the Predicted Benefit Intervention (PBI) score was developed. This score was informed by the T-learner meta-algorithm, utilizing logistic regression as its base learner and trained on data collected both before and after the implementation of the Transitions Program. The process included rigorous cross-validation to ensure model calibration and discrimination.

Eligibility and Data Collection

The study derived its data from the TSL-low risk population, analyzed over three distinct, non-overlapping time periods between March 2019 and June 2023.

1. Pre-Randomization Period (March 4, 2019, to April 25, 2022): This phase involved all hospitalized KPNC patients who survived discharge. During this timeframe, no systematic outreach was conducted for the TSL low-risk group, although some manual selections occurred in May 2021. This retrospective scoring helped identify patients eligible for randomization had they been discharged later during the randomized period.

2. Randomized Quality Improvement Period (May 19 to December 19, 2022): This central period of the study involved participants randomized into two groups: those receiving the Transitions Program intervention and those receiving usual post-discharge care. Both TSL and PBI scores were utilized to qualify low-risk patients for this randomization process.

3. Post-Randomization Period (December 19, 2022, to June 30, 2023): Following the randomized phase, all eligible patients were enrolled in the PBI workflow, regardless of whether they were part of the initial randomized study.

In terms of eligibility, patients were selected if they had a TSL score below 25% and a corresponding PBI score beneath a specified threshold designed to align discharge volume with available outreach resources.

Intervention

During the randomized evaluation period, participants in the intervention arm were referred to the Transitions Program after discharge. If they accepted this referral, they were actively enrolled and received care coordination for 30 days, mirroring the assistance provided to high-risk patients. This comprehensive care was designed to include post-discharge checks, medication reconciliation, and regular consultations with Transitions Program case managers.

With the transition into the post-randomization phase, every eligible patient, regardless of earlier randomization, received the same level of support from the Transitions Program.

Outcome Measures

The study focused on three primary outcome measures, two of which adhered to the HEDIS criteria:

1. HEDIS-Reportable Re-Hospitalization: This metric defined non-elective re-hospitalization occurring within 30 days post-discharge.
2. 30-Day Post-Discharge Mortality: Monitoring patient mortality rates within 30 days of discharge.
3. Composite Outcome: Combining the first two outcomes to reflect the overall impact of transitions on patient care.

To align with local health system metrics, the analysis specifically targeted HEDIS-eligible patients, as these metrics are pivotal for assessing healthcare quality within integrated systems.

Statistical Analysis

This evaluation was set up to enroll approximately 9,000 patients during the randomized period, ensuring sufficient power to detect a 10% relative risk reduction in the primary outcome of 30-day re-hospitalization or mortality.

The primary analysis relied on the intention-to-treat approach, encompassing all randomized patients, irrespective of their final treatment group assignment. Acknowledging potential crossover events—where patients randomized to the intervention declined or were excluded from outreach—additional sensitivity analyses were planned. These analyses involved applying a method to distill the dataset down to a population that was most likely to comply with the intervention.

Statistical methods included augmented inverse propensity-weighted logistic regression, adjusting for various baseline factors at discharge. This comprehensive approach aimed to validate the effectiveness of the targeted interventions while ensuring ethical and robust evaluation standards adhered to the practices outlined by the CONSORT guidelines.

By emphasizing the transition to personalized care through machine learning and quality improvement methodologies, the study aimed to enhance overall patient outcomes in the context of post-discharge care coordination.