Goals/Uses of Information
Program evaluation
Resource allocation
Capitation rates
Compare within/without state – develop comparable indicators
Level the playing field to compare providers
Make equitable comparisons – do fair reporting
Predict service patterns
Get buy-in for the process
Accurately report performance
Quality improvement
Identify and test risk factors for clinical utilization
Do quasi-experimental studies to establish causality
Educate system participants about Risk Adjustment
Develop useful ways to present Risk Adjusted Data

Obstacles
Lack of trust in available data
"Dirty data"
Questionable measures and methodologies
Getting to the "right" data
Risk adjustment is a hard sell
Risk adjustment is hard to explain
It is difficult to decide on variables to adjust
Different models have different purposes and characteristics
Viewing Risk Adjustment as analysis of variance
Cost – un-reimbursed / un-funded
No history of change as a result of using risk-adjusted data

How to Overcome Obstacles
Make good use of existing data
Use sampling to improve data quality – be parsimonious
Match risk adjustment variables to outcomes
Conduct training on risk adjustment
Add appropriate analytic staff
Include data quality as a report card indicator
Make data public – show adjusted and unadjusted data
Don’t analyze bad data
Increase funding levels

Risk-Adjustment Goals:
Grant development
Building better models, e.g.:
Disease specific national models
Intervention-specific models
Inpatient specific models
Provider profiling instead of utilization review
Learning about risk-adjustment and methods generally
Improve federal recognition of the value of risk-adjustment in mental health

Obstacles:
Resistance from funding sources and/or review committees
Small or inadequate data sets
Identifying the "product" and the users
Inadequate knowledge base
Geographic dispersion of researchers
System-to-system variability in data and populations
Lack of provider input
Weak measures of global functioning

Strategies:
Obtain and analyze pilot/preliminary data
Investigate alternative funding sources, e.g., RWJ
Re-frame questions to funding sources as profiling or quality improvement
Collaborate on analysis, grant writing, and to gain interdisciplinary perspective
Develop a listserv for risk-adjustment research
Develop a list of persons and interests/skills from the conference
Conduct methodology seminars/workshops on Missing data, HLM, longitudinal data, equivalence      testing, Bayesian models, decision trees, PPE, risk-adjustment nuts and bolts from A to Z
Bring providers into discussions
Write white papers/working papers, with support of CMHS, Evaluation Center
Work with private BHCOs on answering specific data questions

Recommendations:
Look at data availability in various jurisdictions (all age groups)
Assess the quality of existing data
Assess the existence of special purpose data
Identify available/possible risk adjustment variables in both data sets
Look at the coverage/scope of data in both data sets
Look at the feasibility of doing a study comparing the cost/benefit/value of using each data set
Design a study – example – Compare the Vermont "PPE" model with "HLM" methods

Ideas for actions to do:
List-serv
Resource guide of available collaborators/consultants - distribute to all
Web page resource
Regular advisory group meetings among public representatives at different levels
Seminars, in-person or internet based
Funding support from HSRI Evaluation Center, SAMHSA, STTR
A "How-to" publication from A to Z (choosing variables, methods, etc.)
External funding
Sell to recoup costs
Hard copy or CD – CD could contain real data sets with exercises
Developing a "minimum data set" for standardized risk-adjustment
Use the NRI database of state level performance measures
Examine how data are collected across locations
Conduct external peer review of existing databases for adequacy for risk-adjustment
Take full advantage of existing data sets
Conduct a quick survey on the status of states’ efforts
To select performance measures
In doing risk-adjustment (yes, no, and how)
PI infrastructure
Big problems and solutions encountered

Actionable:
Becky Moore (OK) and Mary Loos (VA) will construct and do the state survey
Phyllis Abbott will put together the listserv
Michael Hendryx will explore funding ideas for the seminars/CD/toolkit.

Uses of Risk-Adjusted Data
-Information/Policy
-Consumer Report Cards
-Contracting
-Reward System/Incentive
-QI
-Represents service recipients within the system, raises the level of discussion
-Make arguments to increase resources
-Transmit Best Practices

Stakeholder Education
-Long-term, multi-year process.
-important to include stakeholders in the process.
-build relationships, continuing contact.
-Information reported informally first "informatic moments"
-phase in reporting
-first raw data, then risk-adjusted data
-Agencies must be prepared to have the information be used for and against them.

Reports to Stakeholders
-compare to expected outcomes
-population characteristics
-Reports/data are tools, don’t over promise results
-Feedback Loops important
-what you are trying to do is change cultural, implement QI culture

What Works?
-how to increase buy-in/compliance
-financial incentives
-recognize stages of changes, take all stages into consideration when planning
-+ rewards, strengths-based approach
-evaluatees should get something in return

Short-term goals
Learn from each other-partnering
List of web-linked pages of organizations reports, report cards, etc.
(e.g. OK, OR, CA, VT, NY state webpages, PacificCare webpage, Jeb Brown Webpage
Contact list- List serve
build out of participant list,
Follow-up: John @ Northwestern
Report at next conference "What happens when you feedback selective information to audiences
Follow-up: Nancy, John, Rochelle, Jeb, Bob, Lucille

Long-term goals
Study of how feeding back information to agencies effects services (does it change line-staff behavior, cost-benefit analysis)
-two independent variables
-level of feedback
administrator
care-manager
clinician
-type of feedback
report only
report and personal contact
(possible collaborators- Michael Lambert)
--specific: giving feedback about seclusion by race/gender
Follow-up: Jeb, Rochelle, John, Sabine, Judy, Lucille

Education of:
-The next generation of risk-adjusters
-Users of information/reports

Back to Risk Adjustment Home