Standardized questionnaires like the Vineland-3 hand payers a tidy number, but a veteran behavior analyst warns they miss what actually shows whether a child is progressing.
Key Takeaways
- A tidy score is not proof of progress. Payers and programs increasingly lean on standardized questionnaires to decide whether ABA is working. Green warns the tools were never designed to track the week-to-week change that defines good therapy.
- The popular instruments give a snapshot, not a measure. A tool like the Vineland-3 is a third-party report sensitive only at roughly yearly intervals, useful as a place to start but not as the gauge of whether an intervention is working. Direct observation and graphed data are the real cornerstone.
- TRICARE’s own program is the cautionary tale. The military’s Autism Care Demonstration requires a fixed battery of standardized, largely parent-reported measures, and in 2025 a National Academies committee urged the Pentagon to drop several of them. The Defense Health Agency has conceded the parent-report bias in its own reporting.
- Measure the child, not the claim. Green argues credible measurement is individual and multi-method, while payers and auditors keep fixating on billing data. Closing that gap is central to telling quality care from volume.
A number is reassuring. A score on a form, tracked over time, has the look of proof, which is why payers and public programs keep reaching for standardized questionnaires to decide whether autism therapy is working. Dr. Gina Green has spent much of her career on the question those numbers are supposed to answer, and she is blunt that the tools are being asked to do something they were never built to do. A past president of the California Association for Behavior Analysis and now its public policy consultant, Green also wrote a commissioned paper on standards and outcomes for the National Academies’ evaluation of the military’s autism program. (She co-authored the field’s practice guidelines and wrote that National Academies paper, disclosures that bear on what follows.) The stakes are not abstract: the same scores that decide whether a family keeps its authorization are the ones a program cites to tell funders the money is well spent.
What the Vineland-3 can and cannot tell you
The instrument at the center of the argument is the Vineland Adaptive Behavior Scales, Third Edition, a widely used measure of everyday living skills. Green’s first point is that no questionnaire, the Vineland included, is enough on its own. “There is no single assessment instrument that is going to give you sufficient information for developing a good ABA treatment plan,” she said in an interview with Acuity. “The standards of the field, now represented mainly by the CASP guidelines, make it really clear that behavior analysts have to use multiple sources of assessment information.”
The sharper problem is how these tools collect what they collect. “Virtually all of those adaptive behavior assessments, and the Vineland happens to be one of the most widely used, are third-party reports,” Green said. “They’re completed by a parent or a caregiver or a teacher based on their impression, their memory, how they think the person is doing. So they’re indirect rather than direct measures.” They are also blunt over time. “They’re not sensitive enough to detect changes in those behaviors over the short run,” she said. “They give you a global picture. At best, most of them are sensitive enough to detect changes at about a year.” An instrument that registers movement once a year cannot tell a clinician whether this month’s program is doing anything.
Even the tests administered face to face, she added, share the limit. An examiner presents a fixed set of items in an office and scores the answers, producing a norm-referenced snapshot. “They give you a general picture, a global picture,” Green said, but the same behaviors “may look entirely different if you view them in the client’s home, in school, in the community,” which is where the behaviors that actually matter to a family play out.
The cornerstone is watching
What the standardized tools are good for, in Green’s account, is pointing. A parent report flags where to look. “If a parent reports on the Vineland that their youngster doesn’t independently wash their face or cross the street safely, that gives me a starting point,” she said. “Then I would do direct observation. I would set up situations where I could see for myself how that client does.”
The direct look is the real measure. “A cornerstone of applied behavior analysis is direct observation. We define each target behavior on a treatment plan in a way we can observe directly and measure how often it happens, how long it lasts, and other dimensions,” Green said. “Assessments like the Vineland can supplement that, but they’re a starting point.” Frequent, graphed data on defined behaviors is what lets an analyst catch a stalled program in weeks rather than a year. It is also the painstaking work that thin clinician staffing tends to squeeze out first.
What TRICARE’s program reveals
The clearest test case is the Pentagon’s Autism Care Demonstration, the TRICARE program that covers ABA for military families and is currently slated to sunset in 2028. To keep services authorized, it requires a fixed battery of standardized outcome measures: the Pervasive Developmental Disorder Behavior Inventory every six months, the Vineland-3 and the Social Responsiveness Scale each year, and a parental stress index. Most are completed by parents. Green’s commissioned paper argued that leaning on an indirect, one-size battery to judge treatment is a mismatch with how ABA progress is actually tracked.
The National Academies committee that Congress ordered to evaluate the program landed in much the same place in its 2025 report. It found the required tools are administered on top of the individualized assessments providers already conduct and do not necessarily capture progress for every child and goal, and it recommended the Defense Health Agency eliminate mandatory assessments that do not support treatment planning and discontinue the parenting-stress measures outright. The committee also noted the agency has reported results to Congress drawn mostly from parent-completed scores, often from small samples. In its own annual report, the agency concedes as much, acknowledging that parents are not objective outside evaluators and that it is unclear how much bias their scoring introduces. One instrument in the battery, the committee observed, was not built for ABA outcome assessment in the first place.
That exposes the harder policy problem. If credible measurement is individual and multi-method, it does not collapse into a single score a payer can line up across thousands of children, which is the very thing a fixed battery promises. Green’s paper pushed further, noting the program applies the same instruments to everyone, gathers no separate consent for the outcome testing, and steers required goals toward the core symptoms of autism rather than the adaptive, daily-living skills families often prize most. The system, in her reading, is tuned for aggregate reporting, not for the child in the room. Payers, she says, are looking in the same wrong place. “The payers are going to look at billing claims, mostly the money aspect, the bottom line, and not necessarily at the variables that influence quality and outcomes for clients.”
Which is why an audit built on claims data and a clinician’s judgment about whether a child is getting better can point in opposite directions, and why fights over assessment and reassessment coding reach well past the billing office. As state rate structures tighten and Medicaid budgets shrink, the question the field has never fully answered, how anyone actually knows the therapy worked, stops being academic.
