Blue Gems ABA’s Director of Outcomes explains how family feedback and CFQL-2 quality-of-life scores sit alongside skill data as Medicaid presses for results.
Key Takeaways
- Mastery counts tell only part of the story. Skill data shows what a child learns in session, not whether it carries outside of treatment, a gap behavior analysts have flagged since at least 1977.
- Blue Gems added two caregiver-reported outcome measures. A Family Feedback Survey and the published CFQL-2 now run alongside other standardized assessments.
- The measures don’t always agree. Research finds caregiver-centered and standardized outcomes can diverge; Blue Gems routes those gaps to clinical leadership and the family.
- Results are still months away. Both tools launched after the October Blue Gems ABA Blueprint Leadership Summit.
For most of her career, Patty Quan, MA, BCBA, has measured progress the way behavior analysts are trained to: skills mastered, maladaptive behaviors reduced, data points plotted one session at a time. The question that now occupies her sits outside the treatment session entirely.
“What does this individual’s life, and what does their family’s life, look like after we’re gone?” Quan, the Director of Outcomes at Blue Gems ABA, said in a video interview with Acuity on Nov. 4. “If we’ve done a really good job at our role, then you’re not just going to see the changes and benefits within treatment sessions. These skills must be generalized and maintained outside of treatment.”
Blue Gems ABA, which serves families in seven states through center-based, in-home, and school-based services, used its October leadership summit, Blueprint, in Long Branch, New Jersey, to announce two additions to its assessment practice. One is the Blue Gems ABA Family Feedback Survey, which the company describes as a social validity measure; the other is the Child and Family Quality of Life, Second Edition (CFQL-2), a published caregiver questionnaire. Both run alongside the standardized assessments and the skill acquisition and behavior reduction data the company already collects.
The additions arrive as payers ask the field for evidence of results. In August, the Centers for Medicare & Medicaid Services released a 173-page toolkit for state Medicaid and CHIP agencies, reporting that payments for ABA services rose from about $1.94 billion in 2021 to $10.1 billion in 2025, an increase of 421 percent. The guidance, which follows a series of federal audits of state Medicaid ABA programs, urges states to require that reauthorization requests document measurable progress.
ABA Generalization and the Limits of Skill Mastery Counts
Before taking this role, Quan said, she encountered few established outcomes programs, noting that formal departments for this work are a relatively recent development in ABA organizations. Blue Gems’ announcement of her hire says she most recently supervised operations for TRICARE’s Autism Care Demonstration.
The problem she describes is an old one. In a 1977 review in the Journal of Applied Behavior Analysis, Trevor Stokes and Donald Baer examined some 270 applied studies and sorted the field’s approaches to generalization into nine categories. The most common they called “Train and Hope”: teach a skill, then note whether it appears elsewhere, without actively programming for it. That approach characterized almost half of the applied generalization literature they reviewed. This foundational critique influenced Quan to introduce social validity and quality-of-life measures at Blue Gems.
“You can’t just train and hope,” she said. “You’ve got to always keep long-term distal outcomes in mind now, and start planning for them now, and tracking progress toward them now.” She explained that true generalization cannot be left to chance and requires caregiver collaboration to support skill maintenance outside of treatment. In Quan’s view, caregivers are more likely to promote and sustain skills when treatment goals align with their values and meaningfully improve their daily family life.
CMS’s toolkit makes a similar point in policy language, describing the aim as a child using a skill consistently “across different routines, people, and settings.” Acuity’s earlier reporting on home-based and clinic-based ABA approached the same problem from the question of where treatment happens, and clinical leaders have since argued that hours-and-location debates should give way to outcomes measurement.
How Blue Gems Uses the CFQL-2 and a Family Feedback Survey to Measure Social Validity
Social validity has its own lineage in the field. In a 1978 paper, Montrose Wolf argued that behavior analysts should ask the people they serve about the significance of a treatment’s goals, the appropriateness of its procedures, and the importance of its effects. The idea has gained currency as value-based arrangements push providers to measure family stress and quality of life. Quan described the Blue Gems ABA Family Feedback Survey, the organization’s social validity measure, in Wolf’s three terms. In addition to gaining feedback about overall service satisfaction, she said, the survey asks families about “the extent they believe the goals, procedures, and outcomes are acceptable and meaningful to them.” The survey evaluates whether treatment targets align with family values, how ABA therapy strategies fit into daily routines and honor client assent, and whether services are producing the results families hoped for when they began services.
The CFQL-2 is a free-to-use quality-of-life questionnaire. Dr. Thomas W. Frazier and colleagues evaluated it in a 2020 study of 566 caregivers whose children were referred to an autism diagnostic clinic, finding that it reliably measured eight domains: child, family, caregiver, financial, social network, partner relationship, coping, and quality-of-life change. The International Consortium for Health Outcomes Measurement includes it, along with the Vineland-3, in its standard outcome set for autism, according to a 2026 study in JMIR Pediatrics and Parenting. The items are written in plain language. One asks caregivers whether “my child deals with daily life hassles and daily activities well.”
At Blue Gems, the primary caregiver completes the CFQL-2 every six months, timed to reauthorization, Quan said. Like other assessments Blue Gems uses, such as the Vineland, VB-MAPP, and AFLS, the CFQL-2 informs the development of individualized treatment plan goals. She pointed to a finding from the CFQL-2 validation study: child externalizing behavior had the strongest negative association with caregiver and family quality of life. When low CFQL-2 scores coincide with maladaptive behaviors, she said, the clinical team needs to conduct a functional behavior assessment to design a behavior intervention plan that promotes safety, teaches replacement skills, and incorporates caregiver coaching.
When Family Feedback and ABA Skill Acquisition Data Disagree
Quan did not rank the measures against each other. “It’s not that one measure is necessarily better than the other,” she said. For example, the Vineland is norm-referenced, comparing a child’s adaptive skills to a peer sample, and each instrument has limitations the others can offset. “I’m not saying that these should replace those measures at all.”
When a family’s responses diverge from what the skills data shows, she said, clinical leadership is notified and helps the supervising BCBA prepare for a direct conversation with the family. “Some of the topics we talk about in these measures are emotionally difficult for families,” she said. She offered an example from her own early practice: “As a young BCBA, I used to write really complicated behavior intervention plans, and the parents would say, ‘This is too complicated. I’m not doing this in real life.’”
Research suggests the divergence is common. A 2022 study in BMC Pediatrics followed 154 children through 24 months of ABA in a large Southern California integrated health system and found limited agreement among progress on individualized goals, caregiver participation goals, and Vineland-II adaptive behavior scores. None of the goal measures correlated significantly and positively with Vineland change, and the authors recommended adding validated measures of what matters to families, such as quality of life. The 2026 JMIR study, which examined a virtual ABA provider and was funded by that provider, likewise found no significant association between caregiver satisfaction and changes in Vineland-3 scores.
For Quan, the gap between measures is where the useful information sits. “Some of the skill acquisition measures are for very specific skills that don’t always capture the impact on the family,” she said. She described a hypothetical parent whose child had transitioned from high-distress crying behaviors to requesting a specific red cup, a single line on a data sheet. In her example, social validity and quality-of-life measures could reveal that the child is self-advocating more for their specific needs, and that the parent feels relief at understanding and being able to better support their child. “These measures help us understand that requesting has made a meaningful and functional difference in the family’s life,” she said.
CMS’s ABA Toolkit, Standardized Outcome Assessments, and Caregiver-Reported Measures
The CMS toolkit, which is guidance rather than regulation, recommends as best practice that states require at least one standardized outcome assessment instrument. It lists five, including the VB-MAPP and the AFLS, and says states should not allow provider-created outcome measures. Quality-of-life and social validity instruments do not appear on that list, and the toolkit notes that no ABA-specific quality measures yet exist. Quan described Blue Gems’ additions as supplements to standardized assessments, not substitutes for them.
The newer measures carry open questions of their own. The JMIR authors note that no threshold for clinically significant change on the CFQL-2 has been established, and the 2020 validation study drew on a single clinical site without a healthy-population comparison sample.
What ABA Providers Need to Replicate Quality-of-Life Outcomes Tracking
Blue Gems rolled out both measures shortly after its leadership summit, Quan said, and is now tracking response rates, staff time, and families’ early reactions. She said those figures are not yet available.
For a provider considering the same approach, she described three requirements. The first is identifying social validity and quality-of-life measures built for the population a provider serves. “There’s actually not a common questionnaire for social validity, because it really depends on who you’re serving,” she said. Providers building their own face design questions the field is still working through; behavior analyst Tom Critchfield argued on ABAI’s blog in 2025 that much survey research in behavior analysis journals skips basic practices, such as evidence that items are reliable and valid.
The second is software that can analyze responses item by item, within states and across the organization. “Sometimes it’s more than what Excel can handle,” Quan said. The problem is familiar across the field: when seven providers benchmarked their own outcomes this year, 360 Behavioral Health Chief Clinical Officer Dennis Dixon spent 40 to 60 hours matching billing records to test scores by hand. The third is a team fluent in data organization and analysis to run it.
The cost to families, by her estimate, is five to 10 minutes per round. “Just the parents spending the five minutes to 10 minutes it takes to fill out the CFQL-2 and our social validity measure, that is an important part of maintaining open communication and strengthening parent involvement,” she said.
Because the CFQL-2 runs on a six-month cycle at Blue Gems, the first repeat administrations are still months away. Quan said she would welcome sharing what Blue Gems learns from early responses, and what it changes because of them, once they arrive. “Evaluating outcomes keeps us accountable,” she said. “We must understand how our ABA services are making a meaningful difference in our clients’ everyday lives.”




