A study released by Student Veterans of America promised to bring much-needed clarity to veterans’ college graduation rates, but irregularities in how the research was done may cast doubt on its findings.
While there are multiple issues with SVA’s newly released Million Records Project, perhaps the most significant is its under-representation of students attending the most-scrutinized — and often lowest-performing — types of schools: for-profit colleges and universities.
Looking at fewer of these students may have inflated the study’s main finding that 51.7 percent of student vets using the GI Bill earned a degree or certificate.
In a budget-conscious Washington and amid anecdotes of some schools taking advantage of former troops, vets’ advocates are eager to emphasize the value of the Post-9/11 GI Bill and other education benefits as a way to protect them.
But despite a 2012 executive order to executive agencies to “develop a comprehensive strategy” for tracking student veteran outcomes, there is still little to no federal data on how vets do in college.
SVA’s study, financed by donations from private companies, could fill in some of these information gaps, but there remains work to be done. Graduation rates at for profit colleges remain unclear and the data cited in this study likely renders unreliable the overall graduation rates.
Just 10 percent of students covered by the study were identified as attending for-profit schools, with 11 percent at private schools and 79 percent at public schools.
How drastically that underestimates for-profit school attendance by vets using the GI Bill is unclear, and each of the three groups that collaborated on the project — SVA, the Veterans Affairs Department and the National Student Clearinghouse — pointed Military Times to the other for an answer.
“I think it highlights that more research needs to be done,” said Michael Dakduk, who led SVA when work on the Million Records Project began and is now with the Association of Private Sector Colleges and Universities, a for-profit schools trade group.
Still, “It’s the best evidence of veteran progression in higher education that we have today,” he said.
But Anthony Dotson, coordinator of the University of Kentucky Veterans Resource Center, called the study “at best, misleadingand , at worst, dishonest.”
“I was just wondering if anyone was going to throw the BS flag,” Dotson said, adding that he thinks students at for-profits may have been weeded out to bolster the graduation rate.
“While I understand the motivation of SVA to positively promote veterans on campus, ignoring the truth means that we ignore the problem and therefore are actually working against student veterans.”
SVA acknowledged the undersampling of students at for-profit schools, but stood by the findings.
“At least one in five veterans who used the [Montgomery] or Post-9/11 GI Bill from the [study’s] time period were selected,” said Chris Cate, SVA vice president of research. “The size of the sample resulted in an extremely high statistical power and low margin of error.”
The project examined the records of 788,915 student veterans, including those who first used the Montgomery GI Bill from 2002 to 2010 and who first used the Post-9/11 GI Bill in 2009 and 2010.
SVA relied on VA to identify veteran GI Bill users and the National Student Clearinghouse to provide data on their academic progress.
But not every school provides data to NSC — and for-profit institutions are much less likely to do so.
During the enrollment years that the study considered for Post-9/11 GI Bill vets, the group received data from an average of 97 percent of public schools and 93 percent of private schools, but just 55 percent of for-profit s according to NSC data.
Those data go back only to 2003, one year after the study period for Montgomery GI Bill vets began, but the averages were 94 percent for public schools, 88 percent for privates and 53 percent for for-profits.
Any student attending a school when it did not submit data was excluded from the study sample.
Just how the sample was chosen is also in dispute. It its report, SVA said VA created multiple filters before choosing records to include, one of which excluded students who had received benefits while attending schools “that were known not to report data to the NSC.”
Not so, said VA spokeswoman Genevieve Billia. “VA eliminated no records of students on account of National Student Clearinghouse status in responding to SVA’s [information] request,” she said.
Meanwhile, Jason DeWitt, NSC’s research manager, said he thought VA excluded “a small number of large for-profits that VA knew were not covered by the Clearinghouse.” He declined to name those schools.
SVA and NSC both maintain that after VA eliminated some records, other records totaling just under 5 percent of the remaining sample were removed for lack of NSC data.
DeWitt said that he thought the number of such exclusions “would have been quite small.” But he couldn’t provide more detail on how many records were excluded, and how the study sample differed from the total population of GI Bill users.
SVA referred that question to VA; VA referred it back to SVA.
VA previously provided information to Military Times on the total number of Post-9/11 GI Bill users by school between August 2009 and June 2011. Of the top 25 schools by enrollment, totaling 234,906 students, 10 were for-profits, with 97,920 students, or 42 percent of that total enrollment.
According to NSC data, of those 10 for-profits, three, with a combined 46 percent of the for-profit enrollment, submitted data for the entire study period. Four schools, with a combined 17 percent of the for-profit enrollment, submitted no data, and three, with 37 percent of the for-profit enrollment, submitted data covering part of the study period.
The 51.7 percent completion rate figure compiled in the study was calculated by counting the number of students who started attending classes under the GI Bill from 2002 to 2010 and graduated by June 2013 with any academic credential, from technical certificates to graduate degrees and everything in between.
That way of calculating graduation rates varies widely from the standard method used by the Education Department and makes comparison of vet and nonvet data difficult to impossible.
The Education Department evaluates how many students in each year’s starting class have graduated by 150 percent of the expected completion time: six years after enrollment for four-year schools and three years after enrollment for two-year schools.
In contrast, the Million Records Project counted the number of students who started attending classes under the GI Bill from 2002 to 2010 and graduated by June 2013 with any academic credential, from a quick technical certificate to a graduate degree in astrophysics, and everything in between.
So a student who began pursuing a four-year degree in 2010 and remained on track to finish by 2014 would count negatively against the graduation rate. Yet the opposite would be true for a student who started pursuing a two-year associate degree in 2002 and didn’t finish until mid-2013.
The Education Department’s most recent available data show that 56 percent of students who began attending four-year schools in 2006 graduated by 2012, as did 33 percent of students who began attending two-year schools in 2009.
The report found that 45 percent of students who began their studies at for-profit schools earned a degree or certificate. The figures for students starting at private and public schools were 64 percent and 51 percent, respectively.
However, those results could be skewed by students who start at one type of institution, such as a public university, and transfer to another, such as a for-profit.
Ryan Gallucci, deputy legislative director for Veterans of Foreign Wars, said that while the project has some shortcomings, it represents a step in the right direction.
“This is the first time we’ve had anything even remotely statistically valid to start the conversation on how student veterans are doing” in school, he said. “We don’t know what we don’t know about the student veteran population.”
No comments:
Post a Comment