NRC releases third report on U.S. doctoral programs
INSIDE ILLINOIS, Oct. 7, 2010 | Robert Kisting, News Bureau Intern
The National Research Council has released its latest assessment of U.S. doctoral programs. The study collected data from more than 5,000 doctoral programs in 62 fields at 212 universities. The assessment will enable faculty members and administrators to compare, evaluate and improve their programs and assist potential students in identifying programs best suited to their needs.
“Overall, Illinois programs fared very well in the NRC assessment,” said Deba Dutta, the dean of the UI Graduate College. “Of our 58 programs that were assessed, about two-thirds are in the top 25 percent of all programs in their respective fields. However, as NRC notes, the assessment is not intended to be used as a definitive ranking of ‘best programs.’ ”
Among those Urbana programs that performed exceptionally well are animal sciences, chemistry, electrical and computer engineering, entomology, and materials science and engineering.
Also, because the study uses a different methodology, the NRC rankings cannot be compared with two previous NRC studies (1982 and 1995) or other national rankings, such as those compiled by U.S. News and World Report or Washington Monthly. The large collection of quantitative data is tied to 20 characteristics of doctoral programs, including information on time-to-degree, faculty research productivity, institutional support for students, and the diversity of faculty and students.
Fifty-eight of Urbana’s 95 doctoral programs were included in the assessment, which had several criteria for inclusion. For example, doctoral programs that awarded fewer than five Ph.D.s over a five-year span or others that are considered emerging fields (not enough institutions offer them nationwide) were not included in the survey. In addition, many programs fall outside the scope of the study, including business, education and social work.
The NRC study does not assign a specific numeric rank to doctoral programs, but rather assigns each program two ranges, referred to as the S ranking (survey-based) and R ranking (regression-based). Both rankings provide information about where a program stands in relation to others in the same field.
For example, if a program scores “14, 28,” (in either category) that means the program is ranked between 14th and 28th overall among all the included programs of that kind in the country, with 90 percent certainty. (In other words, there is only a 10 percent probability that the program rank is outside that range.)
The S rankings are based on a survey that asked faculty members to rate the importance of the 20 different program characteristics in determining the quality of the program. Based on their answers, each characteristic was assigned a weight; these weights varied by field. The weights were then applied to the data for each program in the field, resulting in a range of rankings for each program.
The R rankings are based on an indirect way of determining the importance faculty members place on various characteristics. First, groups of randomly selected faculty members were asked to rate the quality of a sample of representative programs in their field. Based on the sample program ratings, weights were assigned to each of the 20 characteristics using statistical techniques. Again, the weights varied by field. These weights were applied to the data about each program resulting in a second range of rankings.
The report also offers illustrative ranges of rankings for each program on three separate dimensions of doctoral education – research activity, student support and outcomes, and diversity. These assessments are based on the S ranking approach, but only characteristics relevant to each category were included in the calculation.
One possible drawback to the survey is the age of the data; the current rankings are based on information collected from 2005-2006. In spite of that, many feel the rankings to be an invaluable tool for doctoral programs on the UI campus.
“While many programs may have changed since the data were collected, the assessment is to date the most comprehensive study of its kind,” Dutta said. “It provides the opportunity for programs to reflect and consider what changes they have made since the data were collected.”
Dutta noted that people need to take care when addressing the rankings. “The purpose of the study was to help programs and prospective students look at various characteristics that matter most to them and then make comparisons,” he said. “Therefore, it can be an important resource for programs to identify areas where they can improve compared to peers and to set benchmarks. This is the real value (of the study) for programs.”
The report also examines overall trends in U.S. doctoral education since the last assessment was released and offers general findings on graduate education in the U.S.
Richard Wheeler, vice chancellor for academic affairs (interim) and vice provost, served on the NRC Committee on Assessment of Research-Doctorate Programs that undertook the project.
20 Characteristics in the NRC assessment
- Publications per allocated faculty member
- Citations per publication
- Percent of faculty members with grants
- Awards per allocated faculty member
- Percent interdisciplinary faculty members
- Percent non-Asian minority faculty members
- Percent female faculty members
- Average GRE scores
- Percent first-year students with full support
- Percent first-year students with external funding
- Percent non-Asian minority students
- Percent female students
- Percent international students
- Average number of Ph.D.s, 2002-2006
- Average completion percentage
- Median time to degree
- Percent of students with academic plans
- Student work space
- Student health insurance
- Number of student activities offered