This is part two of my analysis of instruction and Common Core’s implementation.  I dubbed the three-part examination of instruction “The Good, The Bad, and the Ugly.”  Having discussed “the “good” in part one, I now turn to “the bad.”  One particular aspect of the Common Core math standards—the treatment of standard algorithms in whole number arithmetic—will lead some teachers to waste instructional time.

In 1963, psychologist John B. Carroll published a short essay, “A Model of School Learning” in Teachers College Record.  Carroll proposed a parsimonious model of learning that expressed the degree of learning (or what today is commonly called achievement) as a function of the ratio of time spent on learning to the time needed to learn.     

The numerator, time spent learning, has also been given the term opportunity to learn.  The denominator, time needed to learn, is synonymous with student aptitude.  By expressing aptitude as time needed to learn, Carroll refreshingly broke through his era’s debate about the origins of intelligence (nature vs. nurture) and the vocabulary that labels students as having more or less intelligence. He also spoke directly to a primary challenge of teaching: how to effectively produce learning in classrooms populated by students needing vastly different amounts of time to learn the exact same content.^[i] 

The source of that variation is largely irrelevant to the constraints placed on instructional decisions.  Teachers obviously have limited control over the denominator of the ratio (they must take kids as they are) and less than one might think over the numerator.  Teachers allot time to instruction only after educational authorities have decided the number of hours in the school day, the number of days in the school year, the number of minutes in class periods in middle and high schools, and the amount of time set aside for lunch, recess, passing periods, various pull-out programs, pep rallies, and the like.  There are also announcements over the PA system, stray dogs that may wander into the classroom, and other unscheduled encroachments on instructional time.

The model has had a profound influence on educational thought.  As of July 5, 2015, Google Scholar reported 2,931 citations of Carroll’s article.  Benjamin Bloom’s “mastery learning” was deeply influenced by Carroll.  It is predicated on the idea that optimal learning occurs when time spent on learning—rather than content—is allowed to vary, providing to each student the individual amount of time he or she needs to learn a common curriculum.  This is often referred to as “students working at their own pace,” and progress is measured by mastery of content rather than seat time. David C. Berliner’s 1990 discussion of time includes an analysis of mediating variables in the numerator of Carroll’s model, including the amount of time students are willing to spend on learning.  Carroll called this persistence, and Berliner links the construct to student engagement and time on task—topics of keen interest to researchers today.  Berliner notes that although both are typically described in terms of motivation, they can be measured empirically in increments of time.     

Most applications of Carroll’s model have been interested in what happens when insufficient time is provided for learning—in other words, when the numerator of the ratio is significantly less than the denominator.  When that happens, students don’t have an adequate opportunity to learn.  They need more time. 

As applied to Common Core and instruction, one should also be aware of problems that arise from the inefficient distribution of time.  Time is a limited resource that teachers deploy in the production of learning.  Below I discuss instances when the CCSS-M may lead to the numerator in Carroll’s model being significantly larger than the denominator—when teachers spend more time teaching a concept or skill than is necessary.  Because time is limited and fixed, wasted time on one topic will shorten the amount of time available to teach other topics.  Excessive instructional time may also negatively affect student engagement.  Students who have fully learned content that continues to be taught may become bored; they must endure instruction that they do not need.

Jason Zimba, one of the lead authors of the Common Core Math standards, and Barry Garelick, a critic of the standards, had a recent, interesting exchange about when standard algorithms are called for in the CCSS-M.  A standard algorithm is a series of steps designed to compute accurately and quickly.  In the U.S., students are typically taught the standard algorithms of addition, subtraction, multiplication, and division with whole numbers.  Most readers of this post will recognize the standard algorithm for addition.  It involves lining up two or more multi-digit numbers according to place-value, with one number written over the other, and adding the columns from right to left with “carrying” (or regrouping) as needed.

The standard algorithm is the only algorithm required for students to learn, although others are mentioned beginning with the first grade standards.  Curiously, though, CCSS-M doesn’t require students to know the standard algorithms for addition and subtraction until fourth grade.  This opens the door for a lot of wasted time.  Garelick questioned the wisdom of teaching several alternative strategies for addition.  He asked whether, under the Common Core, only the standard algorithm could be taught—or at least, could it be taught first. As he explains:

Delaying teaching of the standard algorithm until fourth grade and relying on place value “strategies” and drawings to add numbers is thought to provide students with the conceptual understanding of adding and subtracting multi-digit numbers. What happens, instead, is that the means to help learn, explain or memorize the procedure become a procedure unto itself and students are required to use inefficient cumbersome methods for two years. This is done in the belief that the alternative approaches confer understanding, so are superior to the standard algorithm. To teach the standard algorithm first would in reformers’ minds be rote learning. Reformers believe that by having students using strategies in lieu of the standard algorithm, students are still learning “skills” (albeit inefficient and confusing ones), and these skills support understanding of the standard algorithm. Students are left with a panoply of methods (praised as a good thing because students should have more than one way to solve problems), that confuse more than enlighten. 

Zimba responded that the standard algorithm could, indeed, be the only method taught because it meets a crucial test: reinforcing knowledge of place value and the properties of operations.  He goes on to say that other algorithms also may be taught that are consistent with the standards, but that the decision to do so is left in the hands of local educators and curriculum designers:

In short, the Common Core requires the standard algorithm; additional algorithms aren’t named, and they aren’t required…Standards can’t settle every disagreement—nor should they. As this discussion of just a single slice of the math curriculum illustrates, teachers and curriculum authors following the standards still may, and still must, make an enormous range of decisions.

Zimba defends delaying mastery of the standard algorithm until fourth grade, referring to it as a “culminating” standard that he would, if he were teaching, introduce in earlier grades.  Zimba illustrates the curricular progression he would employ in a table, showing that he would introduce the standard algorithm for addition late in first grade (with two-digit addends) and then extend the complexity of its use and provide practice towards fluency until reaching the culminating standard in fourth grade. Zimba would introduce the subtraction algorithm in second grade and similarly ramp up its complexity until fourth grade.

It is important to note that in CCSS-M the word “algorithm” appears for the first time (in plural form) in the third grade standards:

3.NBT.2  Fluently add and subtract within 1000 using strategies and algorithms based on place value, properties of operations, and/or the relationship between addition and subtraction.

The term “strategies and algorithms” is curious.  Zimba explains, “It is true that the word ‘algorithms’ here is plural, but that could be read as simply leaving more choice in the hands of the teacher about which algorithm(s) to teach—not as a requirement for each student to learn two or more general algorithms for each operation!” 

I have described before the “dog whistles” embedded in the Common Core, signals to educational progressives—in this case, math reformers—that  despite these being standards, the CCSS-M will allow them great latitude.  Using the plural “algorithms” in this third grade standard and not specifying the standard algorithm until fourth grade is a perfect example of such a dog whistle.

It appears that the Common Core authors wanted to reach a political compromise on standard algorithms. 

Standard algorithms were a key point of contention in the “Math Wars” of the 1990s.   The 1997 California Framework for Mathematics required that students know the standard algorithms for all four operations—addition, subtraction, multiplication, and division—by the end of fourth grade.^[ii]  The 2000 Massachusetts Mathematics Curriculum Framework called for learning the standard algorithms for addition and subtraction by the end of second grade and for multiplication and division by the end of fourth grade.  These two frameworks were heavily influenced by mathematicians (from Stanford in California and Harvard in Massachusetts) and quickly became favorites of math traditionalists.  In both states’ frameworks, the standard algorithm requirements were in direct opposition to the reform-oriented frameworks that preceded them—in which standard algorithms were barely mentioned and alternative algorithms or “strategies” were encouraged. 

Now that the CCSS-M has replaced these two frameworks, the requirement for knowing the standard algorithms in California and Massachusetts slips from third or fourth grade all the way to sixth grade.  That’s what reformers get in the compromise.  They are given a green light to continue teaching alternative algorithms, as long as the algorithms are consistent with teaching place value and properties of arithmetic.  But the standard algorithm is the only one students are required to learn.  And that exclusivity is intended to please the traditionalists.

I agree with Garelick that the compromise leads to problems.  In a 2013 Chalkboard post, I described a first grade math program in which parents were explicitly requested not to teach the standard algorithm for addition when helping their children at home.  The students were being taught how to represent addition with drawings that clustered objects into groups of ten.  The exercises were both time consuming and tedious.  When the parents met with the school principal to discuss the matter, the principal told them that the math program was following the Common Core by promoting deeper learning.  The parents withdrew their child from the school and enrolled him in private school.

The value of standard algorithms is that they are efficient and packed with mathematics.  Once students have mastered single-digit operations and the meaning of place value, the standard algorithms reveal to students that they can take procedures that they already know work well with one- and two-digit numbers, and by applying them over and over again, solve problems with large numbers.  Traditionalists and reformers have different goals.  Reformers believe exposure to several algorithms encourages flexible thinking and the ability to draw on multiple strategies for solving problems.  Traditionalists believe that a bigger problem than students learning too few algorithms is that too few students learn even one algorithm.

I have been a critic of the math reform movement since I taught in the 1980s.  But some of their complaints have merit.  All too often, instruction on standard algorithms has left out meaning.  As Karen C. Fuson and Sybilla Beckmann point out, “an unfortunate dichotomy” emerged in math instruction: teachers taught “strategies” that implied understanding and “algorithms” that implied procedural steps that were to be memorized.  Michael Battista’s research has provided many instances of students clinging to algorithms without understanding.  He gives an example of a student who has not quite mastered the standard algorithm for addition and makes numerous errors on a worksheet.  On one item, for example, the student forgets to carry and calculates that 19 + 6 = 15.  In a post-worksheet interview, the student counts 6 units from 19 and arrives at 25.  Despite the obvious discrepancy—(25 is not 15, the student agrees)—he declares that his answers on the worksheet must be correct because the algorithm he used “always works.”^[iii] 

Math reformers rightfully argue that blind faith in procedure has no place in a thinking mathematical classroom. Who can disagree with that?  Students should be able to evaluate the validity of answers, regardless of the procedures used, and propose alternative solutions.  Standard algorithms are tools to help them do that, but students must be able to apply them, not in a robotic way, but with understanding.

Let’s return to Carroll’s model of time and learning.  I conclude by making two points—one about curriculum and instruction, the other about implementation.

In the study of numbers, a coherent K-12 math curriculum, similar to that of the previous California and Massachusetts frameworks, can be sketched in a few short sentences.  Addition with whole numbers (including the standard algorithm) is taught in first grade, subtraction in second grade, multiplication in third grade, and division in fourth grade.  Thus, the study of whole number arithmetic is completed by the end of fourth grade.  Grades five through seven focus on rational numbers (fractions, decimals, percentages), and grades eight through twelve study advanced mathematics.  Proficiency is sought along three dimensions:  1) fluency with calculations, 2) conceptual understanding, 3) ability to solve problems.

Placing the CCSS-M standard for knowing the standard algorithms of addition and subtraction in fourth grade delays this progression by two years.  Placing the standard for the division algorithm in sixth grade continues the two-year delay.   For many fourth graders, time spent working on addition and subtraction will be wasted time.  They already have a firm understanding of addition and subtraction.  The same thing for many sixth graders—time devoted to the division algorithm will be wasted time that should be devoted to the study of rational numbers.  The numerator in Carroll’s instructional time model will be greater than the denominator, indicating the inefficient allocation of time to instruction.

As Jason Zimba points out, not everyone agrees on when the standard algorithms should be taught, the alternative algorithms that should be taught, the manner in which any algorithm should be taught, or the amount of instructional time that should be spent on computational procedures.  Such decisions are made by local educators.  Variation in these decisions will introduce variation in the implementation of the math standards.  It is true that standards, any standards, cannot control implementation, especially the twists and turns in how they are interpreted by educators and brought to life in classroom instruction.  But in this case, the standards themselves are responsible for the myriad approaches, many unproductive, that we are sure to see as schools teach various algorithms under the Common Core.

       

Efforts are moving ahead to reauthorize the Elementary and Secondary Education Act (ESEA) and its broad parameters are becoming clearer. The bill is likely to keep an annual testing requirement, but change accountability structures created under No Child Left Behind (NCLB), allowing states to create their own. But, like NCLB, the new bill will shortchange older students and those at risk of dropping out of school. The Alliance for Excellent Education has called this phenomenon the “missing middle." Federal education spending is U-shaped, with large amounts spent on programs for young students and large amounts spent on college students in the form of student financial aid. The middle is missing, especially for students in secondary schools.

Recently, the press has heralded increases in the rate of high school students graduating on time; it’s now above 80 percent. The on-time graduation rate going up is welcome news, and perhaps NCLB played a role. But 20 percent of students still do not graduate on time. Imagine if one in five new cell phones didn’t work. Policymakers would be deluged by complaints and critics would be decrying the cavalier way the industry treats its customers.

Some students who do not graduate on time may ultimately graduate, or receive a General Education Development certificate. But many never graduate. The National Center for Education Statistics reports that there were about 14.5 million high school students in 2012, and that four percent of them stopped attending high school that year. Four percent seems small but the base is large, and together they imply about 580,000 students stopped attending school. They dropped out. To put this number in perspective, there were 584,000 deaths from cancer last year in the U.S.

Dropping out is a stubborn problem. Even as the number of teen mothers has declined sharply, and juvenile arrests likewise have declined, the dropout rate—the percent of students who stop attending school in a year—has only fallen gradually from six percent to four percent in the last forty years. And the dropout rate is not equal for all students. In 2009, it was two percent for white students, five percent for black students, and six percent for Hispanic students. It was 1.4 percent for students from high-income households and 7.4 percent—five times larger—for students from low-income households.

The American economy over the past three decades has moved toward high-skill jobs. There is no sign that this trend is abating. Yet more than half a million young people will stop attending high school this year without a diploma. They are not ready for college or a career and face a rough road ahead for jobs and earnings. Based on research by Cecilia Rouse, the National Center for Education Statistics estimated that dropouts will earn $630,000 less than graduates in their working lives. So, every year, the economy loses more than $300 hundred billion in foregone earnings. Even in a big country, this is a big number.

And dropouts are more likely to receive public assistance from the alphabet soup of programs: TANF, SNAP, WIC, CHIP, and so on. They also are more likely to end up in jail or prison, which is hugely expensive. Texas alone spent more than $3 billion on prisons in 2010.

The one-two combination of lower earnings and more public assistance means it is very much in the public interest to reduce dropping out. The federal government began actively studying programs to lower the dropout rate starting with the Hawkins-Stafford bill in 1988. Those research efforts wrapped up in 1998 with the release of two reports prepared by Mathematica Policy Research for the U.S. Department of Education (here and here). The reports summarized findings from rigorous studies in 21 school districts (I was the principal investigator for these studies). Not many programs improved outcomes. It was a starting point and clearly there was more to learn.

The federal government followed that research with, well, almost nothing. The What Works Clearinghouse did release a practice guide about dropout prevention in 2008. Most of the research that the guide reviewed was from the nineties. And the WWC has reviewed research for 30 dropout-prevention programs. Many had been studied as part of the same federal evaluation and other reviews found that there is not enough research meeting standards to know whether programs were effective.

More recently, the National Center for Education Statistics surveyed school districts about their dropout prevention programs. It was the first survey NCES had conducted of dropout prevention programs. (The Government Accountability Office surveyed programs in 1987.) A lot was learned from the NCES survey, including that nearly every school district was providing services and programs for its students at risk of dropping out. For example, 99 percent of large districts (ones that enroll more than 10,000 students), offered alternative high school programs.

Though not reported in the NCES survey, it’s evident that districts are funding dropout prevention services with their own resources or state resources. Not much ESEA Title I money reaches secondary schools—only 32 percent of secondary schools get any Title I funds, and they get about $500 per low-income student. Direct federal spending on dropout prevention is only about $5 million a year. That’s less than ten dollars per dropout.

“Early intervention” is the idea that preventing problems can be more effective than treating them. Following that logic might suggest focusing on pre-K and elementary schools so that fewer students struggle in middle and high schools, which can lead to dropping out. But early intervention spends money on students who may not develop problems. Waiting until issues develop can be cost-effective because which students to help becomes evident. Not many first graders are experimenting with drugs or being arrested for delinquency, but when it happens with an eighth grader, the problem is clear.

Spending more money on pre-K or elementary schools may be desirable for other reasons, but if reducing the dropout rate is the focus, middle schools are the key. Students who enter high school poorly prepared and with bad habits—missing a lot of school, acting out, failing subjects—are known to have a high likelihood of dropping out. The Bush Institute currently is spearheading an initiative to promote middle school reform with the University of Texas. The Johns Hopkins University continues to refine its ‘Talent Development’ model for middle schools, but, overall, there are not many efforts related to improving middle schools  in order to reduce dropping out. The National Dropout Prevention Network at Clemson University has accumulated other resources and links on what states and districts are doing to reduce the dropout rate.

The lack of federal attention to dropping out is highlighted by the “here today, gone tomorrow” nature of early warning systems. Early warning systems are databases that flag students showing signs of difficulty in school. Using early warning systems allows resources to be targeted to students who need help. In 2008, drafts of a reauthorized ESEA included support for districts to develop early-warning systems. Recent drafts of ESEA do not mention them at all. Where did they go? The federal government already has granted more than $600 million to states to support their efforts to develop student databases. It would be straightforward for it to provide additional impetus to refine those systems so that they provide early warning capabilities.

Graduating from high school is a win-win opportunity to improve earnings and reduce public assistance. There are ample reasons for the federal government to play a role and not leave it up to states and localities to help students graduate or to study ways to promote graduation. With $300 billion a year on the table, even small reductions in the dropout rate are likely to pay handsome returns.

Of course, there are priorities in policymaking, but reducing dropping out should be one of them. Senator Warren introduced an amendment to ESEA that would direct additional resources to high schools with very high dropout rates. Focusing on middle schools will be a better use of funds.

       

Student refusals to take standardized tests surged in New York State this spring, fueled by support from both parent activists and the state teacher’s union. According to the New York Times, the opt-out movement more than doubled the number of students who did not take federally mandated math and English Language Arts (ELA) tests, with 165,000 kids—about one in six—not taking at least one of the tests.

These total numbers mask enormous variation across communities. According to the Times analysis, barely any students opted out in some schools districts, while in other districts a majority of students refused the tests. Are these differences in opt-out rates random, or are they associated with the characteristics of the community served by each district? Do opt-outs tend to be concentrated among relatively affluent districts, or are they most common in schools that have historically performed poorly on state tests?

The data needed to best answer these questions are the student-level test-score and demographic information collected by the New York State Department of Education. They will allow the state education department to conduct fine-grained analyses, such as comparing the characteristics of students who took the tests to those who did not. But while we are waiting for the state to publish summary information based on those data, some light can be shed on these questions using publicly available data.

My primary data source for the following analysis is a table indicating the number of students who opted out of the math and ELA tests out in each school district this spring. The data were compiled by United to Counter the Core, an opt-out advocacy organization, from a combination of media stories, freedom of information requests, and reports by administrators, teachers, and parents. Data are available for most but not all districts, and there is likely some misreporting.^[1] However, to my knowledge, it is the most comprehensive opt-out dataset currently publicly available.

I merged the opt-out information with the most recent available enrollment and demographic data from the U.S. Department of Education’s Common Core of Data.^[2] I use total enrollment in grades 3-8 to estimate the percentage of students who opted out (i.e. the number of opt-outs, which are presumably for tests in grades 3-8, divided by the number of students enrolled in those grades).^[3] I also calculate the percentage of students in all grades who were eligible for the federal free or reduced-price lunch program, an indicator of socioeconomic disadvantage. Finally, I obtained average student scores on the New York State tests last spring (2014), before opt-out became a widespread phenomenon.^[4]

The 648 districts with complete data available had an average opt-out rate of 28 percent (the rates are averaged across the math and ELA tests). But weighting each district by its enrollment shows that an estimated 21 percent of all students at these districts opted out. The difference between these numbers implies that larger districts tend to have lower opt-out rates.

The table below confirms that opt-out rates vary widely across districts, with 19 percent of districts having an opt-out rate below 10 percent, 30 percent of districts in the 10-25 percent range, 38 percent in the 25-50 percent category, and 13 percent seeing a majority of students opt out. Districts with higher opt-out rates tend to serve fewer disadvantaged students and have somewhat higher test scores (which is not surprising given the correlation between family income and test scores). District size is similar across three of the opt-out rate categories, but districts with the lowest opt-out rates tend to be substantially larger, on average.

District Characteristics, by Opt-Out Rate

Notes: Average test scores are reported in student-level standard deviations.

But no single student characteristic is a perfect predictor of opt-out rates. The figure below plots the opt-out rate and free/reduced lunch percentage of every district. There is a clear association, with more disadvantaged districts having lower opt-out rates, on average, but also a large amount of variation in the opt-out rate among districts with similar shares of students eligible for the subsidized lunch program. Could variation in other factors such as test-score performance explain some of those remaining differences?

Relationship between Opting Out and Percent Free/Reduced Lunch

Notes: Line is based on a Lowess smoother.

I address this question using a regression analysis that measures the degree to which percent free/reduced lunch and average test scores are associated with opt-out, controlling for the other factor. In order to make the two relationships comparable, I report the predicted change in opt-out rates expected based on a one standard deviation change in percent free/reduced lunch or average test scores. Among the districts in the data, one standard deviation in percent free/reduced lunch is 21 percentage points and one standard deviation in average test scores is 0.35 student-level standard deviations.

Perhaps the most surprising result of the analysis, reported in the figure below, is that the modest positive correlation between test scores and opt-out seen in the table above becomes negative once free/reduced lunch is taken into account.^[5] The results below indicate that a one standard deviation increase in test scores is associated with a seven-percentage-point decline in the opt-out rate. That is a large change given that the average opt-out rate is 28 percent. Another way to describe the same relationship is that districts with lower scores have higher opt-out rates.

This analysis confirms that districts serving more disadvantaged students have lower opt-out rates, even after test scores are taken into account. A one standard deviation increase in the share of students eligible for free/reduced lunch is associated with an 11-percentage-point decrease in the opt-out rate. These relationships are even stronger when districts are weighted proportional to their enrollment, as shown in the right pair of bars in the figure below. This may be because these relationships are stronger in larger districts, or because opt-out is measured with less error in larger districts.

Correlation between District Characteristics and Opt-Out Rates

These findings should be interpreted with two caveats in mind. First, the data are incomplete, preliminary, and likely suffer from some reporting errors. Second, the data are at the district level, limiting the ability to make inferences about individual students. For example, just because lower-scoring districts have higher opt-out rates (controlling for free/reduced lunch) does not mean that lower-scoring students are more likely to opt out. It could be the higher-scoring students in those districts that are doing the opting out.

Student-level administrative data collected by the state will ultimately provide conclusive evidence not subject to these limitations, but the district-level data currently available yield two preliminary conclusions. First, relatively affluent districts tend to have higher opt-out rates, with opt-out less common in the disadvantaged districts that are often the target of reform efforts. Second, districts with lower test scores have higher opt-out rates after taking socioeconomic status into account. Potential explanation for this pattern include district administrators encouraging opt-outs in order to cover up poor performance, districts focusing on non-tested subjects to satisfy parents who care less about standardized tests, and parents becoming more skeptical of the value of tests when their children do not score well. Rigorous testing of these and alternative explanation for opt-out behavior await better data.

       

The Best Foot Forward project at the Center for Education Policy Research at Harvard has been investigating the use of digital video to make classroom observations more helpful and fair to teachers and less burdensome for supervisors. In a randomized field trial involving 347 teachers and 108 administrators in Delaware, Georgia, Colorado and Los Angeles, teachers were given a special video camera and invited to collect multiple lessons.  They could then choose a subset of their lesson videos to submit for their classroom observations.  A secure software platform allowed administrators as well as external observers (selected for their expertise in a teacher’s discipline) to watch the videos and provide time-stamped comments aligned to specific moments in the videos.  

In addition to giving teachers a reason and an opportunity to watch multiple instances of their own teaching, the videos served as the basis for one-on-one discussions between teachers and administrators and between teachers and the external content experts.  The comparison teachers and schools continued to do in-person classroom observations.  Although we’re awaiting data from a second year of implementation, we can report five preliminary findings so far:

• Despite teachers’ initial discomfort with collecting and watching video of their own instruction, the intervention did shift the mode of classroom observations, from in-person to video.  Treatment teachers collected an average of 13 videos of their lessons.  The average treatment teacher had 2.85 formal observations based on their submitted videos as well as two no-stakes observations by external observations.  Treatment teachers also reported 1.06 fewer in-person classroom observations.

• Although their final official observation scores were no different than comparison teachers, treatment teachers perceived their supervisors to be more supportive and their observations to be fairer.  They reported fewer disagreements over scoring and were more likely to be able to describe a specific change in their practice resulting from their post-observation conference.  Treatment administrators reported that their post-observation conferences with teachers were less defensive.

• The opportunity to watch their own lessons seems to have made treatment teachers more self-critical.  They rated their own instruction lower than comparison teachers, particularly in terms of time management and their ability to assess student mastery during class.   

• We learned that the use of video did not save administrators’ time; in fact, treatment administrators reported spending more time on the observation process than the control group.  However, the ability to watch video did allow supervisors to shift their observation duties to quieter times of the day or week:  more than half of all log-ins occurred during lunch hour, the two hours immediately after school, evenings, weekends, and holidays.  

• Although both treatment and control teachers volunteered to participate in the project, the treatment teachers were more likely to support use of video at the end of the first year, as a replacement for some or all of their in-person classroom observations.  

In sum, the use of teacher-collected video in classroom observations did seem to improve the classroom observation process along a number of dimensions:  it boosted teachers’ perception of fairness of classroom observations, reduced teacher defensiveness during post-observation conferences, led to greater self- perception of the need for behavior change and allowed administrators to time-shift observation duties to quieter times of the day or week.  In coming months, we will provide evidence on whether or not these apparent improvements in the observation process were sufficient to generate improvements in student achievement.

For more on the project, please see: http://cepr.harvard.edu/best-foot-forward-project.

       

Good morning Chairman Alexander, Ranking Member Murray, and distinguished Members of the Committee. Thank you for giving me the opportunity to be here today to share my thoughts on this very important issue. 

My name is Beth Akers.  I am a fellow at the Brookings Institution where I carry out research on the topic of higher education, with a particular focus on student loans.  I’ve been engaged in research related to higher education policy since 2008 when, in my role as Staff Economist at the Council of Economic Advisers, I assisted the Department of Education as they quickly implemented the Ensuring Continued Access to Student Loans Act.  My testimony is informed by the time that I’ve spent engaged as a researcher in this field, first as a graduate student in the Economics Department at Columbia University and then as a Fellow at the Brookings Institution. 

Over the past two decades there’s been a dramatic increase in the share of young U.S. households with education debt.  The incidence has more than doubled, from 14 percent in 1989 to 38 percent in 2013 (Table 1). Not only are more individuals taking out education loans, but they are also taking out larger loans. Among households with debt, the mean per-person debt more than tripled, from $5,810 to $19,341 during the same period (2010 dollars). Median debt grew somewhat less rapidly, from $3,517 to $10,390 (Figure 1, Table 1). Among all households, including those with no debt, mean debt increased eightfold, from $806 to $7,382 (Table 1).

Figure 1. Trends in Education Debt over Time, 1989-2013

Notes: Based on households age 20-40 with education debt.
Source: Akers and Chingos 2014b

Only a trivial number of households had more than $20,000 in debt (per person) in 1989/1992, whereas in 2013, almost one third of those with debt had balances exceeding $20,000 (the change in the distribution is illustrated in Figure 2). The incidence of very large debt balances is greater now than it was two decades ago, but it is still quite rare.  In 2013, seven percent of households with debt had balances in excess of $50,000 and two percent had balances over $100,000 (Akers and Chingos 2014b).

Figure 2. Distribution of Debt, 1989/1992 and 2013

Notes: Based on households age 20-40 with education debt. All amounts are in 2010 dollars.
Source: Akers and Chingos 2014b

The large increases in education debt levels over the last two decades are often attributed to the increases in tuition charged by colleges and universities. There is also evidence that college students are relying more on debt to finance college costs and paying less out-of-pocket (Greenstone and Looney 2013b), suggesting that student behavior is changing in ways that favor loans over other ways of paying for college. Furthermore, there have been shifts in the level of educational attainment and demographic characteristics of the U.S. college-age population that could impact observed student borrowing.  Estimates suggest that roughly one-quarter of the increase in student debt since 1989 can be directly attributed to Americans obtaining more education (both through increased enrollment and increased levels of attainment) while increases in tuition can explain 51 percent of the increase in debt observed during this period (Akers and Chingos 2014a). 

Recognizing that the increases in borrowing are driven by multiple factors, some of which are less concerning than others, highlights an important point.  The growth in student loan debt is often discussed as a problem in and of itself. However, to the extent that borrowers are using debt as a tool to finance investments in human capital that pay off through higher wages in the future, increases in debt may simply be a benign symptom of increasing expenditure on higher education.  On the contrary, if these expenditures were spent in ways that don’t pay dividends in the future, then the observed growth in debt may indicate problems for the financial future of borrowers.   

Positive Return on Investment

The most direct way to examine whether borrowers are using debt to finance investments that will pay off is to measure the financial return that their investment will yield in terms of lifetime earnings (relative to what they would have earned if they had not enrolled in a program of higher education) and compare it to the upfront cost of enrollment.  Despite the recent recession, the significant economic return to college education continues to grow, implying that many of these loans are financing sound investments. In 2011, college graduates between the ages of 23 and 25 earned $12,000 more per year, on average, than high school graduates in the same age group, and had employment rates 20 percentage points higher. Over the last 30 years, the increase in lifetime earnings associated with earning a bachelor’s degree has grown by 75 percent, while costs have grown by 50 percent (Greenstone and Looney 2010). There is also an earnings premium associated with attending college and earning an associate’s degree or no degree at all, although it is not as large (Greenstone and Looney 2013a). These economic benefits accrue to individuals, but also to society, in the form of increased tax revenue, improved health, and higher levels of civic participation (Baum, Ma, and Payea 2013). 

Studies that seek to identify the causal relationship between education and earnings draw similar conclusions.  A recent study, published by researchers at the Federal Reserve Bank of New York in 2014, suggested that the financial return on a college degree, when expressed as a rate of return, was 15 percent and had held steady at that level (a historic high) for the previous decade.  A valuable insight from this work is that the return on college has not fallen, despite the growing cost of attendance and stagnant earnings growth across the economy.  This counterintuitive result is driven by the decline of earnings among workers without college degrees (Abel and Deitz, 2014).  These statistics indicating large financial returns on investments in higher education suggest that, for the average student, college will pay for itself in the long run. 

Month-to-month Affordability of Student Debt

The long run financial return is an important indicator of affordability, but it could potentially obscure more transient challenges faced by households. For example, an increase in debt may be affordable in the long run but impose monthly payments that squeeze borrowers in the short run, especially early in their careers when earnings are low. However, month-to-month affordability of student debt does not seem to have declined in recent history.  The ratio of monthly payments to monthly income has been flat over the last two decades (Figure 3, Table 2). Median monthly payments ranged between three and four percent of monthly earnings in every year from 1992 through 2013.  Mean monthly payments, which are larger than median payments in each year due to the distribution being right-skewed, declined from 15 percent in 1992 to 7 percent in 2013 (Akers and Chingos 2014b).

Figure 3. Monthly Payment-to-Income Ratios, 1992-2013

Notes: Based on households age 20-40 with education debt, wage income of at least $1,000, and that were making positive monthly payments.
Source: Akers and Chingos 2014

The ratio of monthly payments to monthly income stayed roughly the same over time, on average, at each percentile and for each education category. By this measure, the transitory burden of loan repayment is no greater for today’s young workers than it was for young workers two decades ago. If anything, the monthly repayment burden has lessened.

This surprising finding can be explained in part by a lengthening of average repayment terms during the same period. In 1992, the mean term of repayment was 7.5 years, which increased to 12.5 years in 2013. This increase was likely due primarily to loan consolidation, which increased dramatically in the early 2000s (Department of Education 2014, S-16). Loans consolidated with the federal government are eligible for extended repayment terms based on the outstanding balance, with larger debts eligible for longer repayment terms.  Average interest rates also declined during this period, which would also lower monthly payments (Table 3).

In order to appreciate how much of a burden monthly payments place on households, it’s useful to compare student debt payments to other household expenses.  In Figure 4 average monthly student loan payment (based on data from 2010) is plotted together with the average monthly expenditure in each major consumption category (this data comes from the 2012 Consumer Expenditure Survey, which is administered by the Bureau of Labor Statistics).  The largest categories of monthly consumption expenditure are housing ($1,407), transportation ($750) and food ($588).  Monthly student loan payments are relatively small compared to these expenses, and at $242, are closer in scale to monthly spending on entertainment ($217), apparel ($145) and health care ($296).  There is relatively little variation in monthly loan payments (due to consolidation with longer repayment terms for larger debts) (Akers 2014a). 

Figure 4. Average Monthly Expenditures and Student Loan Payments

Data: 2010 Survey of Consumer Finances and 2012 Consumer Expenditure Survey
Source: Akers 2014a

Student Debt is a Poor Indicator of Economic Hardship       

It might seem reasonable to be most concerned about the plight individuals with large outstanding student loan balances, but evidence suggests that these individuals may not be faring any worse than households with smaller balances or no student debt at all.  The highest rates of financial distress, as indicated by late payments on household financial obligations, are seen among households with the lowest levels of student loan debt.  Households with large debts tend to have higher levels of educational attainment and earnings, on average, and miss bill payments less often.  Among households with outstanding education debt in the lowest quartile of the debt distribution ($0 - $3,386), 34 percent report having made a late payment on a financial obligation in the past year compared with 26 percent of households with education debt in the highest quartile (≥$18,930).  Households with student loan debt do not show indications of financial distress more often than households without student loan debt (Akers 2014b).          

This body of evidence contradicts the notion that a crisis of college affordability exists on a macro level.  However, it is undeniable that many individuals and households are facing serious economic hardship that can be explained completely or in part by their spending on higher education.  Like any other investment, the returns to higher education are not guaranteed.  While the average student will see a large financial return on the dollars they spend on higher education, some students will find that their investment won’t pay off.  We can reduce the frequency of this occurrence by ensuring that students have the information and resources they need in order to make good decisions about college enrollment.  For instance, a national level data base that reports earnings by institution would succeed in helping student to avoid enrolling at institutions that do not have a track record of success.  This would succeed in creating more institutional accountability without additional government intervention. 

An additional way to improve outcomes for students is to simplify the federal lending program both on the front end, with the menu of services, and also on the backend with a more streamlined system of repayment.  Recent work on this issue has revealed that students have relatively little understanding of their financial circumstances while they are enrolled in college.  About half of all first-year students in the U.S. seriously underestimate how much debt they’ve taken on.  Even more concerning is the fact that among all first-year students with federal student loans, 28 percent report having no federal debt and 14 percent report that they have no debt at all (Akers and Chingos 2014c).  Removing the complexity of the federal aid system could potentially succeed in making it easier for students to comprehend their circumstances and to make better informed decisions. 

However, some of the uncertainty about the payoff of college is unavoidable.  For example, some students will invest in developing skills that will ultimately become obsolete due to unanticipated technological or policy innovation.  It’s important that the government provide insurance against these types of occurrences both for the sake of ensuring individual welfare and also to discourage debt aversion among potential students.  Income driven payment programs, like the ones currently in place for the federal student lending program, are the appropriate tool for providing a safety net to borrowers.     

In sum, college is affordable in the sense that on average it will pay for itself in the long run with heightened wages.  However, to ensure that college is universally affordable ex-post, it’s necessary to maintain a robust system of income driven repayment such that students are insured against their investment not paying off.  Lastly, we need to ensure that both the system of federal lending and the safety nets that exist to support it are simple enough that the benefits of these policy innovations can be fully realized. 

References

Jaison R. Abel and Richard Deitz, “Do the Benefits of College Still Outweigh the Costs?” Federal Reserve Bank of New York Current Issues in Economics and Finance, vol. 20, no. 3 (2014), available at http://www.newyorkfed.org/research/current_issues/ci20-3.html.

Elizabeth Akers and Matthew M. Chingos. 2014a.“Is a Student Loan Crisis on the Horizon?” Brown Center on Education Policy, Brookings Institution, available at http://www.brookings.edu/research/reports/2014/06/24-student-loan-crisis-akers-chingos.

Elizabeth Akers and Matthew M. Chingos. 2014b. “Student Loan Update: A First Look at the 2013 Survey of Consumer Finances.” Washington, DC: Brown Center on Education Policy, Brookings Institution, available at http://www.brookings.edu/research/papers/2014/09/08-student-loan-update-akers-chingos.

Elizabeth Akers and Matthew M. Chingos. 2014c. “Are College Students Borrowing Blindly?” Washington, DC: Brown Center on Education Policy, Brookings Institution, available at http://www.brookings.edu/research/reports/2014/12/10-borrowing-blindly-akers-chingos.

Elizabeth Akers. 2014a. “They Typical Household with Student Loan Debt.” Washington, DC: Brown Center on Education Policy, Brookings Institution, available at http://www.brookings.edu/research/papers/2014/06/19-typical-student-loan-debt-akers.

Elizabeth Akers, 2014b. “How Much is Too Much? Evidence on Financial Well-Being and Student Loan Debt.” Washington, DC: Center on Higher Education Reform, American Enterprise Institute, available at http://www.aei.org/publication/how-much-is-too-much-evidence-on-financial-well-being-and-student-loan-debt/.

Sandy Baum, Jennifer Ma and Kathleen Payea. 2013. "Education Pays, 2013." Washington, DC: The College Board, available at https://trends.collegeboard.org/sites/default/files/education-pays-2013-full-report.pdf.

Department of Education. 2014. “Student Loans Overview: Fiscal Year 2015 Budget Proposal.” Washington, DC: U.S. Department of Education. http://www2.ed.gov/about/overview/budget/budget15/justifications/s-loansoverview.pdf (accessed June 13, 2014).

Michael Greenstone and Adam Looney. 2010. “Regardless of the Cost, College Still Matters.” Brookings on Job Numbers blog, October 5. http://www.brookings.edu/blogs/jobs/posts/2012/10/05-jobs-greenstone-looney.

Michael Greenstone, and Adam Looney. 2013a. “Is Starting College and Not Finishing Really that Bad?” Washington, DC: Brookings Institution, Hamilton Project, available athttp://www.hamiltonproject.org/files/downloads_and_links/May_Jobs_Blog_20130607_FINAL_2.pdf

Michael Greenstone, and Adam Looney. 2013b. “Rising Student Debt Burdens: Factors behind the Phenomenon.” Brookings on Job Numbers blog, July 5, available athttp://www.brookings.edu/blogs/jobs/posts/2013/07/05-student-loans-debtburdens-jobs-greenstone-looney.

       

This post continues a series begun in 2014 on implementing the Common Core State Standards (CCSS).  The first installment introduced an analytical scheme investigating CCSS implementation along four dimensions:  curriculum, instruction, assessment, and accountability.  Three posts focused on curriculum.  This post turns to instruction.  Although the impact of CCSS on how teachers teach is discussed, the post is also concerned with the inverse relationship, how decisions that teachers make about instruction shape the implementation of CCSS.

A couple of points before we get started.  The previous posts on curriculum led readers from the upper levels of the educational system—federal and state policies—down to curricular decisions made “in the trenches”—in districts, schools, and classrooms.  Standards emanate from the top of the system and are produced by politicians, policymakers, and experts.  Curricular decisions are shared across education’s systemic levels.  Instruction, on the other hand, is dominated by practitioners.  The daily decisions that teachers make about how to teach under CCSS—and not the idealizations of instruction embraced by upper-level authorities—will ultimately determine what “CCSS instruction” really means.

I ended the last post on CCSS by describing how curriculum and instruction can be so closely intertwined that the boundary between them is blurred.  Sometimes stating a precise curricular objective dictates, or at least constrains, the range of instructional strategies that teachers may consider.  That post focused on English-Language Arts.  The current post focuses on mathematics in the elementary grades and describes examples of how CCSS will shape math instruction.  As a former elementary school teacher, I offer my own personal opinion on these effects.

Certain aspects of the Common Core, when implemented, are likely to have a positive impact on the instruction of mathematics. For example, Common Core stresses that students recognize fractions as numbers on a number line.  The emphasis begins in third grade:

CCSS.MATH.CONTENT.3.NF.A.2
Understand a fraction as a number on the number line; represent fractions on a number line diagram.

CCSS.MATH.CONTENT.3.NF.A.2.A
Represent a fraction 1/b on a number line diagram by defining the interval from 0 to 1 as the whole and partitioning it into b equal parts. Recognize that each part has size 1/b and that the endpoint of the part based at 0 locates the number 1/b on the number line.

CCSS.MATH.CONTENT.3.NF.A.2.B
Represent a fraction a/b on a number line diagram by marking off a lengths 1/b from 0. Recognize that the resulting interval has size a/b and that its endpoint locates the number a/b on the number line.

When I first read this section of the Common Core standards, I stood up and cheered.  Berkeley mathematician Hung-Hsi Wu has been working with teachers for years to get them to understand the importance of using number lines in teaching fractions.^[1] American textbooks rely heavily on part-whole representations to introduce fractions.  Typically, students see pizzas and apples and other objects—typically other foods or money—that are divided up into equal parts.  Such models are limited.  They work okay with simple addition and subtraction.  Common denominators present a bit of a challenge, but ½ pizza can be shown to be also 2/4, a half dollar equal to two quarters, and so on. 

With multiplication and division, all the little tricks students learned with whole number arithmetic suddenly go haywire.  Students are accustomed to the fact that multiplying two whole numbers yields a product that is larger than either number being multiplied: 4 X 5 = 20 and 20 is larger than both 4 and 5.^[2]  How in the world can ¼ X 1/5 = 1/20, a number much smaller than either 1/4or 1/5?  The part-whole representation has convinced many students that fractions are not numbers.  Instead, they are seen as strange expressions comprising two numbers with a small horizontal bar separating them. 

I taught sixth grade but occasionally visited my colleagues’ classes in the lower grades.  I recall one exchange with second or third graders that went something like this:

“Give me a number between seven and nine.”  Giggles. 

“Eight!” they shouted. 

“Give me a number between two and three.”  Giggles.

“There isn’t one!” they shouted. 

“Really?” I’d ask and draw a number line.  After spending some time placing whole numbers on the number line, I’d observe,  “There’s a lot of space between two and three.  Is it just empty?” 

Silence.  Puzzled little faces.  Then a quiet voice.  “Two and a half?”

You have no idea how many children do not make the transition to understanding fractions as numbers and because of stumbling at this crucial stage, spend the rest of their careers as students of mathematics convinced that fractions are an impenetrable mystery.   And  that’s not true of just students.  California adopted a test for teachers in the 1980s, the California Basic Educational Skills Test (CBEST).  Beginning in 1982, even teachers already in the classroom had to pass it.   I made a nice after-school and summer income tutoring colleagues who didn’t know fractions from Fermat’s Last Theorem.  To be fair, primary teachers, teaching kindergarten or grades 1-2, would not teach fractions as part of their math curriculum and probably hadn’t worked with a fraction in decades.  So they are no different than non-literary types who think Hamlet is just a play about a young guy who can’t make up his mind, has a weird relationship with his mother, and winds up dying at the end.

Division is the most difficult operation to grasp for those arrested at the part-whole stage of understanding fractions.  A problem that Liping Ma posed to teachers is now legendary.^[3]

She asked small groups of American and Chinese elementary teachers to divide 1 ¾ by ½ and to create a word problem that illustrates the calculation.  All 72 Chinese teachers gave the correct answer and 65 developed an appropriate word problem.  Only nine of the 23 American teachers solved the problem correctly.  A single American teacher was able to devise an appropriate word problem.  Granted, the American sample was not selected to be representative of American teachers as a whole, but the stark findings of the exercise did not shock anyone who has worked closely with elementary teachers in the U.S.  They are often weak at math.  Many of the teachers in Ma’s study had vague ideas of an “invert and multiply” rule but lacked a conceptual understanding of why it worked.

A linguistic convention exacerbates the difficulty.  Students may cling to the mistaken notion that “dividing in half” means “dividing by one-half.”  It does not.  Dividing in half means dividing by two.  The number line can help clear up such confusion.  Consider a basic, whole-number division problem for which third graders will already know the answer:  8 divided by 2 equals 4.   It is evident that a segment 8 units in length (measured from 0 to 8) is divided by a segment 2 units in length (measured from 0 to 2) exactly 4 times.  Modeling 12 divided by 2 and other basic facts with 2 as a divisor will convince students that whole number division works quite well on a number line. 

Now consider the number ½ as a divisor.  It will become clear to students that 8 divided by ½ equals 16, and they can illustrate that fact on a number line by showing how a segment ½ units in length divides a segment 8 units in length exactly 16 times; it divides a segment 12 units in length 24 times; and so on.  Students will be relieved to discover that on a number line division with fractions works the same as division with whole numbers.

Now, let’s return to Liping Ma’s problem: 1 ¾ divided by ½.   This problem would not be presented in third grade, but it might be in fifth or sixth grades.  Students who have been working with fractions on a number line for two or three years will have little trouble solving it.  They will see that the problem simply asks them to divide a line segment of 1 3/4 units by a segment of ½ units.  The answer is 3 ½ .  Some students might estimate that the solution is between 3 and 4 because 1 ¾ lies between 1 ½ and 2, which on the number line are the points at which the ½ unit segment, laid end on end, falls exactly three and four times.  Other students will have learned about reciprocals and that multiplication and division are inverse operations.  They will immediately grasp that dividing by ½ is the same as multiplying by 2—and since 1 ¾ x 2 = 3 ½, that is the answer.  Creating a word problem involving string or rope or some other linearly measured object is also surely within their grasp.

I applaud the CCSS for introducing number lines and fractions in third grade.  I believe it will instill in children an important idea: fractions are numbers.  That foundational understanding will aid them as they work with more abstract representations of fractions in later grades.   Fractions are a monumental barrier for kids who struggle with math, so the significance of this contribution should not be underestimated.

I mentioned above that instruction and curriculum are often intertwined.  I began this series of posts by defining curriculum as the “stuff” of learning—the content of what is taught in school, especially as embodied in the materials used in instruction.  Instruction refers to the “how” of teaching—how teachers organize, present, and explain those materials.  It’s each teacher’s repertoire of instructional strategies and techniques that differentiates one teacher from another even as they teach the same content.  Choosing to use a number line to teach fractions is obviously an instructional decision, but it also involves curriculum.  The number line is mathematical content, not just a teaching tool.

Guiding third grade teachers towards using a number line does not guarantee effective instruction.  In fact, it is reasonable to expect variation in how teachers will implement the CCSS standards listed above.  A small body of research exists to guide practice. One of the best resources for teachers to consult is a practice guide published by the What Works Clearinghouse: Developing Effective Fractions Instruction for Kindergarten Through Eighth Grade (see full disclosure below).^[4]  The guide recommends the use of number lines as its second recommendation, but it also states that the evidence supporting the effectiveness of number lines in teaching fractions is inferred from studies involving whole numbers and decimals.  We need much more research on how and when number lines should be used in teaching fractions.

Professor Wu states the following, “The shift of emphasis from models of a fraction in the initial stage to an almost exclusive model of a fraction as a point on the number line can be done gradually and gracefully beginning somewhere in grade four. This shift is implicit in the Common Core Standards.”^[5]  I agree, but the shift is also subtle.  CCSS standards include the use of other representations—fraction strips, fraction bars, rectangles (which are excellent for showing multiplication of two fractions) and other graphical means of modeling fractions.  Some teachers will manage the shift to number lines adroitly—and others will not.  As a consequence, the quality of implementation will vary from classroom to classroom based on the instructional decisions that teachers make.  

The current post has focused on what I believe to be a positive aspect of CCSS based on the implementation of the standards through instruction.  Future posts in the series—covering the “bad” and the “ugly”—will describe aspects of instruction on which I am less optimistic.

       

Current drafts of the reauthorized Elementary and Secondary Education Act (ESEA) fall short of a commitment to use research to improve education. The bills—the “Student Success Act” in the House and the “Every Child Achieves Act” in the Senate—no doubt represent compromises and tradeoffs as any major legislation would. But who is arguing for less research and innovation in education?

Much of what is debated about No Child Left Behind is its accountability structure—annual tests, “annual yearly progress,” and the goal of moving every student to proficiency by 2014. But another important theme in NCLB was using “scientifically-based research.” Its steady drumbeat of “use research, use evidence, use scientific methods,” represented an embrace of education research and especially the practice of using causal methods to study program effectiveness. NCLB did not go as far as requiring research evidence as a basis for program funding, and in 2002 not much evidence would have met that standard. Related legislation that year created the Institute of Education Sciences, which followed through on the vision of building and using evidence to improve education.

Evidence from studies showed that some of what had been thought did not prove to be true. For example, after-school programs did not improve outcomes; using education software to support teaching did not raise test scores; teacher professional development did not raise test scores; voucher programs did not raise test scores; a range of programs to promote social and emotional learning had little effect on outcomes. It seems like a list of negatives, but evidence is useful one way or the other. And, as Tom Kane has argued, more than 80 percent of clinical trials fail to show effectiveness. Why would education be different? And some things that were not known became evident: for example, parents and students attending charter schools were more satisfied with the schools but the schools themselves ranged widely in their effectiveness; math textbooks can affect math skills differently, and NCLB itself raised test scores.

Research is included in the current ESEA drafts, but to no greater extent than it was for NCLB, and in some ways, it’s to a lesser extent. The House bill substitutes a new term, “evidence based,” for NCLB’s “scientifically based.” The substitution seems innocuous, but could prove problematic because the bill does not further define “evidence.” (NCLB defined “scientifically based” in Title IX.) If a state conducts a survey and finds that many students participating in a program think it is effective, is that evidence the program is effective? Under some definitions, yes. Under others, not so much. Opinions about effects are not the same as measures of effects.

Adding a definition of evidence will clarify what meets it and what does not. There is language in the House bill that says programs that receive funding under Title II to prepare new teachers need to “reflect evidence-based research, or in the absence of a strong research base, reflect effective strategies in the field, that provide evidence that the program or activity will improve student academic achievement.” So, what are “effective strategies in the field?” How is it determined that they improve academic achievement? Would it not be “evidence-based research” that shows the practices led to improvement?

The House and Senate bills also call for evaluations. The Senate bill calls for a national evaluation of its new literacy program, an evaluation of a program that serves students in foster care, and a demonstration program of innovative assessment systems that states can pilot. That demonstration is likely to be evaluated so it is mentioned here. The House bill calls for an evaluation of the charter school program and the magnet schools program, neither of which is new. It would be the third evaluation of magnet schools.

Of course, ESEA does not have to be directive about what should be studied. It can set aside money that can be used for studies, and allow their topics and focus to emerge elsewhere. Both bills include the key clause that funds research and evaluation. It states that the Secretary of Education can set aside to use for evaluation up to 0.5 percent of funds for all except the first title. Using 2014 appropriations, the set-aside amount is roughly $35 million. IES also receives funding to carry out the National Assessment of Education Progress, to support state development of their data bases, and for other purposes such as studies of special education. The $33 million is to support studies that relate to ESEA.

This is not a lot of money for research, for three reasons. One is that research is a uniquely federal responsibility, not just for education but generally. Fiscal federalism assigns the federal government responsibility for research because states and localities have incentives to underinvest in research. Its costs accrue to them and its benefits accrue to everybody. When the federal government invests in research, costs and benefits align.

A second perspective for viewing the education-research investment as paltry is to compare it with the federal investment in the National Institutes of Health. In 2014, that investment was about $30 billion. It’s hard to argue that there is “too much” investment in health research. It’s vital to the nation’s population. But it’s also hard to argue that it is hundreds of times more important to invest in health research than education research for America’s low-income students. Education is vital to the nation’s population too. There is much more federal spending on health than on education, through the Medicare and Medicaid programs, particularly, but that is not a basis for why federal spending for health research is so much larger than education research. The federal role in supporting research is primary regardless of which level of government spends on services.

A third perspective on research funding also points to enlarging it. On its own, the K-12 public education system is static. It wants to do the same thing. Taxes flow in, students and teachers come to school in the morning, there are classes and graduation ceremonies and sports events, and it is repeated next year. The system wants to be in equilibrium, and when it is pushed out of equilibrium, it wants to get back to it.

For example, in the past decade, how states and districts evaluate the performance of their teachers has seen rapid changes. Many teachers are now being evaluated partly based on how their students score on tests. But has anything really changed? The new systems replaced previous systems believed to rate teachers too positively. Nearly all teachers were “effective.” After putting new systems in place, Rhode Island reported that 98 percent of its teachers were effective; Florida, 97 percent, New York, 96 percent. The system returned to where it was.

The point is not about teacher evaluation per se. It is that research has the potential to put energy into the system. What is thought to be best practice for teaching reading, or math, or any subject, might change if research shows that new methods improve on current methods. Or a program of social and emotional development might prove effective in reducing student behavior problems. Or an approach to teaching English to non-English speakers might prove effective in promoting their language acquisition and academic achievement. The list is nearly endless.

Of course, research needs to be conducted and disseminated, and its timeframes can seem slow to policymakers. But here is a margin for bringing innovation to a system that does not have much incentive to innovate. It seems at least as useful to push on this margin as it is to study how education is delivered in Finland and Singapore, which happened after those two countries had the top scores on the Programme for International Student Assessment. Their systems might have some attractive features, but generalizing them to a vast country with a heterogeneous population and a highly decentralized education system is problematic. (Tom Loveless has warned of the perils of “edu-tourism.”) 

America is entering a new phase in which the majority of its public school students are from low-income households. That’s about 25 million students. Suppose that for each of these students, ESEA set aside $10 a year—a dollar for each month they are in school—for federal research to improve education. That’s $250 million. It sounds like a lot of money, but the scale of the K-12 enterprise is vast and seemingly large numbers can be misleading. Comparing it to the more than $600 billion we are spending each year for K-12 education, it’s four-tenths of a percent. 

The draft ESEA legislation will be modified as it moves to the floor of the respective chambers and then to conference. Increasing funds for research could be done two ways. One would be simply to have the set-aside apply to all spending under the bill. It then would include Title I, which is larger than all the other titles combined. For the president’s 2015 budget request, the change would increase the amount set aside for research to $90 million. Or the set aside itself could be increased, to, say, three percent. It’s not getting to the one-dollar-a-month set-aside, but it’s something.

In the 13 years since NCLB was passed, we’ve seen more clearly that research is essential to improve education, just as clinical trials are essential to improve health care. A commitment to equity lies at the heart of ESEA, and spending $10 a year on research for each of America’s low-income students will help meet that commitment.

       

Student loans make billions of dollars for U.S. taxpayers, at least on paper. These profits attract frequent criticism from politicians, most recently in a letter to the Education Department by six U.S. senators led by Elizabeth Warren, who has previously called the profits “obscene” and “morally wrong.”

Does the U.S. government really make billions of dollars off the backs of student borrowers? Current debates on this issue devolve into an argument about accounting methods that pits the method that government budget analysts are required to use by the Federal Credit Reform Act (FCRA) against an alternative method called “fair value.” It turns out that no accounting method can end government profits on student loans, but a change to the loan program itself could.

The FCRA accounting method says that federal loans make money for the government, while the fair-value method says they cost taxpayers money. In the most recent analysis by the Congressional Budget Office (CBO), FCRA shows a profit of $135 billion over 10 years, whereas fair-value shows a cost of $88 billion.^[1] Put another way, FCRA shows a profit margin of 12 percent, whereas fair-value shows a subsidy rate of eight percent. (Unfortunately many estimates, including these, ignore administrative costs, which the CBO estimates at $35 billion over 10 years.)

The debate over which method is better comes down to whether the government should factor into its cost estimates “market risk,” which is essentially the risk that its budget projections will be wrong.^[2] Those projections could turn out to be wrong for many reasons, such as a weaker than expected economy several years from now (keep in mind that student loans are typically repaid over 10 or more years). Even over a short period of time, budget predictions can swing wildly, with the CBO’s estimate of student loan profits over 10 years (using the FCRA method) falling from $110.7 billion in April 2014 to $47.2 billion in March 2015, less than a year later.^[3] According to the CBO, this decrease in expected gains resulted from increases in expected loan defaults, administrative costs, and participation in income-based repayment programs.

Fair-value proponents argue that the government should calculate the cost of this risk to taxpayers and factor it into budget projections, just as lenders do in the private sector. These proponents specifically point to what Donald Marron of the Urban Institute calls FCRA’s “magic-money-machine problem,” in that it lets the government record a profit in today’s budget based on returns (e.g., interest payments) that are expected over a long period of time. It doesn’t make sense for the government to make a risky long-term bet and then spend the expected winnings today, but that’s exactly what FCRA allows it to do.

Fair-value critics argue that accounting for risk is unnecessary and will exaggerate the cost of federal lending programs. This is akin to what Marron calls fair-value’s “missing-money problem,” in that it ignores the fact that the government expects to make money on some risky endeavors such as making loans to college students. In Marron’s words, “FCRA counts the government’s fiscal chickens before they hatch, and fair value assumes they never hatch.”^[4]

The risk inherent in any lending program is real, regardless of whether it is accounted for in the budgeting process. Who should bear that risk raises questions of fairness. Policymakers are objecting today to forecasted profits on student loans. But if too many students fail to repay, future policymakers may object to taxpayers footing the bill for delinquent borrowers. Because it is impossible to predict the future, it is impossible to set interest rates (and other borrowing terms) today that will ensure no profit is made, or loss incurred, on the loans.

This is true under any accounting rule. A loan program that breaks even under fair-value is often going to end up making a profit for taxpayers, but it could still produce a loss. Conversely, a loan program estimated to break even under FCRA is more likely to leave taxpayers holding the bag if more borrowers failed to repay their debts than expected, but could also still produce profits.

The solution to this conundrum is to shift most of the market risk onto borrowers as a whole, while continuing to protect individual borrowers through income-based repayment. If borrowers bear the risk of higher or lower overall repayment rates, then whether the government accounts for that risk or not becomes a moot point. By definition, the loan program breaks even for taxpayers.

This can be accomplished by reforming the federal student lending program to include a guarantee fund. Here’s how it would work: borrowers pay a fee when they take out a loan that goes into a trust fund used to cover the unpaid debts of borrowers who end up failing to repay.^[5] At the end of the repayment period, any money remaining in the guarantee fund for that cohort of borrowers is returned, with interest, to the borrowers who repaid successfully.

For example, the government currently expects defaults equivalent to about 0.6 percent of loans made. By charging a fee of 2.4 percent, it would protect taxpayers from defaults up to four times what is expected. Under this system, the government never profits off of student loans, and only faces a loss if repayment rates are so unexpectedly low as to exhaust the guarantee fund.

In order to zero out government profits, interest rates would be significantly lower under this system.^[6] The government currently draws much of its “profits” from the difference between student loan interest rates and its (lower) cost of borrowing. For example, each year the interest rate on loans for undergraduates is set at about two percentage points above the Treasury rate on 10-year loans. With a guarantee fund protecting taxpayers from defaults, students could pay an interest rate equal to the government’s cost of borrowing corresponding to the length of their loans. Current Treasury rates are 1.9 percent for a 10-year loan and 2.4 percent for a 20-year loan, both significantly less than the 4.7 percent undergraduates pay.^[7]

A guarantee fund for student loans is not a new idea. In the 1920s, a philanthropic foundation launched a “trial of making loans on business terms to college students, with character and group responsibility as the basis of credit.”^[8] The “group responsibility” component was a guarantee fund which the foundation used to ensure that the money it committed to student loans “is protected by the borrowers themselves at actual cost.”^[9] The foundation noted that this was akin to an insurance program in which “the excess cost of losses is borne by the members of the group in the form of reduced earnings on their premiums.”

This interesting early experiment made an average of $1 million in loans per year (in today’s dollars). The current federal loan program, which makes over $100 billion in loans per year to any college student who asks for one, is far larger and more complicated. Incorporating a guarantee fund would require a number of policy decisions, such as the size of the fee needed and how to distribute refunds given that different borrowers repay over different lengths of time. This policy feature might also entail increased administrative costs.

But a guarantee fund would also have some advantages beyond protecting students from government profits and taxpayers from losing money on bad loans. The current system is a mish-mash of cross-subsidies of different groups of borrowers. For example, both the FCRA and fair-value accounting methods estimate that profits made on loans to graduate students help cover the losses made on loans to some undergraduates. The guarantee fund could be segmented into different pools of borrowers, with higher-risk pools covering their own costs, or policymakers could make an explicit decision to keep these cross-subsidies.

The political benefits of a guarantee fund could end up being more important than its policy benefits. First, it would stop today’s politicians from fanning the flames of a student loan crisis by pointing to government profits on student loans, and tomorrow’s politicians from attacking the program when it starts producing losses.^[10] Second, it would let politicians take credit for reducing interest rates on student loans while not putting taxpayer money at risk. And finally, this policy could help nudge the prevailing narrative around student lending from a “students vs. the government” to a “we’re in this together” mentality.

A guarantee fund would not solve all of the problems facing the federal student lending system, much less higher education more broadly. But in combination with other reforms, such as simplification of loan receipt and repayment, it could boost the long-term economic and political sustainability of a policy tool that has opened the university gates to millions of students.

Note: I thank Donald Marron for helpful comments on an earlier draft of this post. Any remaining errors are my own.

       

Since 2009, more than 40 states have rewritten their teacher evaluation policies. Given that school systems have neglected to manage classroom instruction for decades, it was inevitable that many schools would struggle to implement them. New York Governor Andrew Cuomo re-ignited the controversy by including a second round of teacher evaluation reforms in his budget this year.  Below, I describe the most promising opportunities in the new law.  Hopefully, New York will provide a blueprint for other states as they tweak their own systems in the coming years.

Traditionally, principals have used much too low a standard when granting tenure, viewing the probationary period merely as an opportunity to weed out the worst malpractice.  Under the new law in New York, the length of the probationary period will be lengthened from three to four years and no teacher rated “ineffective” in their fourth year would be able to earn tenure.

Therefore, much depends on what it means to be designated “ineffective.” As New York learned last year when 96 percent of teachers were rated “effective” or “highly effective”, a vague standard is equivalent to no standard.  The department should specify that a probationary teacher is “ineffective” during their fourth year of teaching if:  (i) a teacher’s average student achievement gain during their second through fourth year of teaching falls below that of the average first-year teacher in their district or (ii) the classroom observations done by external observers during their second through fourth year of teaching falls below that of the average first-year teacher.^[1]  

Most teachers improve their practice during their initial years of teaching.  However, if, by their fourth year of teaching, a probationary teacher has not moved beyond the performance of the average novice in their district in terms of student achievement growth and measured classroom practice, students would be better off on average if the district were to commit to fill that teacher’s assignment with a novice teacher every year instead.  A fourth year probationary teacher who has been no more effective than a novice teacher should not receive the long-term commitment which accompanies tenure. 

Such a standard would have a number of advantages:  First, it reminds principals that a promotion decision involves a choice (albeit usually implicit) between two teachers—the probationary teacher and an anonymous novice.  Would an NFL coach forego 25 years of future draft picks in order to sign a mediocre player to a long-term contract?  No. Yet principals in New York and elsewhere have done so every spring.   Linking the standard for tenure to the effectiveness of the average first-year teacher would remind everyone of the opportunity cost involved in every tenure decision.

Second, it would be a self-adjusting standard: if classroom observation scores become inflated or if the quality of those willing to enter teaching were to decline (or rise), the threshold for tenure would adjust accordingly.

Third, by relying on the scores given by external observers, the tenure decision would no longer be at the sole discretion of the local principal.  Because a tenure decision involves thousands of future students as well as future colleagues and supervisors at other schools in a district where a teacher might work, it makes no sense to leave the decision in the hands of their current supervisor alone.

If tenure protections were reserved only for accomplished teachers, just imagine how different our schools would be.

Rather than focus solely on a teacher’s performance during the most recent academic year, the teacher evaluation system should allow tenured teachers to accumulate a longer-term track record of excellence.^[2] 

After the tenure decision, a teacher’s evaluation each year should depend on four parts: 40 percent of the weight should be placed on student achievement gains in all available prior school years, 40 percent should be placed on prior classroom observations and the remaining 20 percent should be split between their student achievement gains and classroom observations in the most recent year. 

As in many professions (including higher education), a past history of success signals that a teacher has the talent and accumulated skill to be successful in the future.  The only reason to place greater than proportional weight on the most recent performance is to preserve teachers’ incentive to maintain effort, and not simply to rest on their laurels.  Only in professions such as sales, where it is more important to incentivize current effort than to retain talent, is it necessary to ask, “What have you done for us lately?” Therefore, it does not make sense to limit evaluations to the current (or most recent) year.

Aside from recognizing the importance of talent and accumulated skill, another advantage of a longer term perspective is that it frees up teachers with a strong track record to separate their own interests from those of their weakest colleagues.  Reform advocates mistakenly believe that the vast majority of teachers have nothing to fear from efforts to root out “grossly ineffective” teachers.  They say, “Only the weakest one or two percent of teachers have anything to fear from new teacher evaluations.”  However, they forget that the absence of any meaningful differentiation in the past has meant that many teachers do not know where they stand.  When a majority of teachers think they could be in the bottom two percent under an unfamiliar and unspecified system, they will resist change.  However, as teachers develop a track record and become less vulnerable to a single bad year, they will be more supportive of efforts to police their own ranks. 

Use Technology to Reinvent the Classroom Observation  

Children will not succeed until all teachers — both tenured and untenured — adjust what and how they teach.  Therefore, a successful teacher evaluation system must also support adult behavior change, and we must not underestimate how difficult that will be.

No one would launch a Weight Watchers club without any bathroom scales or mirrors. Student achievement gains are the bathroom scale, but classroom observations must be the mirror.

Under the new law in New York, one of a teacher’s observers must be drawn from outside a teacher’s school — someone with no personal axe to grind, whose only role is to comment on teaching.  A few other districts—such as Washington, DC and Hillsborough County Florida—have been incorporating outside observers in recent years. However, New York is the first state to require outside observers.

No school community can change the way they teach without starting an honest conversation about their own instruction. When 96% of teachers are rated effective or better despite high student failure rates, it is a sure sign that principals have not been honest. An external perspective will make it easier for longtime colleagues to have a frank conversation about each other’s instruction.

Yet, as valuable as they might be, external observations will also present significant logistical challenges. A lot of time could be wasted as observers drive from school to school. One alternative would be to allow teachers to submit videos to an external observer in lieu of in-person classroom observations.  (For similar practical reasons, the National Board for Professional Teaching Standards has been allowing teachers to submit videos for more than 20 years.)

Doing so would have a number of advantages. For instance, teachers usually struggle because of the clues they are not noticing, or because they lose track of time. It is difficult for such teachers to recognize their mistakes by reading an observer’s written notes after class. In fact, it’s biologically impossible for someone to recall cues they did not notice in the moment.

Giving teachers control of a camera, the opportunity to watch themselves teach, and allowing them to discuss their videos talk with external observers, peers and supervisors will provide be a more effective mirror than any observer’s written notes.

There would be other advantages as well. Harried principals could do their observations during quieter times of the day or week. And when principals do not have sufficient content expertise, they could solicit the views of content experts.

Finally, video evidence would level the playing field if a teacher ever has to defend their teaching against a principal’s written notes at a dismissal hearing—a teacher’s video vs. an observer’s written notes. Video is now widely used to coach improvements in activities such as athletics and dance and public speaking.   The state department of education should encourage districts to use technology to meet the external observer requirement.

Conclusion

New York has not been the only state to struggle with the implementation of teacher evaluation systems.  Many systems are still failing to set a high standard for teaching. Despite the controversy, let’s hope that Andrew Cuomo is not the only governor with the courage to revisit the issue.   Students will not achieve at higher levels until teachers teach at higher levels—and that’s simply not going to happen without quality feedback and evaluation.

References:

Thomas J. Kane and Douglas O. Staiger “Improving School Accountability Systems” Working Paper, May 2002 http://www.dartmouth.edu/~dstaiger/Papers/WP/2002/KaneStaiger_2002.pdf

Thomas J. Kane and Douglas O. Staiger “The Promise and Pitfalls of Using Imprecise School Accountability Measures” Journal of Economic Perspectives (Fall, 2002b), Vol. 16, No. 4, pp. 91-114.

Gibbons, Robert and Kevin Murphy “Optimal Incentive Contracts in the Presence of Career Concerns: Theory and Evidence” Journal of Political Economy Vol. 100, No. 3 (Jun 1992): 468-505.

       

Over the past year, the Common Core State Standards have risen from a topic of interest mainly to educators and school reformers to a top-tier issue in national politics. With likely Republican presidential candidates already staking out divergent positions, the standards and the federal government’s role in promoting them show strong signs of emerging as a key point of contention in the Republican primaries. How might this growing salience shape public opinion on the standards? A comparison of polls we’ve conducted nationally and in the state of Louisiana is instructive—and discouraging.

Several states have endured political battles over the Common Core State Standards, but none has matched the drama and intensity of Louisiana’s. Like most states, Louisiana adopted Common Core in 2010 without fanfare or controversy. The state began introducing the standards the following year with an eye toward full implementation this school year. At the time, the standards had the full-throated support of Republican Governor Bobby Jindal. Since then, Governor Jindal has positioned himself against the Common Core and his former ally, State Superintendent John White. In last year’s legislative session, Governor Jindal pushed bills to pull Louisiana out of the Common Core. When those efforts failed, largely due to the state’s powerful business lobby, the governor issued an executive order to pull the state out of a consortium of states using a Common Core-aligned test. After Superintendent White balked, arguing the governor lacked the legal authority to withdraw the state, Jindal joined a lawsuit against the state board of education (which has since been dismissed). The fray has been marked with recriminations and personal attacks befitting the political traditions of the Pelican State. Although a state court recently dismissed his lawsuit, Governor Jindal has announced plans to go after Common Core again in this year’s session.

Louisiana’s flare-up over Common Core is set upon a broader, national kindling of rising public discontent.  Last summer’s Education Next Poll of a representative sample of American adults showed that although a majority of the public supported the standards, that support had seriously eroded. In 2013, 65 percent of the general public favored the standards, but that portion fell to 53 percent in 2014 (see Figure 1).

Figure 1. Percent Supporting the Common Core State Standards, Overall and by Party, 2013 and 2014

Note. The question read as follows: As you may know, in the last few years states have been deciding whether or not to use the Common Core, which are standards for reading and math that are the same across states. In the states that have these standards, they will be used to hold public schools accountable for their performance. Do you support or oppose the use of the Common Core in your state?

The Education Next poll revealed partisan polarization and widespread misperception, as well.  In the 2013 Education Next Poll, Common Core gathered backers from across the political spectrum. By 2014, support among Republicans fell from 57 percent to 43 percent, even as support among Democrats remained nearly unchanged (64 percent in 2013 and 63 percent in 2014). That year, the majority of respondents were misinformed on several important elements of the Common Core State Standards Initiative. Barely more than one third said it was false that the federal government requires all states to use the Common Core standards (it does not), just 15 percent said it was false that the federal government will receive detailed data on the test performance of individual students in participating states (it will not), and fewer than half said it was true that states and local school districts can decide which textbooks to use under Common Core (they can). 

How does public opinion and understanding change when Common Core becomes a highly visible focal point—as has been the case in Louisiana? The extra attention given to the issue by political leaders and the press does not appear to resolve debates into consensus or clear up public confusion. If anything, it is quite the opposite. 

To examine how the heated rhetoric now attached to Common Core influences opinion, the 2015 Louisiana Survey, an annual survey of the state’s adult residents sponsored by Louisiana State University’s Reilly Center for Media and Public Affairs, used an experiment featuring two versions of a question assessing support for common educational standards.

• One version of the question refers to the program by name: “As you may know, in the last few years states have been deciding whether or not to use the Common Core, which are standards for reading and math that are the same across the states. In the states that have these standards, they will be used to hold public schools accountable for their performance. Do you support or oppose the use of the Common Core in Louisiana?”

• The alternate version drops the program’s name: “As you may know, in the last few years states have been deciding whether or not to use standards for reading and math that are the same across the states.  In the states that have these standards, they will be used to hold public schools accountable for their performance.   Do you favor or oppose the use of these standards in Louisiana?”

When the Common Core label is dropped from the question, support for the concept leaps from 39 percent to 67 percent. With the label, a majority of the public (51 percent) opposes Common Core, but without the label a majority (67 percent) supports common math and reading standards (see Figure 2).

Figure 2. Percent Supporting the “Common Core” and “standards that are the same,” Louisiana (2015) and National Public (2014)

The “Common Core” label also evokes significant partisan polarization in Louisiana. There is no difference in support of Democrats and Republicans when the program is not labeled as “Common Core,” 72 percent and 71 percent respectively. However, when the phrase is used, a majority of Democrats (57 percent) support while a majority of Republicans (62 percent) oppose it. Only 22 percent of Republicans support the standards with the label.

This labeling effect occurs despite the fact that people in Louisiana claim to be more informed about Common Core. In 2014 nearly half of the public (49 percent) described themselves as “very familiar” or “somewhat familiar” with Common Core. This year, 62 percent of Louisiana residents say they are “very familiar” or “somewhat familiar.” (By way of comparison, just 43 percent of Americans surveyed by Education Next in 2014 had even heard of Common Core; among those who had, a majority said they were “not very knowledgeable about it.”)

An examination of Louisianans’ perceptions about Common Core reveals continued confusion. For example, some respondents were asked whether the following statement is true or false: “Under the Common Core, the state of Louisiana and its local school districts decide which textbooks and educational materials to use in their schools.” One fifth said this statement is false, and 43 percent said it is true. However, another set of respondents were asked whether the opposite statement is true or false: “Under the Common Core, the federal government decides which textbooks and educational materials to use in schools.” If 43 percent believe it is true that states and local districts retain this authority, then it is reasonable to expect that at least as many would say the statement about federal authority is false. However, that was not the case. Just 12 percent said the second statement is false. Indeed, 59 percent said that it is true that the federal government makes these decisions. When differences in how a question is phrased produce significant inconsistencies—as they do here—that suggests many responses may be based on little more than guesswork. 

The toxic effect of the “Common Core” name is not a Louisiana phenomenon. The survey experiment in the Louisiana Survey is based upon a similar experiment in the 2014 Education Next Poll. When the “Common Core” label was dropped from the question in the national survey, support for the concept among the general public rose from 53 percent to 68 percent (see Figure 2). Additionally, the pronounced partisan polarization evoked by the label “Common Core” disappeared when the question did not include those seemingly toxic words. In other words, a broad consensus remains with respect to common standards, despite the fact that public debate over Common Core polarizes the public.

Could the choice of the name “Common Core” specifically explain these patterns? Conservative columnist Peggy Noonan wrote that it “sounds common—except for the part that sounds soviet.” It is possible that Republicans share her sensitivities. But we suspect that something deeper is at work. A similar dynamic was evident in public opinion on No Child Left Behind in the waning years of the George W. Bush administration. Support for the law dropped markedly in questions that referred to it by name, as compared to questions that used otherwise identical language to describe its key provisions. These effects were observed mainly among Democrats, suggesting that the law, despite its bipartisan roots, had come to be closely associated in the public mind with the Republican president. The consistency of patterns highlights a key tension facing education advocates seeking to use federal policy to advance their goals: Any benefits from federal involvement may come at the cost of heightened partisan polarization.

The Education Next and Louisiana Survey polls differ in timing, target population, and mode. Nevertheless, the pairing is suggestive. In Louisiana, where the fight over Common Core has been particularly salient, the effect of the “Common Core” label was even more negative than in the American public as whole, and the impact on polarization was greater. As the conflict heats up nationwide, the American public may move further from a latent consensus about common standards toward ever more confused divisions over “Common Core.”

       

A nasty political fight between New York Governor Andrew Cuomo and the state teachers’ union has embroiled the state since Cuomo announced a set of education policy proposals in January that led the state’s union president to declare that Cuomo had “declared war on the public schools.” In the end, Cuomo got much (but not all) of what he wanted, including changes to teacher evaluation and tenure policies, which the State Senate and Assembly approved last month.

The state teachers’ union has not taken defeat lying down, responding in part by encouraging parents to “opt-out” their children from the standardized exams that are used as a factor in the evaluation system. Union president Karen Magee argued that a large number of opt-outs could sabotage the evaluation system: “Statistically, if you take out enough, it has no merit or value whatsoever.”

Is Magee’s statistical argument correct? Taken to enough of an extreme, it surely is. If all students opted out, there would be no test data to include in teacher evaluations. But what would be the likely impact of some, or even many, but not all students refusing to take the tests? My colleague Katharine Lindquist and I used statewide data from North Carolina to simulate the impact of opt-out on test-score-based measures of teacher performance.^[1] We ran two sets of simulations: one where students opt-out randomly, and another in which opt-out occurs among the highest-performing students in each classroom (as measured by their prior test scores).

Opting out adds noise to the data, which increases the amount of variability in the teacher performance measures because each teacher’s score is based on fewer students. A teacher faces a higher risk of being labelled low-performing (or high-performing) as the number of opt-outs in her classroom increases. But the effect of opt-out is quite small unless a large number of students do so.

A teacher in New York State is considered to be ineffective based on her students’ test score growth if her value-added score is more than 1.5 standard deviations below average (i.e., in the bottom seven percent of teachers). If a handful of students opt out, little changes. The risk of getting the lowest score barely changes even if five students in the class opt out—more than 20 percent of the typical classroom.

But if enough of a teacher’s students opt out, her risk of getting a bad score increases.^[2] For example, imagine a teacher who strongly encourages her students to opt out of the tests and succeeds in getting 15 students—a majority of the class—to opt out. That teacher would have a significantly elevated risk of getting an ineffective score: 11-13 percent depending on the simulation.

Percent of Teachers Rated Ineffective, by Number of Students Opting Out Per Classroom

A large number of opt-outs in a classroom also increases the teacher’s chance of getting a high score, as shown in the simulated ratings using the New York scoring system in the figure below.^[3] However, in New York, the punishments for a low score are more significant than the reward for a high score. New teachers must now receive good scores in three out of their first four years in order to be eligible for tenure, and teachers with tenure can now be terminated after two years of low scores.

Distribution of Teacher Ratings, with and without 15 opt-outs

Reducing the number of students who contribute to a teacher’s value-added score not only changes the chance that a teacher will receive a particular rating; it also increases the likelihood that she will receive the wrong rating. One way to assess the potential impact on the fairness of the resulting teacher ratings is to calculate the correlation between teachers’ value-added scores with and without opt-out. If only one student in the class opts out, value-added scores barely change at all—the correlation is 0.99 (on a scale from 0 to 1). With five and ten opt-outs per class, the correlation remains high—0.97 and 0.91, respectively. The correlation eventually starts to break down if a large number of students opt out—it is 0.77 if 15 students do so—indicating a measurably less fair evaluation system than if all students take the standardized tests.^[4]

Governor Cuomo has sharply criticized the evaluation system as “baloney” for classifying 99 percent of teachers as effective. The strange irony is that teachers who convince many of their students to opt-out are likely to help achieve Cuomo’s goal of increasing the share of teachers judged to be low-performing. These teachers would also receive evaluation scores that are less fair than the ones produced without opt-out in a system their union already believes is unfair.

But in the majority of classrooms, where opt-out appears likely to remain at low levels, the data strongly suggest that students sitting out of standardized testing will have only a trivial impact on the ratings received by their teachers. The broader lessons is that while opt-out may have some success as a political strategy, it is unlikely to have much of a direct, broad-based impact on the teacher evaluation system in New York or any other state.

      

This week marks the release of the 2015 Brown Center Report on American Education, the fourteenth issue of the series.  One of the three studies in the report, “Girls, Boys, and Reading,” examines the gender gap in reading.  Girls consistently outscore boys on reading assessments.  They have for a long time.  A 1942 study in Iowa discovered that girls were superior to boys on tests of reading comprehension, vocabulary, and basic language skills.^[i]  Girls have outscored boys on the National Assessment of Educational Progress (NAEP) reading assessments since the first NAEP was administered in 1971. 

I hope you’ll read the full study—and the other studies in the report—but allow me to summarize the main findings of the gender gap study here.

Eight assessments generate valid estimates of U.S. national reading performance: the Main NAEP, given at three grades (fourth, eighth, and 12^th grades); the NAEP Long Term Trend (NAEP-LTT), given at three ages (ages nine, 13, and 17); the Progress in International Reading Literacy Study (PIRLS), an international assessment given at fourth grade; and the Program for International Student Assessment (PISA), an international assessment given to 15-year-olds.  Females outscore males on the most recent administration of all eight tests.  And the gaps are statistically significant.  Expressed in standard deviation units, they range from 0.13 on the NAEP-LTT at age nine to 0.34 on the PISA at age 15.

The gaps are shrinking.  At age nine, the gap on the NAEP-LTT declined from 13 scale score points in 1971 to five points in 2012.  During the same time period, the gap at age 13 shrank from 11 points to eight points, and at age 17, from 12 points to eight points.  Only the decline at age nine is statistically significant, but at ages 13 and 17, declines since the gaps peaked in the 1990s are also statistically significant.  At all three ages, gaps are shrinking because of males making larger gains on NAEP than females.  In 2012, seventeen-year-old females scored the same on the NAEP reading test as they did in 1971.  Otherwise, males and females of all ages registered gains on the NAEP reading test from 1971-2012, with males’ gains outpacing those of females.

The gap is worldwide.  On the 2012 PISA, 15-year-old females outperformed males in all sixty-five participating countries.  Surprisingly, Finland, a nation known for both equity and excellence because of its performance on PISA, evidenced the widest gap.  Girls scored 556 and boys scored 494, producing an astonishing gap of 62 points (about 0.66 standard deviations—or more than one and a half years of schooling).   Finland also had one of the world’s largest gender gaps on the 2000 PISA, and since then it has widened.  Both girls’ and boys’ reading scores declined, but boys’ declined more (26 points vs. 16 points).  To put the 2012 scores in perspective, consider that the OECD average on the reading test is 496.  Finland’s strong showing on PISA is completely dependent on the superior performance of its young women.

The gap seems to disappear by adulthood.  Tests of adult reading ability show no U.S. gender gap in reading by 25 years of age.  Scores even tilt toward men in later years. 

The words “seems to disappear” are used on purpose.  One must be careful with cross-sectional data not to assume that differences across age groups indicate an age-based trend.  A recent Gallup poll, for example, asked several different age groups how optimistic they were about finding jobs as adults.  Optimism fell from 68% in grade five to 48% in grade 12.  The authors concluded that “optimism about future job pursuits declines over time.”  The data do not support that conclusion.  The data were collected at a single point in time and cannot speak to what optimism may have been before or after that point.  Perhaps today’s 12^th graders were even more pessimistic several years ago when they were in fifth grade.  Perhaps the 12^th-graders are old enough to remember when unemployment spiked during the Great Recession and the fifth-graders are not.   Perhaps 12^th-graders are simply savvier about job prospects and the pitfalls of seeking employment, topics on which fifth-graders are basically clueless.

At least with the data cited above we can track measures of the same cohorts’ gender gap in reading over time.  By analyzing multiple cross-sections—data collected at several different points in time—we can look at real change.  Those cohorts of nine-year-olds in the 1970s, 1980s, and 1990s, are—respectively—today in their 50s, 40s, and 30s.  Girls were better readers than boys when these cohorts were children, but as grown ups, women are not appreciably better readers than men.

Care must be taken nevertheless in drawing firm conclusions.  There exists what are known as cohort effects that can bias measurements.  I mentioned the Great Recession.   Experiencing great historical cataclysms, especially war or economic chaos, may bias a particular cohort’s responses to survey questions or even its performance on tests.  American generations who experienced the Great Depression, World War II, and the Vietnam War—and more recently, the digital revolution, the Great Recession, and the Iraq War—lived through events that uniquely shape their outlook on many aspects of life. 

The gender gap is large, worldwide, and persistent through the K-12 years. What should be done about it?  Maybe nothing.  As just noted, the gap seems to dissipate by adulthood.  Moreover, crafting an effective remedy for the gender gap is made more difficult because we don’t definitely know its cause. Enjoyment of reading is a good example.  Many commentators argue that schools should make a concerted effort to get boys to enjoy reading more.  Enjoyment of reading is statistically correlated with reading performance, and the hope is that making reading more enjoyable would get boys to read more, thereby raising reading skills.

It makes sense, but I’m skeptical.  The fact that better readers enjoy reading more than poor readers—and that the relationship stands up even after boatloads of covariates are poured into a regression equation—is unpersuasive evidence of causality.  As I stated earlier, PISA produces data collected at a single point in time.  It isn’t designed to test causal theories.  Reverse causality is a profound problem.  Getting kids to enjoy reading more may in fact boost reading ability.  But the causal relationship might be flowing in the opposite direction, with enhanced skill leading to enjoyment.   The correlation could simply be indicating that people enjoy activities that they’re good at—a relationship that probably exists in sports, music, and many human endeavors, including reading.

A key question for policymakers is whether boosting boys’ enjoyment of reading would help make boys better readers.  I investigate by analyzing national changes in PISA reading scores from 2000, when the test was first given, to 2102.  PISA creates an Index of Reading Enjoyment based on several responses to a student questionnaire.  Enjoyment of reading has increased among males in some countries and decreased in others.  Is there any relationship between changes in boys’ enjoyment and changes in PISA reading scores? 

There is not.  The correlation coefficient for the two phenomena is -0.01.  Nations such as Germany raised boys’ enjoyment of reading and increased their reading scores by about 10 points on the PISA scale.  France, on the other hand, also raised boys’ enjoyment of reading, but French males’ reading scores declined by 15 points.  Ireland increased how much boys enjoy reading by a little bit but the boys’ scores fell a whopping 37 points. Poland’s males actually enjoyed reading less in 2012 than in 2000, but their scores went up more than 14 points.  No relationship.

How should policymakers proceed?  Large, cross-sectional assessments are good for measuring academic performance at one point in time.  They are useful for generating hypotheses based on observed relationships, but they are not designed to confirm or reject causality.  To do that, randomized control trials should be conducted of programs purporting to boost reading enjoyment.  Also, consider that it ultimately may not matter whether enjoying reading leads to more proficient readers.  Enjoyment of reading may be an end worthy of attainment irrespective of its relationship to achievement.  In that case, RCTs should carefully evaluate the impact of interventions on both enjoyment of reading and reading achievement, whether the two are related or not.  

      

As Tom Kane recently wrote in this forum, we should be investing more in research on education if we want to see more progress in education. We want education to embrace positive findings emerging from research. This logic also implies that education should move away from negative or null findings, when evidence shows that a program or policy does not work. But the history of federal afterschool programs suggests that a program that was funded on its potential can continue to be funded based on a kind of wishful thinking in which evidence is viewed through rose-colored glasses. How much evidence or what kind of evidence is needed to offset wishful thinking is a question worth considering.

Afterschool programs, or out-of-school time programs, burst into view in the late 1990s. The federal government—flush with budget surpluses of hundreds of billions—began spending more on the 21^st Century Community Learning Centers (CCLC) program. The program was created by the 1994 Improving America’s Schools Act and had languished as an obscure provision to promote schools as community resources. Initially, the program received no appropriation, until Congress appropriated $40 million for it in 1998. 

Spending exploded after the program pivoted to support afterschool programs. By 2002, the program’s appropriation was $1 billion. For a federal program to grow from $40 million to $1 billion in a few years happens rarely. The agency overseeing the program, the U.S. Department of Education, partnered with the Charles Stewart Mott Foundation to underwrite conferences and technical assistance for program providers, pumping millions more into the program.

In 1999, the Department of Education contracted with Mathematica Policy Research to evaluate the 21^st Century program. The evaluation had elements that were both rigorous and representative. The elementary school part of the study was designed as an experiment; the middle school part was designed as a random sampling of programs around the country, with students participating in the program matched with students in neighboring schools (or the same school, in rural areas) that were not participating in the program. The evaluation collected data on a wide range of outcomes including grades, test scores, attendance, and behavior.^[1]

Ultimately, the evaluation reported on how the program affected outcomes. In a series of reports released between 2003 and 2005 (here, here, and here), the answers emerged: the program didn’t affect student outcomes. Except for student behavior, which got worse. And small samples were not an issue explaining why findings were insignificant. The national evaluation included about 2,300 elementary school students and 4,400 middle school students. The results were insignificant because the estimates of program effects hovered around zero.

In the face of these results, one course of action would have been to at least reduce program spending, if not eliminate the program altogether. The Bush administration proposed a reduction of $400 million in the program budget, advocates rallied to the cause, Arnold Schwarzenegger got involved, and ultimately Congress left program spending unchanged. To this day, the program spends more than a billion dollars each year.

If the national evaluation was thought to be unreliable or errant, a sensible next step would be to do another, possibly with different focuses or features. That hasn’t happened. Or perhaps the evaluation findings were dismissed because other research has shown that afterschool programs are effective. It hasn’t. Echoing a previous 2006 review by Zief, Lauver, and Maynard, a 2015 review of dozens of studies that were published up to 2014 concluded that "mean effects were small and non-significant for attendance and externalizing behaviors."^[2] (This is how researchers say the evidence shows that after school programs do not improve attendance or behavior.)

Two other pieces of evidence add to this picture. First, the U.S. Department of Education continues to collect and summarize the program’s annual performance reports (each state reports on its programs to the Department). Its most recent summary noted that ‘nearly all of the performance targets for the 2009-2010 reporting period were not reached.’ Second, a recent federal study of supplemental services programs found no effects on academic outcomes. The study examined programs that are required to be offered by schools that do not meet target levels of adequate yearly progress under No Child Left Behind. They are tutoring and academic support service programs offered outside the regular school day that have a stronger academic focus than the 21^st Century programs (which can offer snacks, recreation, and youth development activities), and yet they still did not improve academic outcomes.  

Other studies reach conclusions that are more positive. They do so by reporting findings such as this one: “A statewide evaluation of South Carolina’s 21^st CCLC programs found that 79 percent of students believed that the program had improved their academic skills.” Asking students what they think happened to them hardly is a scientific basis for measuring program effects. There’s a good reason why a new drug for, say, reducing blood pressure, would not be approved simply because patients reported that they felt their blood pressure was lower. Studies need to undertake objective measurements of real outcomes.

One afterschool program that did affect academic outcomes perhaps shows why effects are unlikely. The Higher Achievement afterschool and summer academic program in Washington, D.C. recruits students carefully, provides them with hundreds of hours of programming during school years and summers, and spends $4,500 per student each year (21^st Century programs spend about $600 per student each year). A rigorous evaluation of the program reported that the program improved math skills, which is good news. It did not improve reading comprehension, despite the program’s explicit emphasis on reading comprehension. And once again, behavior was negatively affected. Even when all the conditions seem right, success is mixed.

To date, more than $12 billion of federal tax money has been spent on a program that a preponderance of evidence indicates doesn’t help students. There are other beneficiaries of afterschool programs, however. Working parents may need inexpensive childcare, and having their children stay longer at school keeps them in a trusted setting. The national evaluation’s finding that students attended only a couple days a week on average is consistent with parents viewing programs as childcare.

But there is already a federal childcare program, much larger than the 21^st Century program. The $5 billion ‘Child Care and Development Block Grant’ program gives money to states to support childcare for low-income working families. If the basis of the 21^st Century program is to provide childcare, folding its resources into the childcare program seems appropriate.

Recently, the White House called for agencies to conduct more evaluations and embrace evidence to support their budget requests. Yet the 2016 budget request for the 21^st Century Community Learning Centers program is the same as past years: $1.16 billion. Perhaps this is simply program inertia, like a cruise ship that keeps moving forward after its engines are cut. It’s not indicative of an evidence-focused process.

The current House and the draft Senate bills reauthorizing ESEA eliminate the 21^st Century program but allow states to support afterschool programs through other titles, if they want to. That seems like a prudent approach. Some states may want to go all in, as the Higher Achievement program does. Some may have scientific evidence that their local programs improve outcomes. And some may conclude that they want to use funds for other purposes. Whatever path a state takes, let’s look at the evidence without rose-colored glasses.

      

There are many Americans who would benefit from a postsecondary education but who never attend college, or who start college but don’t earn a degree. Many come from low-income families.¹ Addressing gaps in educational attainment by family income, which exist even among similarly prepared students, is one of the most significant challenges facing policymakers concerned about income inequality and socioeconomic mobility.

It’s important that we closely track progress – or regress – in closing these gaps. Are we closing the gap in educational attainment between those who grew up in rich and poor families? How much progress did we make this year? Last year? Over the last five years? The currently available data are simply not up to answering these questions. It should be a priority for statistical agencies, particularly the Department of Education, to focus on this critical gap in our knowledge.

In this essay, we describe the data gaps in tracking income differences in educational attainment and the pitfalls they create for analysts. We also propose how to fill these gaps in our knowledge.

First, what we do well. We can track educational attainment of the population on an annual basis: the Current Population Survey shows, for example, that the bachelor’s degree attainment rate among 24-year-olds rose to 30 percent in 2014, its highest level ever, although it had hovered between 27 and 30 percent in every year since 2008.²

Now, what we do poorly. We can’t say how these statistics, and particularly their annual values, differ by parental income. To calculate such statistics, we need data that links young adults (and their educational attainment) to their parents. The federal government releases such data only every ten years – far too infrequent to guide policy and gauge progress.³ The most recent federal data that lets us calculate income gaps in educational attainment is for the high school class of 2004, most of whom turned 26 in 2012. At that point, the gap between the top and bottom income quartiles in college graduation was 37 percentage points.⁴ We are now three years behind in knowing whether we are making progress (or regress) in closing this gap, and it will be several more years before updated data are available.

 Attempts to produce annual estimates with the wrong data can produce wildly misleading results. A recent publication from the Pell Institute for the Study of Opportunity in Higher Education reported that 99 percent of college students who grew up in the top income quartile went on to complete their BA, compared to 21 percent in the poorest quartile.⁵ These findings received prominent attention from the news media and among policy analysts. Libby Nelson of Vox declared herself “gobsmacked” by the attainment data, and Richard Reeves of Brookings had a similar reaction: “Wow. I mean, WOW.”

To put it bluntly, these statistics are wrong. They are based on data from the monthly Current Population Survey (CPS). It would be wonderful if the CPS provided the income of young adults’ parents, but as a rule it does not. Parental information is present in the CPS only if the young adult is currently living at home, or is temporarily away from home. The primary reason a young adult is temporarily away from home is to attend college. Once a child forms her own household, she disappears from her parents’ CPS record.

If we limit our analysis of the educational attainment of young adults to those who are linked to their parents’ CPS records, we have committed the mortal sin of selection bias: selecting the sample on the variable of interest.⁶

The figure below shows that, in 2013, we have data on parental income for only 35 percent of 24-year-olds, the age group typically examined to track trends in bachelor’s degree attainment. When this same cohort was 16, in 2005, the vast majority (92 percent) were linked to their parents’ records. As the cohort aged, the share missing parental information steadily increased, becoming a majority by age 22.

Source: Authors’ calculations using October CPS data, 2005 (age 16) through 2013 (age 24)

Notes: Living in parents’ household defined as respondents of listed age identified as living in parent’s, grandparent’s, or foster parent’s household.

This shrinking sample would not be a problem if children randomly exited their parents’ households. If that were the case, the randomly-selected 35 percent of 24-year-olds for whom we have parents’ income could still inform us about gaps in BA attainment. But, as noted above, college enrollment determines whether a child is attached to her to parents’ household record in the CPS.

Since there are income gaps in college enrollment, there are also income gaps in the share of adult children for whom we have parental income in the CPS. The figure below shows that high-income families are overrepresented in the CPS data as their children age.⁷ Fifty percent of 24-year-olds from high-income families are attached to their parents’ household, compared to only 28-37 percent of children from all other families.

Source: Authors’ calculations using March CPS data, 2005 (age 16) through 2013 (age 24)

Notes: Percent remaining in parents’ household defined as number respondents of listed age identified as living in parent’s, grandparent’s, or foster parent’s household divided by number of respondents identified as such at age 16. Income quartiles are based on households with children between the ages of 14 and 16 in each year.

Tom Mortenson was kind enough to share with us the methodology he used in generating the statistics for the Pell Institute report. He explained that the CPS data in the report were statistically adjusted to account for the exit of adult children from their parents’ households. The statistical adjustments were made using data from a federal study of the high school class of 1982, High School and Beyond (HSB). The HSB data link parents and their adult children. For the relevant cohort, which was 24 in 1988, this statistical adjustment (by construction) produces fairly accurate CPS estimates of income differences in college attendance and completion.

But a lot has changed since 1988 and, as a result, this decades-old adjustment no longer produces correct CPS estimates. In fact, in some years, this adjustment has indicated that more than 110 percent of some groups had completed college – a red flag that this approach has gone off the rails.

We can see that the CPS estimates are incorrect by comparing them to those produced by the (infrequent) surveys that consistently link adults to data on their parents’ income. One is the National Longitudinal Survey of Youth (NLSY). In the NLSY, among those who turned 24 between 2005 and 2008, 54 percent of the top income quartile completed a BA. The estimate is similar (54-59 percent) in the Panel Study of Income Dynamics⁸ and in Department of Education data recently discussed by Sandy Baum.

These three longitudinal surveys provide a consistent picture for cohorts turning 24 between 2005 and 2008: 54-59 percent of those raised in the top income quartile earned a BA. The Pell Institute estimate for this group is 75-80 percent, which is off by a full twenty percentage points.⁹

The report’s estimates of the graduation rate of college entrants from the top income quartile are also wrong. From the NLSY, we know that for recent cohorts about 65 percent of those raised in the top income quartile earned a BA. The equivalent Pell Institute estimate is 99 percent – the eye-catching, but incorrect statistic that hit the headlines.

We are not cherry-picking these numbers. If we look to other cohorts, we see similar, systematic errors in the Pell Institute’s CPS estimates.

We share the Pell Institute’s deep concern with growing income gaps in children’s educational attainment. That’s why we think it’s extremely important to make sure we get the data on these gaps correct, so we can know if they are getting worse or better and then act on our knowledge.

Margaret Cahalan, director of the Pell Institute, indicates they will be reassessing the data used for future reports: “Our plans for the 2016 Indicators Report will include looking at trends in so far as possible from the NCES high school longitudinal studies as well as more explanation as to the limitations of the CPS data on educational attainment.”

We also shared our analysis and policy conclusions with Tom Mortenson of the Pell Institute, who had this to say: “We need better data! I support a national unit record data system, repeal of the student privacy legislation for research purposes, and an annual or perhaps biennial federal longitudinal study that follows up for ten years after high school graduation.”

We agree. At present, there is no data source that can be used to credibly and consistently measure these gaps on an annual basis. The existing longitudinal databases are administered too infrequently (NLSY and various Department of Education datasets) or have samples too small to reliably measure annual values (PSID).

The federal government could solve this problem at low cost by supplementing surveys with administrative data. The CPS records of young adults could be linked to administrative data on their parents’ income held by the Internal Revenue Service or Social Security Administration. Or, the IRS could release a data series on college attendance by family income, since they have collected data on college attendance (though not graduation) since the late 1990s. This information could be calculated for the nation as a whole as well as for individual states. Finally, NCES could conduct its longitudinal studies more often but with less voluminous surveys.

We end with a caution for consumers of data and statistics. If a statistic seems wildly wrong, it probably is. Most of us probably know enough college dropouts from the top income quartile to know that a 99 percent college completion rate can’t possibly be right. 

1 Bowen, Chingos, and McPherson (2009).

2 Authors’ calculations using March CPS. We thank Katharine Lindquist for exceptional research assistance with the analysis of the CPS data.

3 The Panel Study of Income Dynamics is administered yearly, but the samples for the relevant age groups are too small to produce precise estimates of postsecondary outcomes by family income on an annual basis.

4 Authors’ calculations from the ELS:2002 indicates that 17 percent of students from the bottom income quartile attained a bachelor’s degree or higher by 2012, as compared to 54 percent of students from the top income quartile. Given the use of binned incomes in the ELS survey question, we classify 21 percent of students in the bottom quartile and 26 percent in the top quartile.

5 From the report: “In 2013 the top quartile approached universal completion of a bachelor’s degree among those who entered college,” referring to figure on p. 33, which plots this series over time.

6 See also the discussions of this issue by Dynarski (2000, p. 638) and Cameron and Heckman (1999, p. 119).

7 As noted above, CPS respondents cannot be tracked over time. Instead, we track cohorts from one year to the next (and fix the income quartile cutoffs using the distribution of households with children age 14-16), ignoring any impacts of immigration and emigration. This analysis uses the March CPS data because income is measured continuously, whereas the income question in the October CPS is categorical and cannot be divided into equal-sized quartiles. March is also preferable to October for our purposes because March has a higher response rate to the income question.

8 Duncan, Kalil, and Ziol-Guest (2015).

9 From the report: “In 2013 individuals from the highest-income families were 8 times more likely than individuals from low-income families to obtain a bachelor’s degree by age 24 (77 percent vs. 9 percent),” referring to figure on p. 31.

      

The primary obstacle to faster progress in U.S. education reform is hard to put your finger on, because it’s an absence, not a presence.  It is not an interest group or a manifest social problem.  It is the infrastructure we never built for identifying what works.  It is the organizational framework we’ve not yet constructed for building consensus among education leaders across the country to identify what’s working.  Before you roll your eyes at another call for more research by a self-interested researcher, consider the following argument:

For the largest pharmaceutical companies, more than 80 percent of Phase II clinical trials failed between 2008 and 2010.[i]  Do we have any reason to believe that educational interventions will have a higher success rate?  Student learning and the process of adult behavior change in schools are just as complex as the typical disease process—and probably less well understood.  It is impossible to anticipate every obstacle and complication.  We should anticipate that most new ideas will fail and develop the infrastructure for testing a large number of them on a small scale first.

The concept of a clinical trial—the small scale deployment of a promising idea, with a comparison group for measuring efficacy—is foreign to education.  As with Race to the Top, we tend to roll out reforms broadly, with no comparison group in mind, and hope for the best.   Just imagine if we did that in health care.  Suppose drug companies had not been required to systematically test drugs, such as statins, before they were marketed.  Suppose drugs were freely marketed and the medical community simply stood back and monitored rates of heart disease in the population to judge their efficacy.  Some doctors would begin prescribing them.  Most would not.  Even if the drugs were working, heart disease could have gone up or down, depending on other trends such as smoking and obesity.  Two decades later, cardiologists would still be debating their efficacy.  And age-adjusted death rates for heart disease would not have fallen by 60 percent since 1980.

Sound far-fetched?  That’s exactly how our ancestors ended up practicing bloodletting for 2,500 years of our history.  It’s been six years since the Race to the Top Initiative, and there’s still no consensus on whether the key ideas behind those reforms are producing progress.   How are we ever going to generate momentum for an education reform agenda without systematically testing the various components in limited ways before rolling them out broadly.

I have frequently heard practitioners claim, “We don’t need more research.  We know what to do, but we don’t have the resources—whether it be funding or time or political courage or regulatory flexibility—to do it.”  I’m sure that some of those who make such claims really do have worthwhile ideas they’re championing.  However, the collective implication of their dismissal of the need for evidence is a Tower of Babel, in which leaders pursue their own visions, only to be followed by a successor with their own vision of reform.  We don’t lack innovation in education.  We lack the ability to learn from innovation.

The latest fad in education reform is to empower small teams of practitioners to seek out solutions within their own settings.  It’s an understandable reaction to the high-level policy-driven reforms in education of the past two decades.  However, unless those solutions are systematically tested on a larger scale, with plausible comparison groups, those efforts will just add to the confusion.

It is a matter of arithmetic.  According to an analysis of a variety of national tests by Black, Bloom, Hill, and Lipsey, the annual measured achievement growth of the average student in grade three onward is a half standard deviation or less.  It is the same in math, English, science, and social science.  By grade six, the average annual improvement is less than three-tenths of a standard deviation.  When a whole year of education and life experience—not to mention physiological changes in the brain—results in changes of less than .3 standard deviations, we should expect much smaller improvements from any classroom changes in a single year.

For example, teachers are on a very steep learning curve during their first few years of teaching while they become familiar with the basics of classroom management, lesson design, and delivery.  However, the growth in student achievement associated with such professional learning is less than .08 student-level standard deviations.

It’s probably safe to say that most educational interventions will generate smaller improvements in instruction than the typical teacher undergoes in the first few years of teaching.  However, in order to have better than a fifty-fifty chance of detecting a .08 standard deviation improvement, one would need a total sample of 2,000 students (if randomly assigning individual students to treatment and control).  The student sample size requirements will be even higher if the experiment involves clusters of classrooms or schools. 

If there were 75 students in a treatment group (roughly three elementary school classrooms or the average teaching load of one middle school teacher), the chance of being able to reject an impact of .08 would be roughly seven percent—only slightly higher than the probability of a false positive (using a .05 level for the hypothesis test).    

The small scale, atomized, practitioner-driven search for solutions will not yield reliable evidence on its own.   Rather, the most promising ideas need to be tested across larger samples of classrooms.  Otherwise, we’ll broadcast a cacophony of false positives and false negatives.

Fortunately, the incremental cost of evaluating any new education initiative has dropped dramatically in recent years, as a result of annual testing and investments in state and local longitudinal data systems.  However, state and local governments don’t currently have the staff or structures to take advantage of that opportunity.  We need new organizational structures for identifying schools or teachers implementing any given strategy (as well as for identifying statistical comparison groups pursuing other strategies).  Those networks will be most valuable if they extend beyond a single school district or state.  We also need to make it easier for state and local decision-makers to pilot interventions on a small scale and learn quickly whether those interventions are working.  Finally, we need to create new venues for state and local leaders to make sense of the latest findings.  Until we develop the capacity to systematically test our ideas for reform, we are doomed to continue reinventing the wheel.  It is a system failure and it requires a systemic solution.

Currently, the Institute of Education Sciences provides $54 million per year to regional education labs.   Those dollars are not having much impact.  Instead of funding a national network of 10 regional education labs, suppose those funds were made available on a competitive basis to individual state agencies willing to help districts track the impacts of their own efforts, to identify matched comparison groups for their initiatives (using other students and schools in a state) and to generate reports on impacts.   Districts that are deploying similar initiatives—such as professional development for the Common Core or educational software interventions—could band together and evaluate their efforts jointly, thereby increasing their collective statistical power.   Every region may have its own regional education lab now—but the model is not working.  Once we hit upon effective models for allowing leaders in a few states to test and learn from their own efforts, the model could be deployed nationally.  

      

The Brown Center on Education Policy at Brookings recently released the fourth iteration of its annual Education Choice and Competition Index (ECCI). The 2014 ECCI examines the status of K-12 school choice during the 2013-2014 school year in the 100+ largest school districts in the U.S.^[i]  The ECCI describes the state of school choice based on data derived from the federal government’s National Center for Education Statistics, individual school district websites, surveys of district personnel, and performance by schools on state assessments of academic achievement.  The data are organized based on a conceptual model in which good implementations of school choice provide parents with: many choices among types of school; a supply of comparatively higher performing schools; good information on school quality on which to base choice; a choice process that is efficient and equitable to all students; funding that follows students to their school of choice with policies to close unpopular schools; and free transportation for students from home to any school of choice.

In his keynote address at the release of the 2014 ECCI, Senator Lamar Alexander noted that he felt like a character in the movie Groundhog Day because he had been giving the same speech on school choice every ten years, predicting in 1992 that by the year 2000 school choice would no longer be an issue as all parents would be able to freely choose any K-12 accredited school for their child. 

Is the nation, in fact, stuck in a recurring scene in which parents awaken to the sounds of Sonny and Cher on their radio and trundle their children off to the public school that is closest to their home because that is their only option?  Or have we broken out of the tradition of zip code education in ways that suggest that Senator Alexander’s rosy prediction on school choice, first given in 1992, is closer to realization than many would think?

Heretofore, our annual ECCI release has not covered a long enough period to detect meaningful trends.  In this report, we lengthen our analysis as we introduce and utilize annual data based on the scoring rubrics in the ECCI that extends the series backward in time to the 2000-2001 school year. 

Our present interests are descriptive.  We address some of the dimensions on which school choice has changed in large school districts since the beginning of this century and some of the dimensions on which choice has been static.  We believe this information, provided here as a preliminary first look at our newly constructed dataset, provides important context for several constituencies.  Among them is the U.S. Congress, which is presently about the business of reauthorizing the Elementary and Secondary Education Act.  It should be relevant for federal legislators to know whether school choice is an idiosyncratic policy preference that has always been scattered around America’s largest cities and school districts, or is something that is moving with a speed and direction that suggests a public appetite. If the latter, what should Congress do to address areas in which school choice is impacted by federal policy? 

Information on change and stasis in school choice is also important to decisions by policymakers and voters at the state and local levels.  Where do states and large school districts stand with respect to the counterparts against which they benchmark themselves, and with respect to general trend lines?  Information on long-term trends in school choice also can inform the efforts of advocates (and opponents) of school choice by revealing features of school choice policies that seem amenable to or resistant to change.

Enrollment in alternative schools

An important component of choice is variety.  To the extent that all schools provide the same curriculum to similar students with similar teachers and staff, choice is between Tweedledum and Tweedledee. We see modest growth over the time period we examine in enrollment in alternatives to traditional public schools in the form of magnet schools (up from seven percent to 10 percent) and charter schools.  In contrast, enrollment in private schools has declined (from 13 to 11 percent).  Thus, a parent living in a metropolitan region served by one of our large school districts has, on average, a bit more choice of an alternative school today than in 2000, though regular public schools are still the dominant service providers.  There are substantial differences among districts on this variable, with, for example, a substantial majority of students served by alternative schools in New Orleans, LA and a near parity between alternative and traditional schools in Washington, D.C., whereas nearly all students are served by traditional public schools in districts such as Fort Worth, TX and Santa Ana, CA.

Student Enrollment

Availability of school choice

One of the most central dimensions we tracked is the extent to which school districts make choice easily available, either through a process in which parents have to choose, or through a process in which students receive a default assignment to a neighborhood school but parents can easily seek a transfer of their child to another school.  As depicted in the following graph, changes over time in the availability of school choice have been dramatic.^[ii]  In the 2000-2001 school year, only 24 percent of districts afforded parents school choice (20 percent through easy transfers from default schools and four percent through a full-fledged open enrollment process).  Today, that number has more than doubled to 55 percent of districts allowing choice. Put another way, in 2000-2001, 75 percent of our districts made transferring out of one’s default assigned school difficult or nearly impossible. Today that number has dropped to 45 percent.

Change in Student Assignment to Schools Over Time

Availability of distance (virtual) education

We see similar trends in other facets of choice. In 2000-2001, 13 percent of districts offered virtual programs or allowed their students to enroll in virtual classes that counted towards graduation or matriculation. Today, that has jumped to 88 percent.

Virtual Programs or Courses Allowed

Managing the supply and funding of schools based on their popularity

In 2000-2001, 19 percent of the schools in our sample had a published policy to close or restructure schools based on declining enrollment.  Today, that has more than doubled to 50 percent.

Policy to Restructure or Close Schools with Declining Enrollment

There are even larger changes (from about 10 percent to about 80 percent) in the proportion of districts that fund schools based on a formula in which a substantial portion of each school’s allocation of district funds is determined by enrollment, both in terms of size and student needs (i.e. special education, ELL, etc.).  Such funding policies, when combined with easily available school choice, have the potential of putting competitive pressure on individual schools that are losing students to make themselves more attractive to parents and students.

Popularity of Schools Reflected in Funding

Transportation isn’t on the move

Whereas most of the components of school choice tracked by the ECCI have changed, sometimes substantially, since 2000-2001, the provision of transportation to students has not budged:  10 percent of districts then and now provide transportation for students to any public school of choice within the district.  Limitations on transportation of students to and from school place severe practical constraints on the exercise of school choice for families in which all the adults hold down jobs with 9-5 workdays or do not have a car. 

Summing up

It may feel like Groundhog Day to Senator Alexander, but, in fact, since the 2000-2001 school year, the story of school choice in the nation’s largest districts has been one of change rather than repetition.  The senator’s prediction of a day in which every parent chooses her child’s school is still far from realization, but over a quarter of children in large schools districts today are attending alternative schools that have been chosen by parents, roughly half of districts make it relatively easy for a parent to exercise choice among public schools, and districts are managing their portfolio of schools and budgets in ways that favor popular schools.  For advocates of school choice, that is progress.  And for policymakers, these shifts indicate that there is both public interest and political feasibility for school choice in the nation’s largest school districts. The stagnation of transportation options points to an area for policy improvement, and a possible reason why the parents who may benefit the most from school choice are the least able to choose schools outside of their neighborhood.

This quick look at trends in school choice does not address the impact of changes in school choice on outcomes such as student achievement and school productivity.  But the data we have assembled should be useful to researchers in addressing these questions.  This database is available to qualified researchers for a variety of analyses and potential projects.  For more information please click here.

      

With both houses of Congress moving apace to reauthorize the Elementary and Secondary Education Act (ESEA), the question is not whether the new legislation will reduce the federal government’s footprint in K-12 education; it assuredly will.  The question is whether, in their understandable efforts to rein in Washington’s influence, legislators can preserve those elements of federal policy that stand to benefit students and taxpayers—particularly those that fulfill functions that would otherwise go unaddressed within our multi-layered system of education governance.

One key unresolved issue involves the status of competitive grant programs, through which the Department of Education invites states and school districts to apply for funds to support programs that address federally identified priorities.  In the current environment, Congress may be tempted to eschew all programs structured in this way, preferring to rely on formulas to ensure that schools receive their fair share of federal funds.  That would be a mistake.  Flexible competitive grant programs that encourage innovations in policy and practice and ensure that they are subjected to rigorous evaluation should remain a part of ESEA going forward.  In particular, the Investing in Innovation (i3) fund, a program created through the American Reinvestment and Recovery Act that is not a part of the reauthorization bills now moving through Congress, deserves a second look.

Increased reliance on competitive grants has been arguably the defining feature of the Obama administration’s K-12 education policy.  Its signature Race to the Top program (RTT) asked states to compete for $4.35 billion in federal grants based on their commitment to implement a 19-item reform agenda.  Expansive in its scope, RTT quickly became a symbol of what Senate Health, Education, Labor and Pensions Committee chairman Lamar Alexander has characterized as the Department’s efforts to dictate to states and school districts the details of how best to improve local schools.  Congressional discontent with RTT-style policies is not limited to Republicans, however.  Most legislators prefer to claim credit for funds allocated by formula rather than risk the ire of constituents whose applications are rejected, and rural members in particular often feel as if their districts are at a disadvantage when funding is competitive.  Perhaps because of this discontent, President Obama’s 2016 budget proposal did not include funds for a new RTT competition.

But rather than paint all competitive grants with a broad brush, it is useful to consider differences in their structure.  The table below shows that competitive grant programs can vary on at least two dimensions.  First, the programs can be broad, aiming to incentivize policy changes in multiple areas in one fell swoop, or narrowly focused on a specific challenge facing most school systems.  Second, grants can be awarded based on applicants’ willingness to commit to a detailed set of policy changes and program requirements prescribed by Washington, or they can be awarded based on past success, with funding levels tied to the strength of the evidence the applicant is able to present of their program’s effectiveness.

Federal Competitive Education Grants: A Typology with Examples

RTT epitomized the broad, prescriptive approach to competitive grants.  Although presented by supporters as an opportunity for states to put forward their best and most innovative ideas, in fact the selection criteria amounted to a detailed list of commitments in areas ranging from state standards and data systems to teacher evaluation systems and strategies to turn around low-performing schools.  Because funding was based primarily on future commitments, the program did little to alter the compliance-oriented relationship between federal officials and state and local educators once grants were awarded.  As Rick Hess of the American Enterprise Institute has written, “the aftermath entailed years of invasive federal monitoring…during which junior staff at the U.S. Department of Education exerted remarkable influence over the states that received RTT funds.”  While it is too soon to know whether states awarded RTT grants will see improvements in student outcomes, there is little hope that their efforts will be a source of rigorous evidence on the merits of specific policies they pursued.  The sheer number of policies states were required to implement simultaneously makes it all but impossible to isolate the impact of any one.

Yet RTT was the exception, not the rule.  Its scale reflected the unique circumstances of the post-financial crisis stimulus package, and maintaining a single competitive grant program at this scale has already proven to be politically infeasible.

Other competitive grant programs are structured quite differently, with a narrow focus tied to a distinct federal purpose.  For example, the Teacher Incentive Fund created by the second Bush administration asks school districts and charter schools to commit to implementing performance-based teacher compensation systems.  The rationale is that local officials will be more likely to adopt politically controversial changes to how teachers are compensated when outside resources are available to support their efforts.  For the past few years, the Department of Education has also offered grants directly to Charter Management Organizations seeking to expand or replicate high-quality schools.  Those schools need not adhere to a particular pedagogical model but must instead document a track record of improving student outcomes.  The Teacher Incentive Fund and grants to expand and replicate high-quality charter schools have been included in both the House committee’s bill and in Senator Alexander’s initial discussion draft.

Those bills do not, however, include the Investing in Innovation fund (i3), the second major competitive grant program created through the stimulus package.  Initially funded at $650 million, i3 allowed school districts, charter schools, and non-profit organizations working in partnership with one of those entities to apply for grants to support innovative programs aligned with one of four broadly defined federal priorities (e.g., supporting effective teachers and principals or improving the use of data).  In other words, i3 was broad in its focus but avoided prescription with respect to the design of the programs eligible for federal support. 

The origins and implementation of i3 have been ably chronicled by Ron Haskins and Greg Margolis, who present the program as a cornerstone of the Obama administration’s broader efforts to base spending on social programs on rigorous evidence.  Two specific aspects of the design of i3 are especially noteworthy.  First, the competition used a tiered evidence model to align the amount of funding a program could receive to the strength of the evidence to support its effectiveness.  Second, grant winners were required to conduct rigorous evaluations and were selected in part based on the quality of their proposed evaluation design.  Across the first four funding cohorts, i3 supported 53 randomized-control trials—the gold-standard design for evaluations of program effectiveness and one that until recently was virtually unknown in the education sector.  (Full disclosure: my primary employer, the Harvard Graduate School of Education, has benefited from i3 as a direct grantee and through evaluation contracts; I am the principal investigator on two of those contracts.)

A competitive grant program that includes these design elements need not be called i3.  Indeed, it need not be drafted as a standalone program at all.  The Coalition for Evidence-Based Policy has proposed language that would simply allow the Department of Education to reserve up to one percent of funding of all ESEA programs (except Title I) to award grants for innovation and research, with grant amounts based on the tiered evidence model used in i3.  The proposal is modeled on the Small Business Innovation Research program under which 11 federal agencies since 1982 have set aside a small percentage of their budgets to award grants to small companies engaged in the development and evaluation of new technologies. As the Coalition notes, both the Government Accountability office and the National Academy of Sciences have offered consistently positive assessments of the program’s success.  Importantly, the proposal like SBIR would include small businesses as eligible grantees, addressing a shortcoming of the original i3 program that arguably limited the types of innovations proposed. 

The Coalition’s proposal could be strengthened by giving the Institute of Education Sciences the lead role in assessing the strength of applicants’ evidence of effectiveness and in supporting required evaluation activities.  The risk with these competitions when carried out by the Office of the Secretary is that they become politicized, that they are judged by review panels without methodological competence, and that they are overseen, once awarded, by career staff in program offices that do not have the background to monitor what is, at root, a program evaluation grant.  These risks could be substantially reduced if the competition were funded as a line item in the IES budget, with statutory language requiring that review panels include both practitioners and researchers.

Properly designed competitive grant programs provide an opportunity for Congress to target resources at federal priorities and encourage innovative problem-solving while avoiding federal mandates.  They should avoid prescription and both reward and produce rigorous evidence, thus increasing the share of education dollars spent on evidence-based programs while at the same time fulfilling the federal government’s unique responsibility for producing and disseminating high-quality evidence on the best ways to improve American schools.  The i3 program was a promising step in this direction.  It would be unfortunate if Congress were to miss the opportunity to make something similar a permanent feature of ESEA.

      

Late last year, researchers at the University of South Carolina and the University of California, Los Angeles, published a report^[1] on the relationship between student loan debt and psychological well-being.  This study comes in the midst of a plethora of new research attempting to quantify causal relationships between student loan debt and personal outcomes (including home ownership,^[2] entrepreneurship and the macro economy).  Despite the intense interest in this issue among researchers, this is the first paper that attempts to understand the emotional cost of carrying student loan debt.  This question is, in fact, more fundamental than the others being posed in this genre of research, since it could help to explain the mechanism through which debt may be affecting other outcomes.  Using a strict classical lens to examine this issue might lead one to conclude that the true cost of carrying debt could be measured in strictly financial terms.  However, the widespread and growing discontent among households with student debt paired with the evidence that the financial circumstances of borrowers haven’t radically worsened^[3] suggests that an alternate lens may be necessary.  In particular, a lens that considers the possibility that student loans take an emotional toll on borrowers, even when wealth is held constant.^[4]     

The new study, carried out by Katrina M. Walsemann, Gilbert C. Gee and Danielle Gentile, uses data from the National Longitudinal Survey of Youth (1997) to examine the relationship between student loan debt and self-reported psychological health.  Survey respondents were asked to answer 5 questions relating to emotional health^[5] and the categorical responses were used to generate an index of psychological well-being.  Regression analysis was then used to examine the relationship between this measure and debt, including both cumulative borrowing and annual borrowing during in-school periods.   

Based on their analysis, the authors report, “cumulative student loans were significantly and inversely associated with better psychological functioning.”  In other words, individuals with more student debt reported lower levels of psychological health, when other things are held constant (including occupation, income, education and family wealth).  The effect is statistically significant, but it is quite small.  They also find that “the amount of yearly student loans borrowed was inversely associated with psychological functioning,” which implies that taking on debt is emotionally costly for students. 

This work takes an important first step in helping us to understand the emotional toll of student loan debt, but could benefit from an alternative framework for evaluation.  The researchers attempt to control carefully for covariates that are closely related to psychological well-being and debt.  With this approach, they are effectively examining the relationship between debt and psychological well-being while holding all else constant, including income.  On one hand, this is good, because we know that income is independently related to well-being.  On the other hand, it means that the variation in education debt that is being examined in the study is being driven by variation in wealth, which we also know is related to psychological well-being.  As a result, this research falls short of being able to tell us whether student loan debt pays an emotional toll that is any greater than the toll imposed by other debts or financial obligations. 

An alternative framework, which is arguably more difficult to implement, would hold wealth constant while varying debt.  Since this may seem like a semantic difference, I’ll provide an example.  Consider that an individual, Sally, could be in one of two scenarios.  In the first, she earns a monthly income of $1,000 and has no debt.  In the second, she earns $1,500 each month but has to make a $500 payment on a student loan each month (assume that this payment is made in perpetuity for the sake of simplicity).  The level of wealth is constant between the two scenarios, but she has a debt obligation in one and not in the other.  The question that is relevant to policymakers is; which of these scenarios does Sally prefer?  Or alternatively, by how much does she prefer the scenario in which she doesn’t carry any debt?  This example is overly simplistic, but with a slightly more realistic example, one could also ask how Sally’s distaste for the scenario that includes indebtedness would change if the period (duration) of the loan were different or if payments were building equity in a transferable asset (i.e. a car or house). 

The answers to these questions will tell us how concerned we need to be about the growing reliance on debt as a means for financing investments in human capital.  While debt is an efficient mechanism for enabling access to higher education, it may be the case that debt imposes emotional costs that counter some of the benefits of this efficient mechanism.  If research ultimately indicates that carrying debt imposes a significant emotional cost to borrowers, then there may be a role for policymakers to provide some relief.  One option would be to alleviate the emotional toll of debt through changing the tone of the public discourse on this issue.  The treatment of student debt by the popular media has almost certainly caused some borrowers to worry about their debts more than they would have otherwise.  It may be possible to alleviate some concerns about debt through educational programs that help borrowers to better understand their circumstances and the safety nets that are available to them.  Alternatively, instruments other than personal debt have the potential to alleviate an emotional tax.  For instance, income share agreements, or even a more socialized system for financing education, could succeed in achieving this end.  However, all of this should wait until research can tell us a more conclusive story about this issue. 

      

Debates on reauthorizing the Elementary and Secondary Education Act (ESEA) have focused on its requirement that districts give annual reading and math tests to students in third through eighth grades. On their own, tests just provide scores, but under No Child Left Behind (NCLB), a lack of growth in scores has consequences. Those consequences have fueled the debates about testing.

The NCLB scheme of consequences was designed fifteen years ago. Schools (or whole districts) that were failing to improve test scores had to, first create an improvement plan, then offer parents the choice for their child to attend a better-performing school, then offer tutoring after school for students who were not performing well (along with offering choice), and, lastly, restructure schools that failed to improve five years in a row (along with offering choice and supplemental services).

Two aspects of this scheme stand out. One is that only the first (planning for improvement) and last (restructuring) are about the school. A school that offers parental choice and supplemental services can continue to do whatever it was doing in its classrooms: same teachers, same principal, same materials. Having parents move their children to other schools might induce schools to try to improve to be more competitive and attractive to parents, but a school could ignore these forces. And supplemental services are delivered during after-school hours, usually by third-party organizations, and schools could ignore those too.

Research from the last fifteen years offers evidence about NCLB and the ways its consequences might be reconfigured to be more impactful. Hanushek and Raymond (2005) showed that having consequences improved test scores. Before NCLB, some states had accountability schemes with consequences and others had accountability schemes without consequences (the states only posted score results without tying them to penalties or rewards). Hanushek and Raymond showed that test scores on the National Assessment of Educational Progress (NAEP) rose in states that had consequential accountability but not in states that had what they called ‘report card’ accountability. Dee and Jacob (2010) compared NAEP scores in states that changed their accountability schemes to meet the requirements of NCLB—which in practice meant making consequences stricter—and showed that test scores rose more in these states than in states that did not have to change their consequences (because they already fit with NCLB’s requirements). 

Together, these findings are evidence that consequences matter. Reporting test scores may be useful for letting parents and communities know which schools are doing well and which are not. But making scores count leads to improvements.

More recently, Ahn and Vigdor (2014) provided insight into what it was about the scheme of consequences that improved scores. Using the long history of score data in North Carolina, they showed that schools that reached the first consequence—the schools failed to make adequate yearly progress and had to file an improvement plan—improved more than similar schools that just barely made adequate yearly progress and did not have to file an improvement plan. That this mild first consequence of having to plan for improvement meant schools improved is analogous to a parent saying “don’t make me come up there” to their child and having them behave better. Ahn and Vigdor also showed that the restructuring consequence improved test scores. Offering parents choice and offering supplemental services did not improve scores. Other studies are consistent with these findings.

The evidence points to having consequences being effective—they lead to improved scores—and consequences being more effective when they change what schools do, either mildly (having to plan for improvement), or dramatically (having to restructure). How can the middle consequences be made more impactful? The answer may be to focus on curriculum and instruction, which means teachers and principals.

NLCB’s requirement to administer annual tests in grades three through eight created a mountain of new data. And several states, including Texas, Florida, and North Carolina, had begun amassing score data before NCLB. Findings from research using these new data show that teachers differ hugely in their ability to generate score gains. Goldhaber (2015) summarized this research and noted that in upper elementary grades (under NCLB, required tests begin in third grade), having a lower-performing teacher (one at the 30^th percentile of teachers) is roughly equivalent to a student learning half as much in the school year compared to having a higher performing teacher (one at the 70^th percentile of teachers). These differences have been measured only for reading and math, but these are core subjects, and there’s little reason to believe the magnitudes would differ for science and social studies. 

The same line of research also found that it is hard to predict which teachers will be high performers. The best predictors of teacher effectiveness are how a teacher has already performed, and how long he has been teaching. Generally, high performers stay high performers. A massive study of teaching funded by the Gates Foundation had a related finding. On typical measures that schools use to observe teachers in classrooms, high performers had high ratings on all dimensions.

The fix, then, for schools performing poorly is straightforward but not practical: gauge effectiveness for all teachers in a district, and move high performers to low-performing schools. The Institute of Education Sciences tested something like this approach on a small scale. As part of its study, high-performing teachers were offered financial incentives to move to low-performing schools. Only one or two teachers were moved to any one school. The study found that high performers resulted in an improvement of an entire grade level’s test scores. If the high performer were a fifth grade teacher, for example, the entire fifth grade improved its test scores from fourth to fifth grade. The high performer’s class generally improved the most, but that improvement was so large it was enough to move the whole grade level up.

This fix is about as low-risk as one can get to improve performance of a whole school, like ensuring the U.S. wins an Olympic gold medal in basketball by putting ten NBA all-stars on its team. It’s hard to imagine doing this fix on a large scale, however. A practical though possibly less effective approach would be for low-performing schools to increase skills of their teachers. Upskilling quickly means bringing in skilled teachers as overseers or mentors, possibly transferring weak teachers out of schools and bringing in high performers, as noted already, or providing materials or technologies that improve teacher skills directly or indirectly. This is not “teacher professional development” as it’s usually understood. But a school facing consequences right now has little time for its teachers to attend classes, in-service workshops, or summer institutes. A manufacturing company facing bankruptcy because it is producing defective products does not send its employees to the local community college to take courses. It locates the cause of the defects and fixes them as soon as it can.

Suppose a school continues to perform poorly despite upskilling its teachers. What next? The focus would turn to the principal. (These approaches could also happen at the same time.) Another finding emerging from recent research is that, like teachers, principals differ widely in their effectiveness. Principals of low-performing schools can be assigned a mentor or coach, given added support, or replaced by a known effective principal.

If the school’s performance problem continues, after upskilling its teachers and principal, the last consequence—perhaps admitting defeat—could be to give parents the option to take their school funding or part of it to another school. It could work something like this: any school meeting standards that accepts a student transferring from a designated (low-performing) school receives a bonus in addition to standard per-student funding. These funds can be used to supplement teacher pay through bonus or incentive schemes, improve technology, or upgrade instructional materials. Why pay a bonus? Because better schools may not want more students. More students mean larger class sizes, a more crowded facility, and added clerical and logistical responsibilities.

Whether the federal government should mandate an accountability structure is a different question from whether it should mandate that there be accountability. And the informational complexity of improving teacher and principal skills school by school is an example of why a mandated structure may be too blunt an instrument. Even two neighboring schools in the same district can have different skill needs related to different hiring patterns, experience levels, teaching philosophies, technology infrastructure, and so on. If states design their own accountability structures, and the federal government asks that the structures indicate how they will improve teacher and principal skills, objectives are being identified at a high level and approaches for meeting them are being identified at a local level.

Evidence on the negligible effects of choice and supplemental services also needs to be viewed in perspective. Showing that parent choice and supplemental services under NCLB had little effect is not the same as showing that parent choice and supplemental services cannot be effective. The quality of implementation was weak, and averages can conceal positive outcomes for some. If states and districts had positive outcomes with either one, including these consequences in an accountability structure makes sense. As part of a state’s proposal for the scheme, the federal government could request evidence of why they are viewed as successful. This same approach of asking for evidence was used under NCLB to determine whether state assessments were adequate.

Improving teaching and school leadership will cost money. In the current draft of the reauthorized ESEA, Title II, “High-Quality Teachers, Principals, and Other School Leaders,” includes annual funding of about $3 billion. One of the title’s stated purposes is “increasing the number of teachers, principals, and other school leaders who are effective in improving student academic achievement in schools.” Focusing those resources on schools that need the most help could easily be done. 

      

A curriculum controversy is roiling schools in the San Francisco Bay Area.  In the past few months, parents in the San Mateo-Foster City School District, located just south of San Francisco International Airport, voiced concerns over changes to the middle school math program. The changes were brought about by the Common Core State Standards (CCSS).  Under previous policies, most eighth graders in the district took algebra I.  Some very sharp math students, who had already completed algebra I in seventh grade, took geometry in eighth grade. The new CCSS-aligned math program will reduce eighth grade enrollments in algebra I and eliminate geometry altogether as a middle school course. 

A little background information will clarify the controversy.  Eighth grade mathematics may be the single grade-subject combination most profoundly affected by the CCSS.  In California, the push for most students to complete algebra I by the end of eighth grade has been a centerpiece of state policy, as it has been in several states influenced by the “Algebra for All” movement that began in the 1990s.  Nationwide, in 1990, about 16 percent of all eighth graders reported that they were taking an algebra or geometry course.  In 2013, the number was three times larger, and nearly half of all eighth graders (48 percent) were taking algebra or geometry.^[i]  When that percentage goes down, as it is sure to under the CCSS, what happens to high achieving math students?

The parents who are expressing the most concern have kids who excel at math.  One parent in San Mateo-Foster City told The San Mateo Daily Journal, “This is really holding the advanced kids back.”^[ii] The CCSS math standards recommend a single math course for seventh grade, integrating several math topics, followed by a similarly integrated math course in eighth grade.  Algebra I won’t be offered until ninth grade.  The San Mateo-Foster City School District decided to adopt a “three years into two” accelerated option.  This strategy is suggested on the Common Core website as an option that districts may consider for advanced students.  It combines the curriculum from grades seven through nine (including algebra I) into a two year offering that students can take in seventh and eighth grades.^[iii]  The district will also provide—at one school site—a sequence beginning in sixth grade that compacts four years of math into three.  Both accelerated options culminate in the completion of algebra I in eighth grade.

The San Mateo-Foster City School District is home to many well-educated, high-powered professionals who work in Silicon Valley.  They are unrelentingly liberal in their politics.  Equity is a value they hold dear.^[iv]  They also know that completing at least one high school math course in middle school is essential for students who wish to take AP Calculus in their senior year of high school.  As CCSS is implemented across the nation, administrators in districts with demographic profiles similar to San Mateo-Foster City will face parents of mathematically precocious kids asking whether the “common” in Common Core mandates that all students take the same math course.  Many of those districts will respond to their constituents and provide accelerated pathways (“pathway” is CCSS jargon for course sequence). 

But other districts will not.  Data show that urban schools, schools with large numbers of black and Hispanic students, and schools located in impoverished neighborhoods are reluctant to differentiate curriculum.  It is unlikely that gifted math students in those districts will be offered an accelerated option under CCSS.  The reason why can be summed up in one word: tracking.

Tracking in eighth grade math means providing different courses to students based on their prior math achievement.  The term “tracking” has been stigmatized, coming under fire for being inequitable.  Historically, where tracking existed, black, Hispanic, and disadvantaged students were often underrepresented in high-level math classes; white, Asian, and middle-class students were often over-represented.  An anti-tracking movement gained a full head of steam in the 1980s.  Tracking reformers knew that persuading high schools to de-track was hopeless.  Consequently, tracking’s critics focused reform efforts on middle schools, urging that they group students heterogeneously with all students studying a common curriculum.  That approach took hold in urban districts, but not in the suburbs.

Now the Common Core and de-tracking are linked.  Providing an accelerated math track for high achievers has become a flashpoint throughout the San Francisco Bay Area.  An October 2014 article in The San Jose Mercury News named Palo Alto, Saratoga, Cupertino, Pleasanton, and Los Gatos as districts that have announced, in response to parent pressure, that they are maintaining an accelerated math track in middle schools.  These are high-achieving, suburban districts.  Los Gatos parents took to the internet with a petition drive when a rumor spread that advanced courses would end.  Ed Source reports that 900 parents signed a petition opposing the move and board meetings on the issue were packed with opponents. The accelerated track was kept.  Piedmont established a single track for everyone, but allowed parents to apply for an accelerated option.  About twenty five percent did so.  The Mercury News story underscores the demographic pattern that is unfolding and asks whether CCSS “could cement a two-tier system, with accelerated math being the norm in wealthy areas and the exception elsewhere.”

What is CCSS’s real role here?  Does the Common Core take an explicit stand on tracking?  Not really.  But de-tracking advocates can interpret the “common” in Common Core as license to eliminate accelerated tracks for high achievers.  As a noted CCSS supporter (and tracking critic), William H. Schmidt, has stated, “By insisting on common content for all students at each grade level and in every community, the Common Core mathematics standards are in direct conflict with the concept of tracking.”^[v]  Thus, tracking joins other controversial curricular ideas—e.g., integrated math courses instead of courses organized by content domains such as algebra and geometry; an emphasis on “deep,” conceptual mathematics over learning procedures and basic skills—as “dog whistles” embedded in the Common Core.  Controversial positions aren’t explicitly stated, but they can be heard by those who want to hear them.    

CCSS doesn’t have to take an outright stand on these debates in order to have an effect on policy.  For the practical questions that local grouping policies resolve—who takes what courses and when do they take them—CCSS wipes the slate clean.  There are plenty of people ready to write on that blank slate, particularly administrators frustrated by unsuccessful efforts to de-track in the past

Suburban parents are mobilized in defense of accelerated options for advantaged students.  What about kids who are outstanding math students but also happen to be poor, black, or Hispanic?  What happens to them, especially if they attend schools in which the top institutional concern is meeting the needs of kids functioning several years below grade level?  I presented a paper on this question at a December 2014 conference held by the Fordham Institute in Washington, DC.  I proposed a pilot program of “tracking for equity.”  By that term, I mean offering black, Hispanic, and poor high achievers the same opportunity that the suburban districts in the Bay Area are offering.  High achieving middle school students in poor neighborhoods would be able to take three years of math in two years and proceed on a path toward AP Calculus as high school seniors.

It is true that tracking must be done carefully.  Tracking can be conducted unfairly and has been used unjustly in the past.  One of the worst consequences of earlier forms of tracking was that low-skilled students were tracked into dead end courses that did nothing to help them academically.  These low-skilled students were disproportionately from disadvantaged communities or communities of color.  That’s not a danger in the proposal I am making.  The default curriculum, the one every student would take if not taking the advanced track, would be the Common Core.  If that’s a dead end for low achievers, Common Core supporters need to start being more honest in how they are selling the CCSS.  Moreover, to ensure that the policy gets to the students for whom it is intended, I have proposed running the pilot program in schools predominantly populated by poor, black, or Hispanic students.  The pilot won’t promote segregation within schools because the sad reality is that participating schools are already segregated.

Since I presented the paper, I have privately received negative feedback from both Algebra for All advocates and Common Core supporters.  That’s disappointing.  Because of their animus toward tracking, some critics seem to support a severe policy swing from Algebra for All, which was pursued for equity, to Algebra for None, which will be pursued for equity.  It’s as if either everyone or no one should be allowed to take algebra in eighth grade.  The argument is that allowing only some eighth graders to enroll in algebra is elitist, even if the students in question are poor students of color who are prepared for the course and likely to benefit from taking it.

The controversy raises crucial questions about the Common Core.  What’s common in the common core?  Is it the curriculum?  And does that mean the same curriculum for all?  Will CCSS serve as a curricular floor, ensuring all students are exposed to a common body of knowledge and skills?  Or will it serve as a ceiling, limiting the progress of bright students so that their achievement looks more like that of their peers?  These questions will be answered differently in different communities, and as they are, the inequities that Common Core supporters think they’re addressing may surface again in a profound form.