From the 1990s until the end of the federal No Child Left Behind Act (NCLB) in 2015, state and federal education reform policies had a virtually exclusive focus on holding public schools accountable for student test scores in reading and mathematics. The new Every Student Succeeds Act, the successor to NCLB, provides an opening for states to broaden their accountability regimes by including a non-traditional measure along with academic test scores. One possibility that has been embraced by many advocates is some type of measurement of student soft skills, which include social skills, self-management abilities, academic soft skills such as listening carefully to instructions, and approaches to learning such as willingness to take on challenging tasks.

Attention to soft skills among education reformers is presently skewed towards attempts to enhance and measure broad student dispositions that are abstract, context-free, not directly observable, assessed through self-report questionnaires, and dominated by genetic influences. A much more productive approach would emphasize soft skills that are specific, contextual, socially observable, easily malleable within the environment of classrooms and schools, and widely accepted as a responsibility of schools to support. Grit is an example of an abstract soft skill. An example of a non-abstract soft skill is a particular student working hard on challenging math problems during the first quarter reporting period in Mrs. Thomas’ fifth grade class.

Whereas personality inventories in which students report on their personal dispositions are the preferred measures of abstract soft skills, a student report card completed by a teacher is the embodiment of measurement of specific, non-abstract soft skills. This report introduces a worked example of how to measure specific soft skills, The Brookings Soft Skills Report Card, and uses it to illustrate important functions such a low-abstraction approach provides in contrast to the high-abstraction alternative. These functions include the ease with which teachers and other adults who are regularly around individual students can directly observe the soft skills they are expected to support, the clear implications for intervention suggested by low scores on a particular skill by a particular student or group of students, the signals sent to administrators about teachers and groups of students who may need additional help, and the usefulness in communicating with parents.

The thrust of this report is to demonstrate the value of having measures of soft skills that are simple and close to the classroom. Doing so is not incompatible with system-wide measures of soft skills that can be used for monitoring and accountability—this report illustrates how student characteristics captured in a report card frequently produce artifacts in available administrative records that can be used for system-wide accountability. These two efforts—classroom tools to be used by teachers and administrative records to be used by administrators—can proceed in parallel. Both differ from and are superior in education settings to testing children with personality inventories.

Background

This is the third of a series of Evidence Speaks reports on soft skills in K-12 education. The first two provided research reviews and conclusions that set the stage for a consideration of how to measure soft skills in schools.[i] The central takeaways are that:

• The domain of student soft skills as conceptualized by most education reformers is dispersive.

Soft skills that are the targets of present education reforms range broadly both in the type and in level of abstraction. The circumplex of soft skills represented in Figure 1 captures four categories or domains of behavior: social skills, self-management, academic soft skills, and approaches to learning. The vertical dimension, i.e., the height of the column or circumplex, represents abstraction: the degree to which any particular soft skill or soft skill category is specific, contextual, and socially observable (low abstraction) vs. broad, context-free, and available only as a student report of a self-reflection (high abstraction). An example of a low-abstraction soft skill is whether a student is observed by her teacher to finish math homework assignments on time. An example of a high-abstraction soft skill is whether a student reports on a questionnaire that he or she is a reliable worker. This is illustrated in Figure 1 with an example of high and low abstraction within the category of academic soft skills.

Figure 1: The Soft Skills Circumplex

The mashup of multiple categories of soft skills and various levels of abstraction into unitary school reform approaches is problematic.[ii] It leads to program descriptions and mission statements that are all over the waterfront, and to efforts at implementation that lack granularity and pose severe challenges in aligning goals, program content, desired outcomes, and measurement.

We have a critical need for more specificity, i.e., less abstraction, with respect to what soft skills students are to learn in school and for what purposes; when, how, and to whom those skills will be taught; and how the success of those efforts will be defined, measured, and evaluated.

• The broader and more abstract the soft skill that is the focus of a school reform effort, the more likely is the skill to have a dominant genetic basis.

This doesn’t mean that a student’s school environment is irrelevant to abstract soft skills. For instance, it is possible to teach individual students particular ways of behaving, e.g., submitting class assignments on time, that might look to a disinterested observer like a disposition or trait, e.g., conscientiousness. Further, the particular forms that are viewed as socially desirable vary from culture to culture and setting to setting and thus have to be learned by students (e.g., interrupting teachers to ask questions or to express opinions is standard practice in American classrooms whereas Japanese students are expected to be very quiet during class).[iii] And, finally, there are several research studies that demonstrate that what happens in schools and classrooms can impact measures of abstract soft skills.[iv]

Nevertheless, when behaviors that denote abstract constructs such as conscientiousness, grit, and growth mindset are observed for individual students in a large number of settings or captured through surveys and questionnaires that focus on generalities (“Do you complete tasks successfully?”), the rank ordering of individuals from high to low will have a substantial genetic component whereas the influence of the shared family or school environment will be weak.

Author

Grover J. “Russ” Whitehurst

Senior Fellow - Economic Studies, Center on Children and Families

This is demonstrated, for instance, in longitudinal studies that compare identical twins with same-sex fraternal twins reared in the same families or separated through adoption. Siblings who share the same genes (identical twins) end up being substantially more similar on abstract social and emotional traits such as conscientiousness and grit than siblings who share only half their genes (fraternal twins), regardless of whether they are raised in the same families and attend the same schools.[v] Thus, if we know only the genetic relationship between two students, we can make strong predictions about the degree to which they will be similar in an abstract soft skill such as conscientiousness. But the same effort to predict similarity between students will be weak if it is based only on knowledge of whether the students grew up in the same family or attended the same schools. The same is true for IQ.

This means that schools that intend to teach soft skills are rowing against a strong current when they focus on abstract dispositions such conscientiousness, grit, empathy, and the like. The school’s task will be much more easily accomplished if it is focused on soft skills at the lower end of the vertical dimension of abstraction.

• Neither the dispersiveness of soft skill reform efforts nor the high genetic loading of abstract soft skills argues against the importance of incorporating soft skills into the intentional mission of schools and classrooms.

As observation, intuition, empirical research, and a quick examination of the Department of Labor’s occupational employment statistics[vi] will demonstrate, success in life depends on hard skills: the individual’s capabilities in subject matter and tasks that are valued in society and are passed on through formal and informal instruction, e.g., being able to write computer code, or service heating and air conditioning equipment, or cook gourmet meals, or understand market derivatives.

But soft skills are also important, as evident intuitively, through surveys of businesses, and through systematic research reviews: social skills, self-management abilities, emotional and attitudinal approaches, and a host of situation-specific soft skills and knowledge that are ancillary to hard skills are important factors in success in school and in life.

The challenges for schools and those involved in efforts to improve the teaching and learning of soft skills are significant given the nascent nature of the enterprise and the significant gaps in knowledge. Meaningful progress depends on informed modesty about the likely returns on current efforts; greater specificity and more emphasis on context in the curricula and school-level approaches to teaching soft skills; and the development and use of practical assessments that are closely aligned with a specific framework for teaching and learning.

Defining and measuring soft skills

Should schools focus on high-abstraction dispositions or low-abstraction skills?

A disposition is a customary way of behaving that distinguishes one person from others, e.g., does the person stand out as cooperative, assertive, responsible, empathic, conscientious, persistent, agreeable, anxious, etc. across a large variety of settings and tasks? A skill, in contrast, refers to a person’s ability to carry out a particular activity successfully, e.g., giving effective forms of feedback to others, staying on task in the classroom, self-monitoring whether one’s behavior is having the intended effect, engaging in timely and expected social routines, and engaging in anticipatory thinking about automatic behaviors and biased beliefs that lead to trouble.[vii] Skills can be specific to situations, e.g., a student may be very good at staying on task in a computer game and deficient in doing so with math homework. Dispositions, in contrast, are behavioral tendencies that occur across disparate situations.

Dispositions are difficult to teach, not only because they have a strong genetic component, but also because they are, by definition, not tied to specific situations. Skills, in contrast, are typically acquired through specific instructional practices and observation learning so they readily lend themselves to generating relevant instructional approaches. Skills vary on a dimension of complexity/difficulty of acquisition from something so simple that it can be learned by one observation of someone else doing it, e.g., raising a hand in class to ask the teacher a question, to something that takes a lot of time, instruction, and effort to acquire, e.g., self-monitoring and correcting biased beliefs about other people’s motives.

Implicit in the prior discussion and Figure 1 are strong reasons for schools to focus on skills rather than dispositions: Skills can be taught, are typically publicly observable and specific, lend themselves readily to selection based on what the school or teacher intends students to learn, and aren’t heavily constrained by genetics.

What soft skills should be taught?

There is no single correct answer to this question because what is to be taught is a reflection of values and goals. A military school will almost surely have a different set of priorities for the soft skills it tries to inculcate in its students than a school for the performing arts. The answer will also depend on the age of the student and the student’s particular areas of strength and weakness—adolescents have different needs than younger students and students who already are competent in a requisite category of soft skills have different needs than those who are not. Thus, what students in a particular school system or classroom should know and be able to do with respect to soft skills requires conscious and thoughtful decisions by teachers and school leaders. Those decisions are foundational to everything else, including curriculum, measurement, and evaluation.

That said, there is substantial similarity across different types of schools and educational missions with regard to basic soft skills that benefit all students. The basic soft skills discussed below should have a comfortable fit within the explicit or implicit mission of a large proportion of schools and classrooms. The remainder of this report draws lessons on how to measure soft skills from a worked example, The Brookings Soft Skills Report Card (Report Card).

The Report Card, which is presented below, covers four categories of soft skills that most school leaders, teachers, and parents would agree are within the responsibility of schools to monitor and, when necessary, develop:  social skills, self-management, academic soft skills, and approaches to learning.

The first of these categories, social skills, includes how a student interacts with other students as observed by teachers and other adults. The second category, self-management, refers to observable manifestations of what has been referred to as executive functions or self-regulation, i.e., the student’s ability to take control over what would otherwise be automatic reactions by planning, focusing attention, reframing experiences, and using mental tools. These cognitive processes are frequently not publicly observable. However, the absence of them is, as, for example, when a student blurts out responses that because of their content and short latency suggest a lack of thoughtfulness. They can also be accessed by teachers through direct questions to students, for example: “What were you thinking when you did that?” The third category, academic soft skills, are both social and cognitive. Their defining feature is their ancillary role in carrying out traditional academic tasks, e.g., the ability to work independently. Finally, the category of approaches to learning includes such things as the student’s engagement in school, pleasure in learning, and anxiety about performance.

The vast majority of the individual item descriptions within each category in the Report Card are articulated in terms of observable behaviors, e.g., bullying, being respectful of teachers. The few that are not involve straightforward attributions about internal states of mind, e.g., a child who acts worried and anxious probably is. The items themselves are of my construction, inspired both by items used on an older generation of social behavior checklists designed by psychologists[viii] and by the categories of soft skill that frequently appear in the literature.[ix] Note that the items on the Report Card are for the purpose of creating a worked example. Schools/districts could very reasonably substitute or add items to fit their particular needs.

The low level of abstraction and high level of observability of the student behaviors to which the Report Card refers have two important practical advantages. The first is that it is easy for teachers and other adults who are regularly around individual students to experience directly what the Report Card asks them to score, e.g., does the child have friends, without the requirement for an investment in training. The second is that low scores on a particular item for a particular student or group of students have obvious implications for intervention. For example, students who have low scores on confidence in abilities and willingness to work hard may benefit from growth mindset training.[x] Students who are frequently late to class or absent from school may need counseling and an intervention with parents. Students who are aggressive with peers and quick to anger may benefit from training on how to think about and reframe the actions of others before reacting automatically.[xi] And so on.

Administrators and teachers can also take advantage of the Report Card items to identify both individual students and classrooms that need additional help, e.g., a classroom in which a lot of students are receiving low scores on self-management skills is a classroom in which the teacher needs help in classroom management.

A Report Card that summarizes scores over a reporting period in school also serves as a tool to inform parents.

What about accountability?

One of the consequences of the high-stakes state assessments that were mandated in NCLB and the requirement for a fifth indicator of school success in the present-day successor of NCLB (The Every Student Succeeds Act) is a preeminent concern among school and district leaders with how to measure student soft skills in a way that lends itself to grading teachers and schools.

The Report Card and anything built on a similar template is not intended for or designed to be a high-stakes assessment. That is why it is called a report card rather than an assessment. It is designed to support individual teachers in the task of thoughtful observation of students to identify their strengths and weaknesses in terms of soft skills, and thereby to aid in efforts to help students.

It is also designed to be useful for parents, e.g., sent home at the end of each quarter with the average score for the student for each item indicated, and with notes from the teacher where relevant.

Schools already send home report cards with elements of the Brookings Report Card on them, so this is an expected form of communication. Further, districts/schools often provide information to parents on how to build skills at home, e.g. resources for literacy. They could provide information for parents on a website with direct mapping to the items on the Report Card. This opens the door to inexpensive, low-tech ways parents and schools can help build soft skills.

Of course, there are legitimate needs at the school building, district, and state levels for information on soft skills that can be used for monitoring and accountability. The Report Card is not designed for that, but it provides a framework for thinking about how to go about creating summative measures that can be used for accountability.

The task with respect to accountability is to look at each item or related set of items on the Report Card and ask whether there are administrative data that could serve as an indicator of what the Report Card item describes. In several instances, there will be. With respect to self-management, for example, administrative reports of disciplinary infractions, referrals to the principals’ office, and the like could serve.

There are several possibilities for using administrative data for accountability with respect to academic soft skills. For instance, districts or states could produce a measure applicable to the school or grade level of the proportion of students who are underperforming on state or district academic tests in a particular year with respect to the scores that are predicted for those students from a regression formula that includes demographic information and past performance. Administrative records on late arrivals and absences could capture information at a system level that is similar to what teachers are asked to observe for their own students on the Report Card.[xii] The number of items completed on state tests can be a powerful and unobtrusive measure of what teachers address on the Report Card through the question about whether the student completes assigned tasks. [xiii]

Outcomes related to the Report Card category of approaches to learning can captured in administrative data on student participation in extracurricular activities such as clubs, sports, and music.

Critical for measures of soft skills used for accountability is that they not be easily gamed and that they reflect outcomes that are important. The examples given here with respect to administrative data have these characteristics. Scores on the Report Card can be gamed, which is why the tool is not suitable for high-stakes accountability.

What about psychometrics?

Statistical analysis of the scores generated by the Report Card should be descriptive along dimensions that are directly related to practical action. For the classroom teacher, this might take the form, for example, of a roster of students who are flagged because they are consistently receiving low scores on a particular Report Card item or group of items, e.g., having friends. The teacher could then take steps to address this problem. At the level of the principal’s office, the focus might be on percentages of students with problematic scores by category by grade and classroom. This information could inform decisions on the need for extra efforts in some categories. For instance, if a significant proportion of students in the school are reported as not enjoying school, that would be a call to action. Or if a lot of students were reported as having anger control problems it would suggest the need to provide assistance both to teachers with respect to classroom management and to students with respect to self-management.

It would be straightforward for anyone with the relevant skill set to turn the Report Card into an assessment scale. This would involve, for example, factor analyzing a corpus of completed Report Cards to identify the dimensions that account for the most variance and the items that clump together in terms of providing similar information. This could lead to subscales and to the improvement and substitution of items to generate better psychometric values. So scaled, it would be easy to develop teacher and school scores, norms across a school district, and goals for statistically significant improvements over time.

That said, to anyone interested in turning the Report Card into an assessment scale: Please don’t (unless your intent is to use the resulting instrument only for research purposes). As soon as the Report Card is turned into a test in which a teacher learns not that a student is having trouble making friends but rather that the student is at the 18^th percentile for the district in terms of sociability; or not that four particular students in her class are frequently late or absent but rather that the classroom is at the 40^th percentile on the dimension of student timeliness, the function of the Report Card is lost.

The principal psychometric tasks with respect to the Report Card items are face validity and test-retest reliability. Test-retest reliability over short periods of time is the preeminent psychometric question for report card items because the data are not useful if scores that teachers generate for individual students on individual items are unstable during a period of time in which it is unlikely that the student has changed. For example, we would not expect a student who receives a low score from a teacher on October 14 for the ability to listen to and follow teacher directions to receive a high score from the same teacher for the same item on October 28. For items reflecting skills that take time to develop, changes on the Report Card should be gradual rather than sudden.

With regard to face validity, good report card items should capture things that are by general consensus in the school and community important and foundational in their own right. So, for example, if parents and teachers agree that students should have friends or meet deadlines or dog paddle for five minutes and a teacher observes using a report card whether or not they are able to do these things, then the report card has high face validity. In this context, a traditional psychometric concern with predictive validity, e.g., whether answers to assessment items predict other behaviors in other situations, is not primary—having friends is the end goal assessed by the report card, not having friends as a predictor of something else.

Conclusions

We are at the very beginning of understanding what educators should be doing in schools to advance students’ soft skills, how the outcomes of those efforts can be measured, and who should be held responsible for what, and how. The present report focuses on measurement. The recommendation, exemplified through the worked example of The Brookings Soft Skills Report Card, is to use measures of soft skills that are naturally occurring, easily observed, at low levels of abstraction, relevant to the expressed mission and instructional goals of a teacher or school, and useful as feedback at the classroom and parental levels. This recommendation pushes in a very different direction from the current embrace of survey instruments such as the Grit scale that are intended to capture individual differences in abstract student dispositions. Districts are being pitched a lot of metrics by vendors for the ESSA 5th indicator. These metrics are not only of questionable utility for reasons discussed here, but they are expensive. What is proposed here is organic and easier and nearly free.

There are clear implications for schools of the choice of high- vs. low-abstraction measuring tools for soft skills. The thrust of this report is to keep it simple and close to the classroom. Doing so is not incompatible with system-wide measures of soft skills that can be used for monitoring and accountability—this report illustrates how the student outcomes captured in a report card format frequently have parallels in available administrative records that can be used for accountability. These two efforts—classroom tools to be used by teachers and administrative records to be used by administrators—can proceed in parallel. Both differ from and are superior for use in education settings to testing children with psychological instruments that are forms of personality inventories.

[i] https://www.brookings.edu/wp-content/uploads/2016/07/Download-the-paper2.pdf ; https://www.brookings.edu/research/more-on-soft-skills-time-to-flit-the-grit/
[ii] As an example, the California Core Districts include within their Social-Emotional/Culture-Climate Domain soft skills as disparate in abstraction and focus as suspension/expulsion rates and “the ability to take the perspective of an empathize with others from diverse backgrounds and cultures”: http://coredistricts.org/core-index/
[iii] http://leo.stcloudstate.edu/kaleidoscope/volume3/cultureshock.html
[iv] http://scholar.harvard.edu/files/mkraft/files/teaching_for_tomorrows_economy_-_final_public.pdf
[v] http://psycnet.apa.org/psycarticles/2016-06824-001.pdf
[vi] http://www.bls.gov/oes/
[vii] Daniel Kahneman. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.
[viii] http://pbismissouri.org/wp-content/uploads/2016/08/CaldarellaMerrell1997.pdf
[ix] https://consortium.uchicago.edu/sites/default/files/publications/Noncognitive%20Report.pdf
[x] https://www.mindsetkit.org/
[xi] http://www.hamiltonproject.org/assets/legacy/files/downloads_and_links/v10_THP_LudwigDiscPaper.pdf
[xii] http://www.hamiltonproject.org/papers/lessons_for_broadening_school_accountability_under_the_every_student_succee
[xiii] http://www.corwin.com/books/Book245000

      

 

 

Two aspects of a Trump administration create the prospect of significant disruption in the way things have been.

First is unified control of the Senate, House, and the presidency by the Republican Party. If, for example, Republicans in Congress want to restore and expand the voucher program that provides publicly funded scholarships for students to attend private schools in Washington, D.C., which was killed by President Obama in 2012, they can have it. Or, at the very least, if they want it but don’t get it, they will have nobody to blame but themselves.

Second is that while Trump the candidate didn’t have a lot to say about education, his few statements and the campaign’s positions have a strong populist tone. Included are his early and emphatic opposition to the Common Core State Standards (thought of by many as an elitist takeover of the nation’s school curriculum); commitment to making school choice available for all students from disadvantaged backgrounds; concern with high levels of college debt; comments, no matter how unartful, on the plight of urban public schools; and attention to child care support, which leaves decisions on providers to parents and stands as a clear alternative to the top-down universal pre-K programs favored by his opponent.

What this portends for actual federal law, approach, and regulation is guesswork, made chancier than might otherwise be the case by the lack of what in previous new administrations has been a predictable cast of characters with known positions from previous administrations likely to fill domestic policy and U.S. Department of Education leadership positions. A Trump administration will surely have some such players, but is likely as well to bring in people who are newcomers to federal government with backgrounds that are not traditional to education management and policy. Their positions on education, abilities to manage and lead within the federal bureaucracy, and impact on policies and practice are wild cards.

With that caution, the following are policies that are likely to be pursued by President Trump, with parenthetical comments on research on their effectiveness:

• Common Core – getting rid of it is largely taken care of by provisions of the new ESSA, but there will be a search-and-destroy mission for any remaining sources of federal support for common state standards and assessments.

(State standards have little impact on student achievement so nothing lost here[i], particularly since states can and will still adopt on their own);

• School choice – a strong effort to provide additional federal funds to states that allow funding to follow students to their public or private school of choice. It would be likely be funded by redirecting of a portion of current Title I dollars (federal dollars targeted to poor students) from states that don’t allow choice.

(Parents prefer schools of choice and competition among schools for students can increase productivity and promote innovation so a good thing if done right[ii]);

• Charter schools – an effort to reduce or eliminate current state caps on charter schools, and to increase their federal support – something the teachers’ unions have been fighting for years.

(Urban charter schools as a class are much more effective in raising student achievement than the traditional public schools with which they compete whereas suburban charter schools are not, so the impact depends on which charter schools are favored[iii]);

• D.C. Opportunity Scholarship Program – this pilot program would be reauthorized, funded, and expanded.

(The program substantially enhances high school graduation rates and increases parental satisfaction at lower cost per student than education in the regular public schools of the District of Columbia[iv]);

• Higher education – fiddle with loan interest rates and repayment periods, seek ways to reintroduce a private market for student loans; use the tax code to incentivize institutions with large endowments to lower tuition costs; and create a friendlier environment for for-profit providers.

(Not much new here, problems with previous similar efforts, and doesn’t address core of the problem[v]);

• Early childhood – get the federal government out of the universal pre-K business; shift existing early childhood expenditures, including those for Head Start, to tax credits and cash transfers that flow directly to the family; use the tax code to incentivize employer-supported family leave.

(Providing cash transfers to families may be more beneficial for children than providing government preschool programs so this may be a win[vi]);

• K-12 – see school choice and relevant research above;

• U.S. Department of Education’s leadership and structure – trim programs and staffing; eliminate the Office of Civil Rights and transfer its functions to the Justice Department; restore the bully pulpit and reduce the use of regulation as the Secretary’s principal lever for affecting state and local policy; and create collaborative relationship with relevant House and Senate leadership.

(No direct research evidence);

• Funding – shifts in priorities but no increases (see priorities above for relevant research),

None of the education priorities and programs signaled by Trump and his campaign is unreasonable, many have a history and good research backing them. Higher education is the murkiest area. Details and implementation will matter a lot but on the surface there is nothing that hasn’t previously been part of the public discussion of education policy or that is scarily radical.

[i] https://www.brookings.edu/research/the-2012-brown-center-report-on-american-education/
[ii] https://www.brookings.edu/wp-content/uploads/2016/02/brown_20160211_ecci_report.pdf
[iii] https://www.brookings.edu/research/massachusetts-charter-cap-holds-back-disadvantaged-students/
[iv] http://ies.ed.gov/ncee/pubs/20104018/pdf/20104032.pdf
[v] https://www.brookings.edu/research/the-trouble-with-student-loans-low-earnings-not-high-debt/
[vi] https://www.brookings.edu/wp-content/uploads/2016/07/Family-support3.pdf

      

 

 

“Everybody who looks at the evidence, and their intuitions, believes that soft skills are important.  I don’t think there’s any doubt that they can predict later outcomes – given a choice between two people with the same hard-skill set and one that’s easy to get along with, easy to work with, who are you going to choose? So these things are very important,” Whitehurst explains. “The question is what can schools in general be held accountable for in this area, and what can they themselves learn about how to impart these skills in ways that are efficient and effective?”

“Over the last twenty years, the labor market payoff to having math skills has gone up and the labor market payoff to having noncognitive skills has also gone up, but the increase in the payoff to having noncognitive skills has gone up by much more,” Schanzenbach says.

Related Links:

Seven facts on noncognitive skills from education to the labor market

Hard thinking on soft skills

Fourteen economic facts on education and economic opportunity

This policy would help poor kids more than universal pre-K does

More on soft skills: Time to Flit the grit

With thanks to audio producer Gaston Reboredo, Vanessa Sauter, Basseem Maleki, Fred Dews, and Richard Fawal.

Subscribe to Brookings podcasts here or on iTunes, and send feedback email to intersections@brookings.edu.

Intersections is part of the Brookings Podcast Network.

      

 

 

Examining empirical research that measures the effect of interventions to provide low-income and minority students with access to higher-performing and more racially heterogeneous schools, the authors conclude that positive impacts on student achievement are driven by school quality.

Looking at trends in racial segregation over time, the authors find that changes in schools’ racial makeup in recent decades have been driven largely by the increase in the Hispanic and Asian American populations. While black and white students have become much more likely to share classrooms with Hispanics, blacks and whites are not more likely to share classrooms with each other than they were decades ago—the original intent of the Brown v. Board of Education landmark Supreme Court ruling that desegregated schools 60 years ago.

But while many school districts are less racially segregated, school segregation by family income has increased since 1990 both within and between school districts. The authors note that race and economic status are of course highly correlated, with black students four times as likely to be in a high-poverty school as a low-poverty one; for whites the ratio is the other way round.

The authors also focus on how charter school systems influence segregation, finding that while national averages show that charters serve almost equal numbers of white, black, and Hispanic students, individual charter schools are generally more racially segregated than traditional public schools that serve the same geographical area.  In urban areas, the charter schools into which black students transfer from traditional public schools are substantially more segregated racially than the traditional public schools from which they exit.   Despite that segregation, studies of urban charter schools present compelling evidence that high quality urban charters can overcome the effects of school segregation on student achievement.

The authors conclude, “The U.S. is an increasingly diverse nation, but remains a highly segregated one. Our schools reflect both our separateness and our inequality…The desire for more integrated schools is understandable. But it is helpful to be as clear as possible about what lies behind that desire. If the main objective is to narrow racial achievement gaps, we need to understand to what extent, and in what way, segregation influences those gaps. The weight of evidence suggests that, at least in the context of the education system, the worse educational outcomes for minority students are the result not of the racial composition of their schools, but the economic backgrounds of their fellow students, and the quality of the school itself—both of which are strongly correlated with race.”

To learn more, read the full paper here.

      

 

 

For a half-century, our nation has focused on school-readiness programs such as Head Start as the best way to help low-income children escape the cycle of poverty. The idea is to level the playing field in cognitive and social skills by the time these children enter kindergarten so that they can keep pace with their more advantaged peers as they progress through school. In the next decade, we will spend $100 billion at the federal level just on Head Start, and all but a few states are funding their own pre-K programs.

Unfortunately, children who attend Head Start do no better in school than equivalent children who do not. Even the best pre-K programs’ positive impacts fade away in a couple of years, and some early-childhood programs actually leave children worse off than if they hadn’t participated at all.

Yet, early-childhood programs continue to get large amounts of taxpayer dollars, evidence be damned — something that is true of so many programs in Washington. Well-intentioned conventional wisdom wins out. Because low-income and minority kids enter school far behind their higher-income counterparts and don’t catch up, the theory of intervening early seems like common sense.

Which is why lately there has been a push by politicians to go one step further and create preschool programs for all, regardless of income. New York Mayor Bill de Blasio recently established such a program; Boston and the District are implementing them. Democratic presidential candidates Hillary Clinton and Bernie Sanders campaigned on plans to make universal pre-K a national priority. President Obama has proposed a federal-state partnership, called Preschool for All, that would leave taxpayers with a bill on the same order of magnitude as that for Head Start. The argument for this approach draws on the secret to the success of Social Security: The social compact (and the willingness to pay for it) works better if a program provides an entitlement for everyone.

But if our goal is to help poor families, is universal pre-K really the best, most efficient way? The answer is no.

I have compared the effects of direct income transfers to low-income families (such as the earned-income tax credit, or EITC) with programs designed to increase school readiness (universal preschool and Head Start). It turns out that putting money directly into the pockets of low-income parents, as many other countries do, produces substantially larger gains in children’s school achievement per dollar of expenditure than does a year of preschool or participation in Head Start. The results throw water on the conventional wisdom.

The results show that while the EITC isn’t specifically designed to boost academic achievement, it does so anyway — and not just for younger kids. The EITC is also a bargain compared with the programs specifically designed to help poor kids academically.

Specifically, each of four evaluations of U.S. family income support programs found substantially larger test score increases per $1,000 of public expenditure than resulted from programs specifically aimed at improving educational outcomes by focusing on school readiness. In particular, neither pre-K nor Head Start provided the same amount of improvement as the family support programs did. Other studies of the EITC also show impacts on even later outcomes — such as college enrollment and earned income.

The current annual federal expenditure on the EITC is about $65 billion. During the 2013 tax year, the average EITC was $3,074 for a family with children. In contrast, Head Start runs about $8,000 per child. Boston’s and the District’s pre-K programs run more than $16,000 per student. Spending less (EITC) is actually more effective than spending more (Head Start, universal pre-K). It’s a win-win.

Former senator Daniel Patrick Moynihan likened government bureaucracies dispensing social services to the poor as “feeding the sparrows by feeding the horses.” The school readiness option feeds the horses. Perhaps it is time to rethink our paradigm for supporting poor families. Let’s give them what they desperately need — more money — and let them decide how to spend it on the early care and education of their children.

      

 

 

Two recent research studies add strong support to those conclusions. Both focus on “grit”, the soft skill popularized by Angela Duckworth and a favorite target for intervention and measurement in education reforms that target soft skills.

Grit, as conceptualized and measured by Duckworth and colleagues, consists of two connected constructs: perseverance in the face of difficulty and a passionate, enduring commitment to particular goals. In a foundational and influential study,[i] Duckworth reported that West Point cadets who scored higher on a paper-and-pencil grit scale were more likely to complete the challenging Beast Barracks program in their first two weeks than those who scored lower. Those scoring higher on grit would have been more likely to self-report on the grit scale that, for example, they had overcome setbacks to conquer an important challenge (perseverance) and less likely to report that their interests change from year to year (enduring commitments). Informally, we can imagine the extremely gritty cadets as ones who had previously always finished what they started (perseverance) and who may have had a long-term goal of being an officer in the military (consistency of interest). And we can imagine that such cadets in the face of a series of physically and mentally challenging West Point training exercises might be more likely to stick to it than cadets with lower levels of self-reported perseverance and consistency of interests.

Accepting these results at face value, there are three important questions that arise with respect to education programs intended to enhance soft skills such as grit. The first is the extent to which grit is a malleable characteristic that schools can be reasonably expected to influence and for which they can be held accountable. The second is the extent to which grit influences the other outcomes for which schools are clearly held accountable, most particularly academic achievement. The third is the extent to which grit is unique vs. just another way of describing a human personality characteristic that has been recognized and studied for most of the history of scientific psychology.

A brief overview of the two recent studies is in order before addressing the questions. Both studies are published in the Journal of Personality and Social Psychology.

The first research report (hereafter the “twin study”), by Rimfeld et al.,[ii] is a study of 2,321 twin pairs in Great Britain. The subjects were administered as 16-year-olds Duckworth’s Grit-S scale, along with an assessment of the Big Five personality traits (openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism). Grit and Big Five scores were used to predict the participants’ grades on the achievement exams that are administered UK-wide at the end of compulsory education. By comparing the correlations of Grit-S and Big Five scores between identical twins (who share the same family environment and 100 percent of their genes) and fraternal twins (who share the same family environment but only 50 percent of their genes) estimates of genetic vs. shared environment contributions can be derived.

The second study (hereafter the “meta-analysis”), by Credé et al.,[iii] is a systematic review of data from all the published studies that could be found in which participants who were at least of middle school age and in which correlations were reported or could be calculated between scores on any of Duckworth’s grit scales and other variables. There were 73 such studies involving roughly 67,000 participants. The investigators addressed questions such as the correlation between grit and achievement and the relationship between grit and other measures by combining relevant data from all the studies. Meta-analysis, when done well, can address issues that arise when trying to generalize results from one or two studies that may not be representative.

Is grit a malleable characteristic of students that schools can be held accountable for influencing? No. The twin study finds that the heritability of the perseverance component of grit is 37 percent. In other words, identical twins are substantially more similar on grit than fraternal twins. Importantly for the school malleability issue, the contribution of the shared environment to grit is essentially zero. In other words, children who are raised in the same family and attend the same schools are no more similar on grit than children who do not share their family and school environments (once the contribution of shared genes is removed). As I suggested in my previous report, it seems unreasonable to hold schools accountable for influencing something over which evidence suggests neither the family environment nor schools has any meaningful influence.

Does grit influence other outcomes for which schools are clearly held accountable, most particularly academic achievement? Not much. In the twin study, grit scores account for only 2 percent of the variance in achievement scores when used as a single predictor and account for hardly anything at all when Big Five personality scores are entered into the equation. In the meta-analysis, grit accounts for about 3 percent of the variance in academic achievement outcomes when it is examined as a single predictor. As the authors of the meta-analysis point out, there are many known, malleable predictors of achievement test scores that have much higher associations with achievement than measures of grit, e.g., study skills, test anxiety, and learning strategies. Yet school districts and schools that are investing in interventions for and measures of grit seem convinced that academic outcomes for their students will be substantially improved. That is a poor bet based on the evidence reported here.

Is grit unique or just another way of describing a human personality characteristic that has been recognized and studied for most of the history of scientific psychology? Grit is old wine in new bottles. In the twin study, the perseverance component of grit is very highly correlated (.53) with the Big Five trait of conscientiousness. This simple correlation is not corrected for unreliability of the measures of grit and Big Five traits (a measure that has a test-retest reliability of .80 cannot correlate with another measure at a higher level than .80—correlations can be corrected for this attenuation). The meta-analysis used reliability-corrected correlations and found a correlation between grit and conscientiousness of .84. In the twin study, the genetic correlation between grit and conscientiousness was .86, which indicates the high extent to which the same genes are affecting the two constructs.

Another way to approach this question is to determine the incremental contribution of grit to outcome predictions when other measures of related constructs are entered into the prediction equation. As I noted previously, in the twin study, nothing remains of the correlation between grit and academic achievement once Big Five personality measures are included. In the meta-analysis, grit explains no variance in either overall academic performance or high school GPA after controlling for conscientiousness.

When I was a child growing up in the South, my family would gather on my grandparents’ front porch on warm summer evenings to do what southern families were pretty good at back then—gossiping and telling stories. The porch door would swing open as people came and went and there were surely some holes in the screens. Pretty soon the call would go out to get the Flit, which was the commercial name on the side of the pump sprayer of insecticide we used on the bugs that were starting to bother us.

Grit is a personality trait, not a skill to be taught. It is highly heritable. We have no validated interventions for teaching it that can be used by schools. And grit is just conscientiousness, about which we know a lot, by another name.

The focus on grit by schools has huge opportunity costs—the money, time, and reform energy spent on it would be much better invested in things that we know have an impact. Within the domain of soft skills, schools should encourage and reward students for persistence and hard work rather than trying to increase their grit. And instead of trying to impact students’ conscientiousness, they should provide task-specific training on how to manage time and complete assignments, and meaningful consequences for doing so. Measuring grit and using it as a performance measure for schools has been rejected even by Duckworth, the creator of the grit construct.[iv] It is, I maintain, time to turn the Flit on grit.

      

 

 

Unfortunately, children who attend Head Start do no better in school than equivalent children who do not. Even the best pre-K programs’ positive impacts fade away in a couple of years, and some early childhood programs actually leave children worse off than if they hadn’t participated at all.

Yet, early childhood programs continue to get large amounts of taxpayer dollars, evidence be damned – which is true of so many programs in Washington. Conventional wisdom wins out, well-intentioned – let’s do what we can to help poor kids by intervening educationally when they are as young as possible. Because low-income and minority kids enter school far behind their higher income counterparts and don’t catch up –the theory of intervening early seems like common sense.

Which is why lately there has been a push by politicians to go one step further and create preschool programs for all, regardless of income. New York Mayor Bill de Blasio has recently established such a program; Boston and DC are already implementing one. Democratic presidential candidates Hillary Clinton and Bernie Sanders have campaigned on a plan to make universal pre-K a national priority. President Obama has proposed a federal-state partnership, called Preschool for All, that would cost more than Head Start. Think of it like Social Security: the social compact (and willingness to pay for it) only works if it is an entitlement for everyone.

But if our goal is to help poor families, is this the really best way and an efficient use of taxpayer dollars? No.

In a recent study, I compare the effects of direct income transfers to low-income families (such as the Earned Income Tax Credit or EITC) with programs designed to increase school readiness (universal preschool and Head Start). It turns out that putting money directly in the pockets of low-income parents, as other countries do, produces substantially larger gains in children’s school achievement per dollar of expenditure than a year of preschool or participation in Head Start or some other more expensive education reforms like class size reduction. The results throw water on the conventional wisdom.

It turns out that while the EITC isn’t specifically designed to help boost academic achievement, it does, and not just for the younger kids, but even for the older ones in the same family. Not only that, the EITC is a bargain compared to the programs specifically designed to help poor kids academically. Each of four evaluations of U.S. family income support programs found substantially larger test score increases per $1,000 of public expenditure than programs specifically aimed at improving educational outcomes by focusing on school readiness: neither pre-K (two to four years later) or Head Start (four years later) showed the same improvement as the family support programs. Studies of the EITC also show impacts on even later outcomes — such as college enrollment and earned income.

The present annual federal expenditure on EITC is about $65 billion. During the 2013 tax year, the average EITC was $3,074 for a family with children. In contrast, Head Start runs about $8,000 per child. Boston and Washington, D.C.’s pre-K programs run about $18,000 per student. It this case, spending less (EITC) is actually more effective than spending more (Head Start, universal pre-K). It’s a win-win.

But there is no need to pit the policy choice of family support programs against school readiness – it’s a matter of emphasis rather than mutual exclusivity. In other words, expenditures that have a primary goal of strengthening and supporting families in carrying out their responsibilities as parents need not and should not ignore children’s development, including children’s cognitive and social emotional readiness for school. But in a family support model, school readiness is one branch on the tree, not the trunk. Such a support model could take many forms, either at the state level or on a federal level, including giving an annual scholarship for every low-income child under five to be spent by parents on the child care and early education services they want and need, as presidential candidate Jeb Bush proposed.

Senator Patrick Moynihan likened government bureaucracies dispensing social services to the poor as “feeding the sparrows by feeding the horses.” The school readiness option feeds the horses. In light of the evidence I’ve reviewed, perhaps it is time rethink our paradigm for supporting poor families. Let’s give them what they desperately need, more money, and let them decide how to spend it on the early care of education of their children.

Editor’s note: This piece originally appeared in The Washington Post.

      

 

 

In that inaugural Evidence Speaks post, I asked what remains a very important question: What role will evidence play in the policies that candidates put forward in the 2016 election? Policymakers and presidential candidates use research findings to support their positions, but their evidence is frequently cherry picked and of low quality. Consumers of political and policy claims are typically not in a position to judge the flaws of the evidence that is cited, much less to determine whether there is contradictory evidence that has been omitted.

I posit that part of the problem is the lack of trusted translation mechanisms capable of distributing the research to those who need it in a consumable, accessible format. As the editor of Evidence Speaks, I seek to remedy that problem.

To-date, Evidence Speaks has published 37 memos and reports, by leading experts at Brookings and elsewhere, on everything from universal preschool to Title I spending to school vouchers in Louisiana.

We post a new piece to Evidence speaks every Thursday morning, and you can find them all here. In the meantime, here are some major findings from the past few months.

1. Student loans aren’t pushing down homeownership rates

For several years, leading economic thinkers such as Larry Summers and Joseph Stiglitz have proposed that high levels of student debt are creating a drag on the housing market.

New Evidence Speaks research from Nonresident Senior Fellow Susan Dynarski challenges that assumption, finding that student debt isn’t the reason homeownership rates are dropping. Rather, the main division between the home ownership “haves” and “have-nots” is their education level—not their debt.

Dynarski finds that while those without a college degree are more likely to own a home at an earlier age than those who went to college and accrued debt, the college-educated catch up fast. By 27, those with a college degree overtake those without degrees in homeownership. By 35, the gap in homeownership between those with and without a college education is about 14 percent.

“The college-educated—even those with student debt—are winners in our economy,” Dynarski concludes.

2. Free college proposals like Bernie Sanders’ would help the rich more than the poor

On the campaign trail, Bernie Sanders’ plan to make tuition free at public colleges and universities has received a lot of attention. It caught the eye of Evidence Speaks contributor and Urban Institute Senior Fellow Matthew Chingos, who sought to uncover who would really benefit from the plan.

Chingos’ analysis of the free college proposal found that families from the top half of the income distribution would receive 24 percent more in dollar value than students from the lower half of the income distribution, largely because the wealthy tend to attend more expensive institutions.

Making tuition free, Chingos notes, wouldn’t cover the other costs of going to college, such as living expenses, that are often larger than the costs of tuition and fees for most students attending in-state schools. These annual out-of-pocket college costs would still leave families from the bottom half of the income distribution with nearly $18 billion that would not be covered by existing federal, state, and institutional grant programs.

Chingos writes: “It is important to emphasize that this analysis is only a starting point for considering the potential distributional consequences of making college free. The most significant limitation of this analysis is that it does not consider the likely impacts on enrollment of eliminating tuition and fees…but the ultimate design of proposals to change how students and taxpayers pay for higher education should carefully consider their likely distributional consequences and the tradeoffs between targeted and universal programs.”

3. To improve school achievement, cash transfers are a better bet than preschool or Head Start

For decades, public policy has focused on improving the prospects of children from low-income families through early education programs, like Head Start, that aim to improve school readiness. The federal government alone spends more than $22 billion per year on early childhood programs, with the largest expenditures being for Head Start and the Child Care and Development Block Grant.

But are early education programs the most effective interventions for improving school achievement? My own research raises the possibility that they’re not.

In a recent Evidence Speaks piece, I compared the impact of pre-K for four-year-olds or Head Start, with the impact of programs designed to support low-income families, for example the Earned Income Tax Credit (EITC).

The family income support programs I examined increased test scores several times more per $1,000 of public expenditure than programs specifically aimed at improving educational outcomes. Neither pre-K (two to four years later) or Head Start (four years later) showed the same improvement as the family support programs. Studies of the EITC also show impacts on even later educational outcomes such as college enrollment.

The findings suggest that a family support model of early childhood programs in the U.S., which could take many forms at the state or federal level, would be more effective at improving education outcomes than the current emphasis on Head Start. One example, proposed by the Jeb Bush presidential campaign, would give an annual scholarship for every low-income child under five to be spent by parents on the child care and early education services they want and need.

More evidence-based research on education policy

The three reports above are only a small sample of the research published through Evidence Speaks. To learn more, visit the Evidence Speaks homepage or sign up to receive an RSS email each time we publish a new report.

      

 

 

In the United States, public policy and expenditure intended to improve the prospects of children from low-income families have focused on better preparing children for school through Head Start and universal pre-K. This school readiness approach differs from the dominant model of public support for early care and learning in Northern Europe, which places more emphasis on supporting families. It also differs from other government programs in the U.S., such as the Earned Income Tax Credit, that support low-income parents of young children by boosting income. Empirical comparisons of the impact on school achievement of boosting family income vs. providing free pre-K for four-year-olds, summarized in this paper, suggest that supporting family income is a more cost effective expenditure. A policy midpoint between more money for families vs. more money for pre-K is more money for families to spend on their young children. All these policy options should be on the table and subject to test as the nation moves towards increased attention to and investment in the early years. 

The dominant focus on school readiness

Federal spending on programs to support children’s early learning and care, including through Head Start and the Child Care and Development Block Grant program, amounts to roughly $22 billion a year. Several billion more is spent by states from their own tax revenues. Some large urban school districts, including New York City and Boston, are spending their own funds on universal pre-K programs. The Obama administration has proposed in its last two budgets the addition of $12 billion in annual federal and state expenditures for a federal/state partnership called Preschool for All.

Public expenditures on early childhood programs are nearly always justified as investments that will eliminate socioeconomic and racial gaps in school readiness and elevate subsequent student achievement and life success.

The purpose of Head Start as defined in the Head Start Act:

“to promote the school readiness of low-income children by enhancing their cognitive, social, and emotional development”[i]

President Obama’s rationale for new spending on early childhood programs:

“Every dollar we invest in high-quality early childhood education can save more than seven dollars later on—by boosting graduation rates, reducing teen pregnancy, even reducing violent crime. In states that make it a priority to educate our youngest children, like Georgia or Oklahoma, studies show students grow up more likely to read and do math at grade level, graduate high school, hold a job, form more stable families of their own. We know this works. So let’s do what works and make sure none of our children start the race of life already behind. Let’s give our kids that chance.”[ii]

New York Mayor Bill de Blasio on the purpose of his administration’s signature universal preschool initiative:

“Full-day pre-K is the cornerstone of the de Blasio administration’s plan to transform public education in New York City…high-quality pre-K represents change at a scale that will raise achievement and reduce inequality across all communities.”[iii]

Presidential candidate Hillary Clinton on preschool:

“One of the best investments we can make as a nation is to give our kids the ingredients they need to develop in the first five years of life. We will help bring together the tools that will give children the chance to succeed by the time they’re five, so that when those kids get to school, they’re able to compete, they are more able to pursue their own dreams.” [iv]

There are consequences, intended and not, of defining the purpose of spending on early childhood almost entirely in terms of school readiness. Among them are a focus within preschool programs on teaching pre-academic skills; the conceptualization of the role of the adults who provide center-based care as that of a teacher; a bias towards delivering pre-K services through school districts; a press towards common standards and curriculum across pre-K providers; accountability regimens that are tied to children’s performance on measures that correlate with later school success; disproportionate spending on four-year-olds as opposed to younger children; and marginalization of the family’s responsibility. In general, just as kindergarten became the new first grade in a previous era, we now see pre-K becoming the new kindergarten with all that implies for curriculum, staffing, funding, and aegis.

Family support as an alternative

There are other models of public support for young children but they have been lost in the preoccupation in the U.S. with school readiness and its corollary: using center-based programs for four-year-olds to make up for what are viewed as deficiencies in parenting among low-income and minority parents that leave too many children behind at the start of formal schooling.

Consider, in contrast, the mission of the Danish childcare system, which is similar to most of the state systems in Northern Europe:

“Danish child and family policy is based on the overall principle that the family is the foundation of a child’s upbringing and that the living conditions of children are mainly the responsibility of their parents. Public authorities have an overall responsibility for providing a good social framework and for providing the best possible conditions for families with children. In addition, public authorities must protect children and young people against abuse and neglect, and they must offer advice and guidance to parents so that the parents will be able to meet their parental obligations and responsibilities.”[v]

This is a fundamentally different framing of the state’s responsibility for young children than illustrated in the quotes above from U.S. law and high officials. It too has consequences, intended and not. Among them are the goal of providing care based on the needs of parents, which means proportionate spending across the early childhood years; the availability of care that is responsive to the working hours of parents; and a requirement that initiatives taken in relation to individual children must be agreed to by the parents (which reinforces the importance in the Danish model of parents having the overall responsibility for the development of their children).

One can look elsewhere in Northern Europe for interesting variations on the general model outlined for Denmark. In Finland, for example, working parents of young children can choose from a variety of providers. They can also opt to receive a financial subsidy that allows them to reduce their work hours in order to be home more with their child. They can also take unpaid leave.[vi] Perhaps as a result, Finland has one of the lowest rates in Northern Europe of enrollment of children under four years of age in center-based care—it is less than half of that in Denmark and much lower even than in the U.S.[vii]   

My point is not that the U.S. should adopt European models of early child care. Rather, it is that there are other ways to spend as much or more public funds on early education and child care than under the dominant school readiness model. Those policy counterfactuals should be on the table and subjected to thoughtful consideration as the nation is poised to increase attention to and investment in early childhood programs.

One way to broadly categorize the policy choice is family support vs. school readiness. This a matter of emphasis rather than mutual exclusivity. In other words, expenditures that have a primary goal of strengthening and supporting families in carrying out their responsibilities as parents need not and should not ignore children’s development, including children’s cognitive and social-emotional readiness for school. But in a family support model school readiness is one branch on the tree, not the trunk.

What would a family support model of early childhood programs look like in the U.S.? It could take many forms. In fact, the details should differ if the 50 individual states rather than the federal government were in control. One possible model for the federal government, called Early Learning Family Grants, is outlined in testimony I gave to Congress.[viii] Governor Jeb Bush proposed something similar in his presidential campaign in the form of an annual scholarship for every low-income child under five.[ix] To flesh this out, under such family support models:

Individual families would:

• receive financial subsidies on a sliding scale based on family income to support the costs of raising young children from birth through age five;
• receive additional targeted support for special needs, e.g., children with significant disabilities; and
• choose what kind of out-of-home care they need and when they need it.

The Federal government would:

• provide the basic subsidy and financial support to states for carrying out their responsibilities to low income families; and
• provide technical assistance.

State governments would:

• regulate out-of-home providers to insure basic levels of safety and performance;
• help parents select a provider by making available results from consumer satisfaction surveys and other information on the performance of individual providers; and
• supplement the federal subsidy.

Various governmental entities, non-profits, and others would:

• work to align supply with demand;
• support the training needs of child care workers and the management needs of providers;
• provide linkages between providers and guidance on how providers and parents can promote the successful transition of children into elementary school; and
• support parents in meeting their obligations and providing a caring environment for their children.

The devil is in the details of each of these features, and there is every reason to believe that the impacts of a family support policy could shift in size and sign depending on those details. For example, a subsidy that is too small in dollar value to cover market rates for out-of-home care, or too delayed in receipt to be available when the bill is due, or too uncertain in prospect based on unpredictable short-term changes in earned income could leave children in care settings that do harm and could increase levels of stress for parents. There is some evidence that the federal Child Care and Development Block Program does exactly that.[x] Likewise, a program that is not cleverly structured could entice low-income parents to leave the labor market entirely during long stretches of their young children’s lives, making reentry difficult after the children age out of the family subsidy. Annual subsidies that start at birth and don’t rollover could incentivize parents to put their child in center-based care at an earlier age than necessary or desirable, and so on through each of the bullets above.

There is, as well, variety in the details of school readiness programs. Head Start is not the same as Boston’s pre-K program, although both serve the same age group with related goals. Likewise, Florida’s universal pre-K program for four-year-olds differs in many ways from Oklahoma’s. And nearly all current publicly funded pre-K programs differ in important respects from earlier pre-K programs whose short and longer term impacts have been estimated and the results of which have been extrapolated to justify claims of return on investment in current programs.

Estimating the return on investment in family support vs. school readiness

This squishiness in what, specifically, we’re talking about brings peril to any attempt to provide an empirical comparison of the returns on public investment in family support vs. school readiness. But there is sufficient information available to support a preliminary estimate of the return on investment in children of public spending on center-based programs for four-year-olds vs. family support. I attempt that here in an effort to foster greater attention to the subject by both researchers and policymakers.

Consider what follows a back-of-the-envelope exercise in which I have avoided the heavy lifting that would be necessary to generate point estimates that are defensible. For example, I have not adjusted expenditure levels across studies to current dollars, or scaled the outcomes across studies based on the lag in years between a child experiencing the input being evaluated and the outcomes being assessed, or tried to restrict data on family support to only those inputs a child would experience in early childhood. It would be desirable for someone to do all these things, which in some cases would require data not now available. I do not think that I would need to change the general conclusions I draw in the present report if the approximations I provide were cleaned up in the future. But caveat emptor.

I use child achievement outcomes to generate effect estimates rather than measures of family functioning such as the labor market participation of parents. A priori I would expect that this advantages the school readiness approach since programs that operate under that umbrella are designed to enhance child achievement whereas that is an ancillary goal of family support.

Before getting to the meat of the matter, which is examining the impact of family support and school readiness expenditures in the U.S., consider an important and well-designed study that evaluated the impact in Norway of receipt by lower income parents of a subsidy for child care.[xi]

Because the subsidy was received after parents had already decided to send their child to center-based care and pay their required share of the tuition, it ended up increasing the family’s disposable income by lowering their tuition bill rather than affecting their use of child care. Functionally, this made the program a family support investment rather than a preschool program per se, but one conditioned on the family making an expenditure on their child.

The researchers took advantage of a strict income discontinuity in eligibility to create a treatment and control group of families that just made or missed the cutoff point for the subsidy. They used variation across municipalities in the income cutoff for eligibility to examine differences in impacts by family income. They estimated the long-term outcomes on the children in the two groups using results of national examinations taken by the children when they were in middle school. They also examined impacts on the achievement of older siblings of the treatment and control children, and impacts on labor market participation and family income of parents. Families in the treatment group had a one year boost of about 10 percent of gross family income, or about $1,500 in U.S. dollars.

There were strong positive effects on grade point average and oral exam scores in middle school. Older siblings in the families receiving the supplement did better as well. And the parents receiving the supplement went on to earn more in the years following the supplement. The effect size for middle school grade point average for children who qualified for the subsidy as five-year-olds was .30, a very large impact. The effect was largest in municipalities in which the threshold for eligibility was at lower levels of income, e.g., receiving or failing to receive the $1,500 subsidy had much larger consequences for a family with a gross income of $15,000 than for a family with a gross income of $30,000.

This study shows that a modest boost in income for low-income families during one year of the lives of their preschoolers had substantial long-term impacts, including enhancing school performance in middle school. These effects were not mediated by preschool attendance. Instead, the additional income appears to have allowed the parents to function better and thereby increase their own and their children’s human capital. Of course, Norway differs from the U.S. on many dimensions that may influence the impact of a program like the one that was studied.  

For evidence from the U.S. on the impact of a family support model I turn to research on various welfare reforms and on the Earned Income Tax Credit (EITC), a federal program that provides a tax refund to lower-income workers based on amount of earned income and number of children. The present annual federal expenditure on EITC is about $65 billion, which amounts to roughly $2,200 per recipient. About half the states provide their own EITC supplement, which varies in generosity.

Studies of the EITC that have examined impacts on child outcomes typically take advantage of nonlinear variation across time in the generosity of the benefit.[xii] For example, in 1993 the amount of the federal EITC increased substantially for families with more children, allowing comparison of differences in child outcomes for larger families before and after that change. Another approach uses experiments on welfare reform rather than the EITC and takes advantage of variation in the earnings supplements that were provided to families in the treatment conditions.

I focus on methodological strong studies that examine student achievement in school. These include three studies of the impact of the EITC[xiii] and one synthesis[xiv] of several experimental studies of welfare reforms in the 1990s. I compare the impacts of increased family support on student test scores from these four studies with the impacts of pre-K school readiness interventions using, first, a synthesis of findings from 67 pre-K evaluations of test outcomes 2-4 years after pre-K,[xv] and, second, the follow-up findings from the Head Start Impact Study[xvi] for 3^rd graders. As a further point of comparison I provide results from a study[xvii] of Project STAR that examined the impact on middle school test scores of exposure to class size reduction in the early grades. Additional detail on the studies is provided in Table 1 in the appendix to this report.

It is conventional in comparing impacts across studies in which the intervention can be expressed as a dollar expenditure to standardize the size of the expenditure in a round number that is related, in general, to the scale of actual expenditure. The size of the difference in outcomes is then adjusted proportionally. This results in the ability to report what the impact of, for example, $1,000 in additional annual family income was on academic achievement in a particular study even though the actual increase in expenditure was only in the ballpark of $1,000. For instance, if the actual difference in the EITC between the treatment and control families was $1,500 and the actual difference in test scores between children in those two groups resulted in an effect size of .12, the findings could be described by downscaling both the intervention and outcome as “an increase in the EITC of $1,000 produced an effect size of .08.” The same approach allows us to compare interventions that are simply financial supports for families, such as the EITC, with substantive programs to which a cost per participant can be attached. For example, the impact of a school readiness program such as Head Start can be compared with the EITC by expressing the return on investment of both as the effect produced by $1,000 of expenditure.

I report child outcomes as an “effect size,” which places an outcome difference between a treatment and comparison group on a scale based on the standard deviation of the outcome (a measure of the spread of scores of the individual cases). Thus a mean difference of 2 points between a treatment and control group on a test that has a standard deviation of 10 would be described as an effect size of .20 (2/10), whereas that same 2 point difference would represent an effect size of .10 on a test with a standard deviation of 20 (2/20).

The figure below presents the effect size on tests of cognitive skills for low-income children of $1,000 of expenditure. It uses data from the studies described above and in Table 1 of the appendix. In the case of the EITC studies the effect is for a single year of a $1,000 of expenditure. For the welfare experiments the effect is for $1,000 per year of expenditure for 2-5 years. For the school readiness interventions the expenditure is derived from the cost per participant for a single program year. Cost data are not available from the reported studies so I use a conservative estimate of $5,000 per year for pre-K and $7,000 per year for Head Start. In all cases, cognitive outcomes are measured during the elementary school years or the elementary and middle school years. The first author of each of the underlying studies is listed below the horizontal axis in the figure.

The results illustrated in the graph suggest that family support in the form of putting more money in the pockets of low-income parents produces substantially larger gains in children’s school achievement per dollar of expenditure than a year of preschool, participation in Head Start, or class size reduction in the early grades. The finding that family financial support enhances academic achievement in the form of test scores is consistent with other research on the impact of the EITC showing impacts on later outcomes such as college enrollment.[xviii]

The findings I have presented are with respect to a particular form of family support, more income. They have implications for but are not a direct test of the more constrained forms of family support I presented previously that condition subsidies to activities that directly involve the care of young children. I make the reasonable assumption, based on the research findings illustrated in the figure, that children in low-income families will do better in school if their families receive more income from taxpayer appropriations.

I hypothesize that those children will do even better if the additional family income were delivered with constraints that tie it to direct investments in children, as was the case for the program in Norway in which parents were reimbursed for a portion of their costs of purchasing center-based care for their preschooler. The principal constraint would be that the funds would have to be used to support the direct care of young children, e.g., subsidizing out-of-home child care or the reduction of work hours so that parents can spend more time with their children. Think of the constrained family support option as akin to the federal Supplemental Nutritional Assistance Program, usually referred to as food stamps—a direct financial support that can only be spent on certain types of consumption. Contrast this with the EITC or a subsidized minimum wage in which the family receives extra income with no restrictions on how it can be spent.

There no research of which I’m aware that directly addresses in a rigorous way the relative effectiveness for children of unconditional cash transfers to families vs. cash transfers that reimburse or subsidize parents for spending on their children. Would the EITC be more effective in boosting the human capital of children if it incorporated the conditional characteristics of the federal Child Care Tax Credit? This and related questions are important and unanswered.

In this context we have three broad policy options on the table with regard to public expenditure to support early development and learning in low-income families. The first is investment in particular program services in the form of school readiness programs such as Head Start and statewide universal pre-K. The second is providing parents with additional income that they can spend only on services for their young children, such as center-based care. The third is providing parents with additional income that they can spend on anything. In my view, and in light of the evidence I’ve reviewed, the overwhelming focus by politicians, child advocates, and the research community on option one, the school readiness option, is undesirable. It is possible that options two or three are more effective than providing low-income families with a year of free school for four-year-olds or that they are multipliers of the impacts of doing so. We will never know unless the family support options are included in the policy menu and subject to test.  

      

 

 

The nation’s PK-12 education ecosystem is poised to embrace programs intended to enhance soft skills. Soft skills are generally defined by exclusion as personal qualities other than the formal knowledge transmitted by schools that affect student adjustment, i.e., the effort that students put into their work and their social skills. Such soft skills are far too important for the education reform effort associated with them to suffer the fad-like fate of far too many education reforms of the past. There are danger signs in that regard.

One problem is that advocates of soft skills reform have approached the conceptualization and measurement of soft skills in ways akin to how psychologists approach human personality, i.e., as relatively enduring, trait-like individual differences in broad patterns of behavior. Such patterns of behavior are highly heritable, meaning that schools will have difficulty influencing differences among students. They are also abstract and general, meaning that they provide little of the specificity that is needed for the design of curriculum for students in different grades or for the provision of useful feedback to teachers or students. Further, the theory and measurement of soft skills in schools is in its infancy, with many critically important questions unanswered.

Also troubling are recent research findings that charter schools that are both effective in raising student achievement and focused on character development either have no impact or a negative impact on students’ self-reported soft skills. Such findings conflict with the implicit theoretical model of soft skills reform in which the causal path to better academic achievement and life outcomes flows through students’ soft skills as enhanced by schools.  

A prudent way forward for educators given the many acknowledged unknowns in soft skills reform is to substantially enhance efforts that fall within traditional school practices and responsibilities rather than to boldly make risky bets on unproven programs and measures. Practical steps for school and district administrators include: 1) focusing on improving student behavior, not personality traits; 2) implementing schoolwide rule systems focused on respectful social interactions; 3) using measures of soft skills that are naturally occurring and useful as feedback at the classroom and individual level; 4) establishing priorities around students who are significantly off-track in their social-emotional behavior or self-management skills; 5) establishing priorities around remediation or removal of teachers whose interpersonal behavior toward students is likely to be doing harm; and 6) putting in place systematic ways to learn from and improve the reform efforts.

Background

You live on the north coast of England in the 10^th century. A lookout, seeing the longboats make landfall, shouts the alarm—“the Vikings are coming.” The fiercest warriors in the western world are about to descend on you. Your panic and fear are unimaginable.

Cut to the 21^st century. Someone says “the Scandinavians are coming.” You feel mild unease because you don’t like high taxes and jellied fish. Rape, pillage, and plunder are not among your concerns.

It doesn’t take a difference of a 1,000 years in Nordic culture to make the point that human interpersonal behavior is malleable and consequential. I travel a lot. If I don’t consciously remind myself where I am, the chances of faux pas rise substantially. I might, for example, make eye contact and say good morning to a stranger on an elevator in Manhattan, or fail to do so in Raleigh. Over the years and many school visits I’ve been struck more than once with conspicuous differences between how students and adults behave in buildings that look pretty much the same and serve similar students—orderly vs. chaotic is the essence of the contrast.

These time and place examples support the intuition that interpersonal behavior of groups of people is a product of cultural institutions. This belief, with a dash of evidence thrown in, is the bedrock of the advocacy movement for the elevation of soft skills in the education curriculum.

Surely soft skills are important and schools have an important role in shaping them. But the reality is that research on soft skills is soft. It isn’t even clear what we’re talking about (stay tuned), much less what works in schools that are trying to improve student competences in this domain, or who should be held accountable for what and how.

It is also important to understand that individual differences in soft skills are as or more important than the central tendencies of cultural and group differences: some Vikings were more aggressive than others; among people trying to be friendly on an elevator, some pull it off effortlessly and others strain; and the orderly school contains some students who are tumultuous. The rank order of individuals on dimensions such as aggressiveness and sociability is relatively stable even as the forms of expression vary with time, place, culture, and circumstance. This has implications for the priorities and focus of effort by schools trying to impact soft skills.

The nation’s PK-12 education ecosystem seems poised to embrace programs intended to enhance soft skills. In part, this is due to a new requirement in the recently reauthorized federal Elementary and Secondary Education Act (ESSA) that state accountability systems include at least one nonacademic measure. Nine school districts in California that serve roughly 1 million students have already organized as the CORE Districts and instituted an accountability system in which 40 percent of the total weight for school success is assigned to measures of social-emotional outcomes,[i] including a component based on direct self-report assessments of students on growth mindset, self-efficacy, self-management, and social awareness.[ii] Also notable are eight large urban districts scattered across the country that are part of the Collaborative for Academic, Social, and Emotional learning (CASEL).[iii] They are in the process of carrying though on a commitment to adopt social and emotional learning standards, put in place social and emotional learning programs for students and professional development programs for teachers, and carry out social and emotional learning assessments.

Soft skills are far too important for students for the education reform effort associated with them to suffer the fad-like fate of far too many past education reforms, including those that were based on a valid core assumption. The goals of the present report, which will extend over subsequent reports, are: first, to raise important questions in the context of the expansion of efforts by schools to enhance the soft skills of their students and measure outcomes; second, to suggest what prudent school officials and policymakers should do with respect to incorporating soft skills into the school curriculum given the number of unanswered critical questions about how to proceed; and, third, to spur the organizations and individuals that are at the forefront of the movement to increase their interest and investment in the many unknowns of soft skills reform.

Defining the domain

The student dispositions, skills, traits, and abilities that are this report’s subject matter have been variously labeled as: soft skills, emotional intelligence, social and emotional learning, personal qualities, character, virtue, non-cognitive skills, 21^st century skills, and so on. The topic spans vastly disparate categories of student behavior from easily observed actions such as completing homework, to abstract dispositions and ways of thinking such as optimism, grit, social awareness, and a growth mindset. The complexities and challenges for schools and educators of including such disparate behaviors, thoughts, and dispositions into the overarching grab bag of soft skills are large.

Schools that try to do everything are likely to accomplish nothing well. Thus, the first challenge for soft skills education reform is a coherent answer to the question: What are we talking about and trying to influence? 

Existing approaches to handling the dispersion, abstraction, and lack of coherence of a miscellany of soft skills largely take the form of creating subcategories that offer greater similarities among their members than the overall collection. For instance, Stecher and Hamilton distinguish between interpersonal competences such as the ability to work with others vs. intrapersonal skills such as having a growth mindset (believing that your abilities can be developed through hard work).[iv] The CASEL identifies five core competences: self-awareness, social awareness, self-management, responsible decision making, and relationship skills.[v] The authors of a Chicago Consortium report offer a different five categories: academic behaviors, academic perseverance, academic mindsets, learning strategies, and social skills.[vi] 

A leading researcher in the field, Angela Duckworth, and her colleague, Daniel Yeager, conclude that “the debate over the optimal name for this broad category of personal qualities obscures substantial agreement about the specific attributes worth measuring”[vii]—in Shakespearean terms, a rose by any other name would smell as sweet.

Duckworth and Yeager are correct that the issue is not the particular name that is used to refer to what researchers and advocates have substantially agreed should be measured. If that is all there was to it, we could take inspiration from The Artist Formerly Known as Prince and get the major players to adopt an unpronounceable symbol to refer to the domain. The important question, an affirmative answer to which Duckworth and Yeager implicitly assume, is whether the specific personal attributes that have been substantially agreed to as worth measuring are what schools should be measuring and held accountable for.

Given the number of unanswered questions about efforts to inculcate and measure soft skills in schools, a topic which I flesh out in the remainder of this report, it is premature and unhelpful for educators to define a school’s mission, select its curriculum and programs, measure its success, and be held accountable for something as amorphous as the various synonyms for soft skills. The “substantial agreement” among academic researchers about what should be measured is, at this point in time, unwarranted.

Distinguishing traits from behaviors

There is a strong relationship, both conceptually and empirically, between present approaches to the definition and measurement of soft skills and efforts in psychology going back almost 100 years to understand human personality. The earlier history of the taxonomic effort in the psychology of personality bears a striking resemblance to present day efforts to catalog soft skills. Just as soft skills are addressed now from a variety of perspectives and at different levels of abstraction, so too was personality addressed then.

The taxonomy of personality traits began to come together with the mid-century efforts by Raymond Cattell to isolate unique personality factors (he claimed to have found 16) using a statistical technique called factor analysis. Progress along this empirical pathway accelerated dramatically in sophistication and replicability of results as later researchers took advantage of advances in computing power and larger datasets.

Today, the field of personality psychology has converged empirically on the so-called Big Five personality traits: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. Each is dimensional, i.e., a person can be high or low or somewhere in between on each of the traits. Each is statistically unique, i.e., an individual’s placement on the trait of agreeableness does not predict well where that person will land on conscientiousness. Each is typically measured using self-report questionnaires. And each refers not to specific behaviors in particular situations but to relative stable dispositions to respond in similar ways across a broad range of circumstances, i.e., there is no specification of what a person high on agreeableness will do or should do to be agreeable at school or in interactions with her peers or family members, only that she will find a way and be better at it than most.

Soft skills lack the century of empirical development that led to convergence on the Big Five personality traits, so the field is presently a Tower of Babel when it comes to constructs and measures. It is only in that regard that the field of personality traits and the field of soft skills differ. They share almost everything else, including the approach to measurement through questionnaires, the intent to capture broad patterns of behavior, and the goal of identifying individual differences that are predictive of later outcomes.

There is another, even more important, reason to link the domain of soft skills to the study of personality traits: they overlap substantially in coverage. For example, when the Chicago Consortium describes the social skills component of their model as “such interpersonal qualities as cooperation, assertion, responsibility, and empathy,”[viii] they are describing components of three Big Five personality traits. “Cooperation” and “Empathy” in the Consortium definition of social skills are defining characteristics of the Big Five trait of Agreeableness whereas the Consortium interpersonal quality of “Assertion” is one of the defining characteristics of the Big Five trait of Extraversion. Finally, the Consortium social quality of “Responsibility” is found in the Big Five trait of Conscientiousness.

The overlap is more than conceptual. In one of the largest studies of soft skills in school settings, the researchers used a Big Five self-report questionnaire as the basis for their measure of conscientiousness.[ix]

There is a significant challenge for soft skills reform if the focus of intervention and measurement is on broad trait-like patterns of behavior such as responsibility, cooperation and empathy. The problem is that the role of genetics and neurobiology looms large compared to the agency of any particular cultural institution, including schools.

As a case in point, the Big Five personality traits are highly heritable. In other words, behavioral differences among individuals in the qualities of openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism are due in large part to genetic differences.

In behavioral genetics, the genetic contribution to individual differences in behavior is estimated by comparing the behavioral similarity across varying environments of people with known genetic similarities. The intuitively easiest case to understand is the comparison of the behavioral similarity of identical twins, reared together and apart.

Researchers find strong behavioral similarities in identical twins separated at birth and reared apart by different families. In fact, such twins are almost as similar in their behaviors and personalities as identical twins reared together. Consider one of many similar stories from media accounts: Paula Bernstein and Elyse Schein, identical twins adopted by different parents ended up living very similar lives: both became writers. As one of the twins remarked after getting to know her twin for the first time at the age of 35: “It’s not just our taste in music or books; it goes beyond that. In her, I see the same basic personality…”[x]

Several independent studies of twins converge on the estimate that a bit more than 40 percent of the variance in the Big Five Personality traits is due to genes whereas only 7 percent is due to the environment that is shared by twins raised together, i.e., home and school. For conscientiousness, the component of the Big Five that was used directly as a student self-report measure in a study of soft skills in the schools of Boston, the estimate of heritability from the four most recent studies is 49 percent.[xi]

School-level impacts on personality traits and trait-like soft skills are not incompatible with a 50 percent heritability of personality traits. In that regard, it is worth keeping in mind that we have lots of evidence that schools affect academic outcomes even though the heritability of scores on standardized tests of achievement at the end of school is, in some models, even higher than it is for the Big Five personality traits.[xii] Also, we know that the quality of the teacher and classroom to which kindergarteners are assigned affects teacher ratings of those students’ social skills years later and their earnings in adulthood.[xiii] But, is it reasonable to think that the school environment is more important than the family in shaping individual differences in broad behavior patterns such as cooperation and social awareness? I think not, although this is an empirical question.

In one of the most revealing studies to date on this issue, KIPP charter schools, which have the development of student character as one of their mission pillars, had an impact on only one of a wide range of student self-report measures of soft skills (collaboration/cooperation with other students), whereas these schools had significant impacts on academic achievement. The math and reading results and some of the soft skills results are presented in the following figure.

The implicit theory of action for soft skills education reform is that, first, student self-perceptions of their self-control, grit, confidence (self-efficacy), and so forth are causally related to their achievement in school and their later life success; second, that schools can affect these self-perceptions through curriculum, school climate, and focused training; and, third, that the school impact on soft skills leads to improved student outcomes in other domains, including achievement.

The KIPP results illustrated in the figure above are not encouraging for this theory of action. KIPP is strongly committed and seriously invested in improving student character, which they view as important in its own right and a critical pathway to student success. The large and methodologically rigorous study on which the figure is based found that the only character/soft skill measure on which KIPP had an impact was student self-report of the extent of collaboration with other students. On everything else, including survey variables not included in the figure, KIPP students scored no better than identical students who wanted to attend the same KIPP middle schools but lost their lottery for admission. However, KIPP had a strong impact on the academic test scores of lottery winners. Thus, the route to better achievement in these KIPP middle schools did not flow through the enhancement of students’ soft skills and character. Or, if it did, none of the measures that were used in the study, which include those that soft-skills advocates cherish, picked up the changes in students’ soft skills that KIPP produced.

The evidence I have described and the larger body of research from which it is drawn suggests that there is a relatively low ceiling on the extent to which schools can affect individual differences in the broad patterns of behavior that are measured through self-report student surveys and conceptualized as soft skills. This does not mean that schools cannot affect the set point for perceptions of students about such things as their efficacy and effort. Although there is little evidence of that in the KIPP study, another rigorous study found that charter schools in Boston actually lowered their students’ self-ratings of soft skills while raising their test scores.[xiv]   

The heritability evidence I have noted means that it is very difficult for schools to change the relative ranking of students within a school on self-ratings of trait-like personal attributes such as conscientiousness. The evidence from the KIPP study and elsewhere suggests, further, that we know little empirically about how schools can exercise influence in ways that enhance soft skills and thereby improve academic achievement and life outcomes for students.

The need for specificity in what should be taught and learned

Consider the following definitions of “key competences” within the broad domain of social and emotional learning that are available from leading advocates of soft skills reform: social awareness: “The ability to take the perspective of and empathize with others from diverse backgrounds and cultures…”; responsible decision making: “The ability to make constructive and respectful choices about personal behavior and social interactions…”; soft skills: “personality traits, goals, motivations, and preferences that are valued in the labor market, in school, and in many other domains”; academic behaviors: “those behaviors commonly associated with being a ‘good student.’”

Contrast these abstractions with the specificity one finds when pivoting to the hard skills of reading and mathematics: For example, a Common Core State Standard for literacy for fourth graders is that students will “use combined knowledge of all letter-sound correspondences, syllabication patterns, and morphology (i.e., roots and affixes) to read accurately unfamiliar multisyllabic words…” A parallel Common Core standard for math for fourth graders is that they will “fluently add and subtract multi-digit whole numbers using the standard algorithm.” The clear force of such standards is that the curriculum that is delivered to fourth grade students has to include instruction and assessment on phonics and whole number arithmetic, and ideally leads to basic competency for all fourth graders in those skills.

Within the domain of soft skills there is nothing remotely close to this level of specificity in terms of what needs to be learned and when. We can probably agree, for example, that it is desirable for children and youth to engage in “responsible decision making.” But what should that look like for a second grader vs. a twelfth grader?

Existing efforts to merge Common Core academic standards with soft skills don’t get much beyond pointing out that there are opportunities to enhance soft skills while teaching the core academic subjects, i.e.,

The mathematical practices standards call for perseverance in solving problems, which supports the development of self-direction and productivity. In modeling, statistics and probability, for example, students can work in collaborative, project-based units in ways that emphasize Life and Career Skills.[xv]

Advocates for the inclusion of soft skills in the curriculum need to be a lot less nebulous. Without specificity at the level of what students need to learn and examples of how to teach it, there is no clear path to the development of curriculum and instructional practices, teacher training, or meaningful assessment and accountability.

Measurement and accountability

Reflecting its focus on broad patterns of thought and behavior and an academic grounding that is related to personality psychology, the soft skills movement has developed three types of measures of individual differences in soft skills: self-report questionnaires by students, teacher questionnaires about their students, and short performance measures, i.e., the famous Mischel Marshmallow Test assessing delay of gratification in which children are asked to choose between eating one marshmallow in front of them or holding out for a promised two—how long a child waits is the measure.[xvi]

In a recent report in Evidence Speaks, Martin West provides an informative account of some of the psychometric characteristics of the four self-report questionnaires being used in the CORE Districts in California—districts that are in the lead nationally in measuring students’ soft skills and holding individual schools accountable for how their students score.[xvii] In brief, West reports that four measures of student soft skills administered to middle school students (self-management, social awareness, self-efficacy, and growth mindset) have respectable levels of internal reliability, i.e., answers by individuals to questions within each scale correlate highly with each other. Examining school averages, the soft skills measures are positively correlated with math and ELA scores and negatively correlated with student suspensions and absences. Based on a dataset of 240 schools, the correlations between the four soft skills measures and math and ELA state test scores range from 0.33 to 0.69. Correlations with suspensions and absences are somewhat lower although still statistically significant. West concludes that his findings “provide a broadly encouraging view of the potential for self-reports of social-emotional skills as an input into [the CORE Districts] system for evaluating school performance.”

There are many questions that need to be answered and assumptions that need to be addressed before districts should be encouraged by such findings to give soft skills assessments to students and use the results to hold schools accountable, including:

• What are the appropriate indicators of the validity of self-report measures of soft skills? It isn’t obvious conceptually that we should expect or want high cross-sectional correlations between soft skills and academic test scores in middle school. James Heckman, and others who have taken his lead, hold that interventions that impact soft skills such as pre-K play out later in life rather than in better test scores in school.[xviii] For example, Chetty et al. report that the differences between good vs. bad kindergarten classrooms can be seen in measures of student soft skills in middle school but not in academic test scores, and that it is these soft skills that lead to higher wages in adulthood for people who previously had a good kindergarten teacher.[xix] Further, the KIPP study previously referenced demonstrates a substantial independence between achievement gains and soft skills as measured through contemporaneous student self-reports. A study by West and colleagues indicates a negative relationship between achievement and soft skills impacts.[xx] At this point in time, we don’t know what level of correlation between a measure of self-efficacy and a measure of academic achievement is desirable or what any level of such correlations means with respect to the validity of the measure of self-efficacy.

• Where is value-added? Data showing that schools vary in their mean scores on measures of soft skills can no more reasonably be used to hold those schools accountable than data showing that those same schools vary in their mean scores on measures of student vocabulary. In both cases family background and the selection of students into schools is presumptively more likely to be responsible for the measured differences than anything the school has or has not done for which it can to held to account. An accountability system based on test scores on measures of soft skills would need to capture school-level and grade-level gains on those measures so that schools are being evaluated on the changes they induce in students’ soft skills, i.e., their value-added.

• Where is the teacher? One of the most powerful insights to emerge from the last 15 years of education research is the disproportionate role of the teacher and classroom relative to the school or district (or most anything else other than the family) in student outcomes.[xxi] Yet nearly all the attention among advocates of soft skills reform is at the district and school levels. In the CORE Districts, for example, it is schools that will be held accountable for social and emotional outcomes. It is likely that what teachers do in the classroom with respect to soft skills, just as for traditional academic skills, will be where most of impacts on students are to be found.[xxii] Further, the low-hanging fruit for improving soft skills will be found in interventions intended to improve the practices of or to remove from the profession those teachers who are negatively impacting students’ soft skills. We need classroom-level measures of soft skills to bring focus on where the education system is having the greatest impact for good or ill.

• Why use proxy measures of valued outcomes when direct measures are available? West demonstrates that scores on soft skills questionnaires correlate negatively with student suspensions and absences.[xxiii] Which should a district, school, or teacher care most about, a student’s score on a measure of growth mindset or a student’s behavior that results in a suspension? Which should be a priority, an intervention that increases a student’s self-rating of conscientiousness or an intervention that increases timely completion of homework? Standardized test scores are a valuable and needed index of how much students are learning in the core academic disciplines. The test questions are close proxies for what students are expected to learn in their studies, the predictive validity of the assessments has been demonstrated repeatedly, and practical alternatives are not available. In contrast, a large portion of the domain of soft skills is directly observable by teachers and generates administrative records. To the credit of the CORE Districts, they collect and use the direct measures of suspensions and absences in their accountability system.

We need more such measures. At the classroom level, more detailed report cards that include teacher ratings of such things as student effort are promising.[xxiv] Of relevan