How Children Succeed, or, Grit is the New IQ

February 21, 2013 Leave a comment

This post takes its title from Paul Tough’s excellent new book “How Children Succeed“, which draws its inspiration from the growing scholarship and research in non-cognitive attributes and their role in success and failure. Tough’s goal is to collect research that covers this from a wide range of angles, and substantiate his thesis that “grit, perseverance, and the hidden power of character” are crucial to childhood (and beyond) success in school and other endeavors.

My institution is what they call “highly selective”, meaning that the admission screening for applicants is quite serious, and every student entering the University has been academically and personally successful in the past.  We use what admissions people call a “holistic” process to review applicants, and I really do believe that the admissions folks read every word of every essay.  There can be no doubt that we get a tremendously strong set of students to join our community.  The question is what happens to them once they get here, and why?

I’ve done a little analysis of a recent first-year class;  this is data from one of the last few years.  Turns out that there is quite a lot of scatter in the data and that SAT is not an especially good predictor of first year GPA.  This is perhaps not surprising, since the complaints about the predictive power of the SAT seem pretty well established.


Does SAT Predict First Year Grades? The figure at left shows both a scatter plot of the data (n = 588) and a red line indicating the best fit linear model.  A change in SAT score of 10 points corresponds to a change in first year cumulative GPA of 0.02 point (95% CI: 0.016-0.023).  These results are statistically significant (p < .001). This means that the difference between a 1400 and a 1600 on the SAT corresponds to a GPA difference of about 0.4.  Non-trivial.  This data has a correlation coefficient of 0.42, however.   There is a lot of scatter in the data, the residuals are all over the place, and in general the data doesn’t seem to be very tidy.  Nonetheless, these observations are not out of line with this, or this, or the summary here.  There is a huge amount of “validity” literature about the SAT, so go have a look for yourself.  But the bottom line from this data is that SAT is a weak predictor of first year GPA.

fall_springDoes First Semester GPA Tell Us Anything About GPA in Later Semesters?  Things tighten up quite a bit when we look at how first semester GPA predicts second semester GPA.  The figure at right shows a scatter plot of the data (again, n = 588) and a best fit linear model illustrated as a red line.  A change in fall GPA of 0.1 points corresponds to a change in spring GPA of 0.086 points (95% CI: 0.079-0.093).  This result is statistically significant (p < 0.001).  The correlation coefficient is 0.70.  The fall-spring GPA relationship of nearly 1-to-1 is important and useful.  This fit is more robust, the data hug the linear model much more tightly, and in general the predictive power of the model is higher.

So, now back to the original question about why some kids succeed and why some do not.  By any measure, the students entering Uva are academically talented.  In fact, these data show:

  • our applicant pool is tremendously strong
  • GPA is not a great predictor of first year academic performance
  • our second semester is harder than our first (since the slope of the fall-spring GPA correlation line is less than 1)
  • most of our students do well the first year (mean GPA = 3.18, median GPA = 3.20)
  • only about 56 students ended the first year with a GPA below 2.5

My theory, based upon a huge volume of conversations with student about why they struggle, is based upon grit.  My sense is that students in the low-GPA crowd are intellectually talented and capable, but are not prepared for the adversity and challenges that they face in college.  Many of these students are unable to adjust their approach to learning to suit their new environment–whatever they have been doing to achieve academic success in high school and even before…well, they just keep doing it (even though it’s not working).  They don’t have the tools to adapt and adjust, to make the hard decisions about how to succeed, and most importantly to rebound from failure.

Grit is defined as perseverance toward a long-term goal.  It characterizes how we respond to failure, how we rise to meet a challenge, how we engage with our work.  Go ahead, get your grit score.  In Paul Tough’s work, grit is just one of a set of non-cognitive skills that plays a fundamental role in childhood and lifelong success.  And I am pretty well convinced that these sorts of traits are the ones that see a student with 1500 SAT score end his or her first year of college with a 2.2 GPA.  It’s not about intellect.  It’s not about IQ or SAT or AP exams or anything else.  It’s about preparation to meet a challenge, and preparation to grow as a person through experiences and–gulp, dare we say it?–failure.  It’s about openness to experiences and having a malleable mindset.

The challenge:  what is the right intervention in higher education to cultivate grit and other key non-cognitive skills in students who clearly have the intellectual capacity to succeed?


I am the 90%

January 20, 2013 Leave a comment

Yes, I am.  I’m this 90%.  And this one.  And this one too.  (But I hope not in this one.) But I’m also in this one: the 90% of MOOC students who do not finish the course.  And I’m okay with that.  On balance, my MOOC experience has been quite positive: my first course in Fall 2012 was “Computing for Data Analysis”, taught by Roger Peng from Johns Hopkins, and it was a well-constructed and nicely-delivered course.  Not shift-the-Earth-off-its-axis great, but very serviceable and for self-motivated learners it served a nice purpose.  I am currently enrolled in “Data Analysis”, offered by Jeff Leek also from Johns Hopkins.  It’s also a well-thought-out course that (so far) has given a gentle but useful overview of doing data analysis, especially on large datasets.  Great.

But think of the pedagogical challenges associated with developing a MOOC.  Your students, perhaps over 100,000 of them, are:

  • from dozens or more different countries around the world, with different cultural views and experiences of education
  • from all age groups and levels of previous academic achievement
  • equipped with different levels of academic ambition (i.e., some want to take the course for very specific career-related reasons, some might want to simply “sit in” and observe)
  • confronted with different levels of constraints upon their time available to devote to the course
  • and so on…

Essentially, when designing a MOOC you are trying to develop an educational experience that respects and reflects all those differences listed above (and more), yet serve some segment of the student population–presumably you try to teach to the students that will finish the course–whose thirst for the course content is highest.  Moreover, you are probably modeling the course after an existing, in-person course offered at a brick-and-mortar institution somewhere, and “translating” it to the MOOC domain.

So back to why I am the 90%.  I work, have kids, engage in the community, and do all the things that lots of other people do with their time.  I am interested in the subject matter of my two courses, and it’s certainly helpful for my job, but I’d hesitate to say that it makes a discrete difference in the quality or quantity of my work.  So I’m not motivated enough to actually complete all the work in the course.  One night, I say down in bed at 10 pm to do one of the programming assignments for the Computing for Data Analysis course, and it was literally about 5 am when I realized what had happened.  I was immersed in the material, it was interesting, I was definitely learning about the R programming language, but this was no way to live.  It took me two days to recover from doing my homework.

So I am happily part of the 90%.  I learned the course material to a large degree, I can perform lots of basic functions in R and am using R right now to analyze some large-ish-scale data we collected from our students (about 1000 rows and about 40 columns).  So I continue to get smarter even though the course is over.  And it’s helpful that the course I’m in now (that I’m not planning to complete) also uses R as the computing platform.  But I am fully happy to be 2-for-2 (or is it 0-for-2?) in MOOCs.  And perhaps for this segment of the population (i.e., working professionals interested in the subject matter), this is the best we can hope for. I still wonder about the students for whom the course subject matter and skills would make a discrete difference in their life and/or employment prospects…how do we get them out of the 90%?  Is it possible that if the course really would make a discrete difference, they would be self-motivated enough to not end up in the 90%?

To be sure, not all MOOCs are created the same. Many are excellent, some are okay, and a few are not so good at all.  But this is an experiment worth doing, despite people like me in the 90%.  What we learn about delivering course content via the MOOC platform could add a lot of value to how we teach face-to-face.  The content-instruction-assessment triad of teaching and learning takes on new importance in the MOOC, and some very thoughtful people are working, right now, on compelling MOOC pedagogies.

But what can the 90% learn about teaching by taking a MOOC?  That’s a better question, and here’s what I think.  My first three observations so far are, you are saying, the obvious things that any conscientious teacher will do for his or her class (and you’d be right):

  • to think very carefully about the preparation of students in the class, and more specifically the variation in preparations especially in a large class
  • to consider how students can access help via the instructors and TAs, especially online and asynchronously
  • to develop assessments that are sensible and try to measure the things that are important

And of course we should follow some basic best practices in how to present materials, use hand-written or PPT notes, etc.  The really enlightening thing for is this:  the social constructivist part of this, including peer support, peer review for grading, and essentially group construction of knowledge and meaning around the course material is exceptionally powerful in a MOOC.  This notion (i) turns that variation in preparation into an asset by enlisted more prepared students to help and support the less prepared students (both formally and informally), and (ii) the peer review part of it absolutely falls into the category of “sensible” (i.e., scalable) and, if you are careful and deliberate in planning your exercises and assessments, will measure the right things in a meaningful way.

This is powerful, for sure.  Where MOOCs fall down as an educational endeavor might be there relative lack of interactivity and the all-important active learning strategies that we talk about so much in educational circles there days.  I think we shouldn’t be too hard on MOOCs in this regard, because on any college campus, on any given day, in any discipline, I bet we can find a face-to-face class with the most dismal, non-active, disengaging lecture approach that has ever existed since the dawn of time.  So, let’s not hold face-t0-face instruction as the gold standard here, because the abuses of face-to-face class time are many, honed by years and years of dedicated practice (ha), and so saddled by instructor inertia as to be virtually unsolvable.

But MOOCs have at least started a new conversation, or perhaps revived an old one, about what an educational environment should look like, how a course should be constructed, what assessments should look like, and what student expectations should be.  And for this, we–the 90% and the 10%–should thank them.


Skeuomorphs and Teaching

December 20, 2012 Leave a comment

Back in early 2010 (wow, three years ago already), I was giving a plenary talk at a conference for Virginia K-12 teachers at a teaching and technology conference.  There were about 400 people in the audience, and the basic gist of my talk was that technology continues to change every facet of life, and of course education should be no different.  And in particular, technology allows/encourages us to use specific conceptual metaphors to understand information.  Obviously, technology is not pedagogy, but at the same time technology-mediated pedagogies can be very powerful.  At one point, I showed a clip from a much longer interview with Bill Gates and Steve Jobs.  Gates says something about how finally–finally!–we are at the point where technology can really do something for education.  After much optimism and many false starts, technology is now really a central part of new, emerging, powerful and effective pedagogies, and we have an “ecosystem” that supports this kind of work.

In the talk, I set up a great analogy between educational innovation, and the innovations of Apple’s iBooks platform (which had just been released when I gave this talk).  The idea was that Apple was about to do for books what it did for music:  radically change the way we conceive of the book, engage with the book, and think about the printed page.  So I went through a very over-hyped introduction (see Slide 14 of the talk), and the showed a picture of the Apple iBooks icon…which looks exactly like a bookshelf. In the talk, I made a sort of exasperated and exaggerated gasp of chagrin that Apple, for all its amazing innovation and sleek design thinking, couldn’t come up with something better than a bookshelf.  You can even see the grain of the wood.  Incidentally, if you are curious about the future of the book, my colleague Michael Suarez is as smart as anybody in thinking about this.

Alas, this bookshelf serves a purpose: it is a (digital) skeuomorph.  I was way ahead of the curve by talking about this in 2010.  Since then, and in particular the latter part of 2012, skeuomorphic design has been much talked about in design circles.  Why use skeuomorphs?  The main reason is familiarity.  When introducing new ideas or new technologies, we often need to anchor our understanding in comfortable conceptual metaphors;  this is why we use terms like computer “desktop”, or Microsoft Word “document”, or web “page”.  These things are not literally desktops or documents or pages, but that terminology immediately lets us know what functions those things serve.

Skeuomorphs serve a particular purpose that can be fruitfully considered in the diffusion of innovations framework championed by Rogers.  In brief, the diffusion of innovations notion of technology or idea adoption within a community depends upon five basic issues:

  • relative advantage:  compared to existing solutions, what relative advantage does this innovation provide?
  • compatibility:  how consistent is this innovation with the cultural norms and values existing in the community?
  • complexity:  what is the perceived difficulty in adopting and using the innovation?
  • trialability:  how easy is it for people to try out and experiment with this innovation?
  • observability:  how readily visible is the impact of this innovation?

Skeuomorphs, then, speak to compatibility, complexity, and trialability.  The iBooks icon clearly signals to prospective users that: (i) the “books” contained within are exactly consistent with your understanding of what books are (high compatibility), (ii) if you know how to use a bookshelf, then you know how to use iBooks (low complexity), and (iii) using these books is as easy as walking over to a bookshelf, selecting a book, and starting to read (high trialability).

How does all this relate to education?  We have learned through our HigherEd 2.0 project (the hard way, sometimes), that early adopters (say, the faculty deploying the innovations) have a larger appetite for technology innovations that non-early-adopters (say, students in the class).  We simply cannot make too large a leap at a time with educational innovations, especially when technology is involved.  With students, I believe the key is relative advantage and observability–students need to see clear and immediate evidence that the innovation supports their learning better than their previous approaches and strategies (relative advantage) and translates into higher achievement (i.e., higher grades) in the class (observability).  Instructors simply cannot go too far of the regular track here.  Instructors must build skeuomorphs into their teaching.  How do you do that?

  • use thoughtful pedagogy: integrate the educational innovation into the class in a direct and well-explained way
  • make it easy for students to do:  this relates to compatibility, complexity, and trialability and respects how students live and learn
  • model innovation usage: show students how to integrate innovative practices into their workflow by doing the same in class (and telling students what you are doing while you are doing it)
  • explain the scholarly basis behind the innovation:  this is in my mind the most important;  explaining to students what you are doing and why you are doing it (i.e., explaining your ideas about the relative advantage for them) goes a long way toward easing students’ concerns about adopting new approaches

Perhaps this is an emerging skeuomorphic pedagogy, necessitated by the rapid evolution of technology, but inhibited by the general, rather inertia-laden approaches to teaching in higher education.  Early adopters and educational innovators will do well to consider skeuomorphic cues in their teaching so that their innovations can be greeted acceptingly by students and colleagues alike.

Nate Silver: Rock Star

November 8, 2012 Leave a comment

Nate Silver, the data-wonk-cum-blogger-cum-NYT-contributor-cum-statistical-demi-god-cum-media-darling of this week’s election…well, he got it right (compare his “forecast” with his “nowcast”).  But what, exactly, did he get right, and how did he do it?  The media and various pundits are enamored with Silver’s moxie and uncanny accuracy in predicting the election’s outcome.  But it appears to me that the big winner of this story is cold, hard, sober data analytics.  His blog is a playground of interesting, practical, well-founded analysis of data, data, and more data.  This is big data, huge data, culled from multiple sources and giving specific state-by-state snapshots of the situation on the ground over time.  This is no trivial task to synthesize all this information into a set of predictions.

Why are we so enamored when someone uses math productively?  Think of it this way:  in popular culture, when someone is good at math, people say: “wow, you must be really smart”.  But when someone is good at, say, history, people say: “wow, you must really like history, and you probably studied a lot to get to be so knowledgeable about history.”  It’s a bias, plain and simple, against the kind of basic quantitative literacy that will only become more important to this nation and the world over time.  How can we evaluate election results, pollution data, SOL outcomes, or any other quantitative information without a basic foundation in, and respect for, general quantitative literacy?

So what did Silver get right?  He looked at the whole range of polling data available over time, and worked to evaluate the quality of that data by examining potential error/bias embedded in it.  The key was that he aggregated data from multiple sources and finessed an understanding of the sources of (and magnitude of) the error in the aggregated dataset.  This is quite contrary to John Oliver’s Twitter-based, real-time approach to prognostication (starts at about 2:45 of the clip). Silver’s brand of sober analysis leads to forceful predictions like this (from his blog, 11/3/12):

To be exceptionally clear: I do not mean to imply that the polls are biased in Mr. Obama’s favor. But there is the chance that they could be biased in either direction. If they are biased in Mr. Obama’s favor, then Mr. Romney could still win; the race is close enough. If they are biased in Mr. Romney’s favor, then Mr. Obama will win by a wider-than-expected margin, but since Mr. Obama is the favorite anyway, this will not change who sleeps in the White House on Jan. 20.

My argument, rather, is this: we’ve about reached the point where if Mr. Romney wins, it can only be because the polls have been biased against him. Almost all of the chance that Mr. Romney has in the FiveThirtyEight forecast, about 16 percent to win the Electoral College, reflects this possibility.

Yes, of course: most of the arguments that the polls are necessarily biased against Mr. Romney reflect little more than wishful thinking.

It is both unfortunate and energizing to think that the general public (and the pundits in particular) might not fully appreciate how math works, how practical it can be, and why a systematic consideration of not just the mathematical operations, but also the quality of the input data, can lead to better predictions, or better policies, or better profits, or better quality of  life.  Unfortunate and frustrating, perhaps.  But it’s also an important opportunity for us, as academics and people in the science/technology/mathematics literacy world (we are, after all, in higher education), one that should energize us with the challenge that lies ahead.

A Bold Proposal:  Let’s develop a course on information literacy, required for every student at UVa, and continuously measure the outcomes and impact of that course on how students approach their academics and their life.  An educated, global citizenry requires nothing less.

Big Data

October 13, 2012 Leave a comment

I’ve been thinking a lot lately about the skill set engineers graduating in the next few years will need to prepare them for a vibrant and productive career.  I can think of very few skills that rival big data analytics in importance for the next generation of engineers.  We continue to collect data on a vast scale every day.  We need to look no further than, the repository of an astounding array of data from a huge swath of government agencies on subjects ranging from the crucial to the mundane.  Data and analytics are everywhere, in every aspect of our lives; to wit:

  • commerce: the NYT ran a fascinating article on Target and its data collection and analytics efforts on its customers
  • health: from the Nike+ system (which I’ve been using since 2007) to the Zeo sleep system, personal data collection systems are popular, growing in their diversity, and increasingly powerful
  • sport:  perhaps my favorite example, the Manchester City Football Club (England) has started an MCFC Analytics initiative, in which they release player performance data to the community, and the community is encouraged to analyze, graph, and otherwise break down the data “however you see fit”

Big data is here to stay, and everyone–engineer or not–needs basic literacy about how to access, analyze, interpret, and otherwise engage with data.  So it is incumbent upon the faculty in higher education to give students opportunities, experiences, and training around this critical skill set.

Big data hits upon several of the key student learning outcomes that educators have wrestled with for many years:

  • an ability to pose research questions and gather sufficient resources/evidence to answer those questions
  • a general comfort with uncertainty, lack of complete information, poor signal-to-noise ratio, etc.
  • the ability to conduct data analysis, especially on large data sets, using modern computer tools
  • the ability to visualize data, and use graphics to tell a persuasive story about what the data means

These basic skills are part of the new literacy for all engaged citizens–not just scientists and engineers.  And developing curricula around big data opens up some enticing new possibilities on student motivation and engagement:  students can choose to ask and answer research questions about which they care, in topical areas that are meaningful to them.  A student interested in environmental issues could analyze public data sets about pollution, air quality, or water quality.  A student passionate about economics could look at unemployment rates, pay scales, or international trade.  A student interested in energy could examine subsidies for green energy companies, consumption locally and worldwide, or performance data for various alternative energy technologies.

There’s much more to say here.  But preliminarily the point is that when it comes to education and the future of an educated citizenry, basic literacy about how to understand data is more important than ever before.

On my bookshelf

September 18, 2012 Leave a comment

What I’m either reading now, or have recently read:

  • How Children Succeed, by Paul Tough.  A thought-provoking look at the critical factors that promote “success”.  Turns out it’s not as important to have a child who excels in math, or is doing algebra in first grade, or who is trilingual.  What matters most is what Tough and others call “grit”–that character trait that makes us resilient, committed, determined, and resolved.  Perhaps the anti-Tiger Mother?
  • The Imperfectionists, by Tom Rachman.  For a first novel, this is a great one.  It really spoke to me on a number of levels, but Rachman’s ability to sketch out the idiosyncrasies (and downright lunacy) of human relationships is spot-on.  These are all just people trying to make their way in the world, with ambition but no plan, the best way they know how.
  • The End of Men, by Hanna Rosin.  I have not yet started this one, but I am fully prepared to either love it or hate it–no middle ground.  The reviews have been mixed, some saying its thesis is unproven and stretched well beyond what the thin facts and anecdotes should allow (NYT).  Others see it as giving voice to the new reality of female power and influence (WSJ).  I’ll decide, but Rosin is whip-smart and certainly passionate about the subject.
  • The Immortal Life of Henrietta Lacks, by Rebecca Skloot.  I’ve been working on this for what seems like forever, and I’m still only scratching the surface.  But it’s a terribly compelling story of science, race, dignity, and fairness.  Need to find time to really dig in to this one.
  • Crazy U, by Andrew Ferguson.  On the recommendation of a friend, I read this book about the college admission circus currently gripping the United States and much of the rest of the world.  Completely insane.  This book turns from belly-laughter about the more absurd elements of the process, to the uncomfortable chuckle that accompanies acknowledgement that this game–the one that everyone plays–must be played.  It is, seemingly, not optional.

If you glued Paul Tough and Andrew Ferguson together, what would you get?  The author of a book about how our educational system measures and rewards the wrong things, creates false and potentially harmful incentives for specific behaviors, and generally stresses out students and parents alike. Oh yeah, and it costs too much, and we spend our education money on the wrong things.  Deep and provocative.

Categories: Books Tags: