Posts Tagged ‘curriculum’

The New, New Math (no, this time, really…)

June 15, 2013 1 comment

Last night, two delightfully clean-cut college students came to my door.  Their “Aggies” license plate and A&M visors let me know that this was the outfit who recently incited a scare in the local neighborhoods because they have out of state plates and are looking for houses with small children.  Maybe not the best opening line to a sales pitch.  I didn’t catch the name of the product they were selling, but it was some combination of actual books and website access, and somehow these two things were supposed to work together to improve student reading, especially for young kids.  It was never quite clear how the system worked, and the details are not exceptionally important.  What’s interesting about this situation is:

  • the website access was promised to add value (“it’s fun, kids love learning online…”)
  • the books seemed like, well, books (of which I have a house full)
  • this was a cold-call sales job

It’s almost as if there’s something magic about having a supporting website that makes this sort of thing (at least in the eyes of the sellers) attractive, effective, or worth the money.

My son just finished third grade in a local school, and throughout this school year we received the following proclamations about how he would be learning math this year (note: I am intentionally not linking to any of these).

  • Some time in the fall, my son’s math teacher was “pleased to tell [us] that we will be using a website called IXL in our classroom this year.”  The letter promises that in addition to “making math practice exciting, IXL is designed to help your child learn at his or her own pace.”  A laudable goal, to be sure.  The letter closes by asking parents to “encourage your son or daughter to use IXL daily.”  Well, okay, maybe.
  • On Jan. 29, we received another nice letter about using XtraMath to “increase speed and accuracy in arithmetic”.  Yes, well, okay, speed might be important, accuracy is certainly important, and so yes, right, we can have our child “spend a few minutes each day practicing math on the computer.”  Got it.
  • On Feb. 7, we received another letter, this one stating that students in this math teacher’s class “have an opportunity to work with an exciting new math product.  ExploreLearning Reflex is an online, game-based program that helps students build fast and effortless recall of math facts.”  Hmm.  Okay, well, yes, I’d like my son to have fast and effortless recall, but the Jan. 29 letter told me that XtraMath was going to increase his speed and accuracy too. So I am not sure what to do here.
  • Throughout the entire school year, the class was using something call ST Math.  ST stands for “spatial-temporal”, and the premise of ST math is that students are introduced to a math problem visually first, then using more traditional mathematical symbols.  The visual narrative is driven by the adorable (ahem…) penguin Jiji.  Students essentially help Jiji solve math puzzles so that whatever obstacles are in Jiji’s way can be removed.  The obstacles represent the visual part of the math problem.

I’m not complaining about any one of these products in particular.  In fact, ST Math in particular claims to have a strong basis in research, and the group at UC Irvine who created the system has some scholarship available to substantiate their claims about how the system supports achievement on standardized tests.  Whatever.  And apparently it costs on the order of $100 per students per year.  Yikes.

What is shocking and distressing is this hodgepodge of introduced-then-quickly-forgotten websites that promise learning, sharpening of skills, and (!) fun.  Other than ST math, which really permeated the whole school year, I have never heard of the other three again.  And it might be worth noting that my son, whom I consider to be a very visual guy with a really good talent for visual/spatial mapping, hated, really abhorred ST math.  He thought the whole storyline with the penguin was silly, he thought the adaptivity was weak (in the sense that the questions didn’t adapt fast enough, and he had to endure too many questions on the same topics), and he generally felt like it was not an effective tool for him to learn math.  Call them learning styles, or learning preferences, or whatever you want, but the point is that not everyone learns the same way, and this way didn’t work for him.

I’ve learned a lot about the enterprise of education over the past few years of public schools.  Public schools have it rough.  The range of preparation of students entering the system, the demands of NCLB and standardized testing, the differential commitment of parents to the school and its mission, the constant sense that resources are not spent wisely, on the “right” things.  Teachers are overworked and underpaid.  The burnout rate is high, and teacher turnover is costly (This is astonishing:  “The total cost of turnover in the Chicago Public Schools is estimated to be over $86M per year.”  And the cost of a single teacher leaving the system is on the order of $17,000.  In Chicago, there are around 23,000 teachers, and this data means that around 4,800 of them [20%] turnover in a given year.).  There’s no doubt that teachers deserve more love, more professional development, more efforts at retention and general job satisfaction, and more respect from the public.

But I am an online skeptic, a MOOC skeptic, unlike some others.  And I make constructive use technology within my pedagogy as much as anyone.  But I feel strong dismay at the notion that students are sophisticated enough in their understanding of how they learn to be able to make good judgments about how to productively engage with these technologies.  Yes, perhaps the role of the teacher is changing to something more like a coach or mentor.  I get that, and I generally like the idea.  But the teacher plays a central role;  not as gatekeeper of information, but also not as an incidental part of the educational process either.  The teacher must be directly involved in the student’s experience, and here’s what I think are the important things teachers can do:

  • motivate students, and using their knowledge of a student’s personality and personal circumstance to tap into their desire to succeed
  • challenge students, by pushing them to meet and even exceed their own expectations
  • inspire students, by being the positive role model for learning that a computer could probably never be
  • engage students in the critical thinking and the memorable and crucial give-and-take of classroom discussions, whether about arts, literature, or even math

This is not intended to be a polemic attack on public schools, or teachers, or parents, or students.  It is, however, a strong lament about the current state of technology in education, especially K-12 education.  I am concerned, more than ever, that just because technology is ubiquitous, people will use it for all sorts of things that it isn’t ready to be used for.  And just because technology is all around, it is perceived to be disposal–or worse yet, interchangeable.  The three websites I never heard about again are perfect examples of a technology pop culture:  pretty, shiny, disposable, and nobody will remember them even a year from now.


What Should Teachers Do, Part 1?

March 25, 2013 1 comment

The past few years have seen exciting and inspirational ideas emerging from a variety of sources, all focused on how to make the next generation of Americans productive, happy, efficient, insightful, innovative.  In short, the next great generation of world leaders.  I’ve read a number of books that talk about education writ large, meaning K-12 and higher ed, and in fact life long learning.  The similarities of these books with each other are fairly striking, and they are largely getting at many of the same topics from different angles.  But there are differences too.  Let’s look at some of the recent books on my bookshelf and see what advice they give us.

The Global Achievement Gap, by Tony Wagner (2008).  Tony Wagner is co-Director of the Change Leadership Group in the Graduate School of Education at Harvard and has been a leading voice in education policy and practice for quite some time.  Despite the ominous subtitle (“Why even our best schools don’t teach the new survival skills our children needs–and what we can do about it”), the book presents a compelling prescription for what the problem is.  Wagner’s “7 Survival Skills for Teens Today” hit on the key habits of mind that emergent adults need to thrive in the modern economy.  The 7 survival skills are:

  1. critical thinking and problem solving
  2. collaboration across networks and leading by influence
  3. agility and adaptability
  4. initiative and entrepreneurialism
  5. effective oral and written communication
  6. accessing and analyzing information
  7. curiosity and imagination

I don’t think any of these is really a modern skill, one brought on the by the information age, with the possible exception of collaboration across networks (which is often mediated by technology).  I think of these as time-tested skills that had assumed new urgency and importance in today’s uber-competitive world.

A Whole New Mind, by Daniel Pink (2005).  This book’s subtitle also jabs at the notion of quantitative thinking as the key to mastering the modern world.  “Why right-brainers will rule the future” proclaims the small print on the cover, as it unveils a series of ideas about how the right side of the brain holds the key to unlock the future’s piggy bank of health and wealth.  Pink’s ideas about “high concept, high touch” enterprises (“1. can someone overseas do it cheaper? 2. can a computer do it faster? 3. Is what I’m offering in demand in the age of abundance?”) are certainly the right questions, and I actually like this book a lot.  Pink also present six “senses” for the modern age, which are:

  1. not just function, but also design
  2. not just argument but also story
  3. not just focus but also symphony
  4. not just logic but also empathy
  5. not just seriousness but also play
  6. not just accumulation but also meaning

What I like about this framing is that it starts with actions on the low end of the cognitive taxonomy (gathering, knowing, understanding;  more on this later…) and ends with the high-end cognitive skills (designing, creating, composing, etc.).

How Children Succeed, by Paul Tough (2012).  One of my favorite books in the category, this one provides perhaps the most progressive view of how we engage students in authentic learning while simultaneously building character.  This is also the most thoroughly modern book, in the sense that it provides a very accessible review of the currently-hot literature on “grit” and similar non-cognitive factors in success.  The idea is that the traditional cognitive skills (math, reading, and so on) we teach in schools are important, but a more critical task is to cultivate students who possess this non-cognitive strength, can overcome challenges, are resilient in the face of failure, and have that elusive “it” that makes them driven to succeed.  A vast simplification of a subtle and interesting book would focus on the three qualities mentioned in its subtitle:

  1. grit:  perseverance and passion toward long-term goals
  2. curiosity: the drive to learn more about the world
  3. the power of character: resilience in the face of setbacks

This book encapsulates what I believe to be the best thinking about success and failure, and the role that these non-cognitive skills play.  I’ve written about this recently, using data from my own institution to make the point that the input credentials of our students are fairly uniform and very strong.  But once they arrive in my school, their ability to succeed changes.

But what, you say, about the specifics of engineering education.  Glad you asked.

The Engineer of 2020, the National Academy of Engineering (2004).  This book in some ways generated the long line of urgent calls for reform in engineering education, including publications like the Gathering Storm report or the Duderstadt report.  The question it works to address is:  what are the competencies engineers will need in the world of 2020 and beyond?  The answer will not surprise you.  In addition to the basic literacies in mathematics and science, plus discipline-specific expertise, the engineer of 2020 needs:

  1. strong analytical skills
  2. practical ingenuity
  3. creativity
  4. communication skills
  5. business fundamentals
  6. leadership skills
  7. high ethical standards and professionalism
  8. dynamism, agility, resilience, and flexibility
  9. the passion to be a lifelong learner

It is rather striking the entirety of the “traditional” engineering curriculum (math, science, and discipline-specific knowledge) is lumped into a single entry on this list!  Okay, so it’s number 1 on the list.  But still, it’s only one of many important entries.

In part 2 of this, I’m going to engage in some analysis of this information and frame it in the context of learning taxonomies.  Yes, sounds geeky.  But it’s a useful way to synthesize all this into a more concrete understanding.  Stay tuned.

How Children Succeed, or, Grit is the New IQ

February 21, 2013 Leave a comment

This post takes its title from Paul Tough’s excellent new book “How Children Succeed“, which draws its inspiration from the growing scholarship and research in non-cognitive attributes and their role in success and failure. Tough’s goal is to collect research that covers this from a wide range of angles, and substantiate his thesis that “grit, perseverance, and the hidden power of character” are crucial to childhood (and beyond) success in school and other endeavors.

My institution is what they call “highly selective”, meaning that the admission screening for applicants is quite serious, and every student entering the University has been academically and personally successful in the past.  We use what admissions people call a “holistic” process to review applicants, and I really do believe that the admissions folks read every word of every essay.  There can be no doubt that we get a tremendously strong set of students to join our community.  The question is what happens to them once they get here, and why?

I’ve done a little analysis of a recent first-year class;  this is data from one of the last few years.  Turns out that there is quite a lot of scatter in the data and that SAT is not an especially good predictor of first year GPA.  This is perhaps not surprising, since the complaints about the predictive power of the SAT seem pretty well established.


Does SAT Predict First Year Grades? The figure at left shows both a scatter plot of the data (n = 588) and a red line indicating the best fit linear model.  A change in SAT score of 10 points corresponds to a change in first year cumulative GPA of 0.02 point (95% CI: 0.016-0.023).  These results are statistically significant (p < .001). This means that the difference between a 1400 and a 1600 on the SAT corresponds to a GPA difference of about 0.4.  Non-trivial.  This data has a correlation coefficient of 0.42, however.   There is a lot of scatter in the data, the residuals are all over the place, and in general the data doesn’t seem to be very tidy.  Nonetheless, these observations are not out of line with this, or this, or the summary here.  There is a huge amount of “validity” literature about the SAT, so go have a look for yourself.  But the bottom line from this data is that SAT is a weak predictor of first year GPA.

fall_springDoes First Semester GPA Tell Us Anything About GPA in Later Semesters?  Things tighten up quite a bit when we look at how first semester GPA predicts second semester GPA.  The figure at right shows a scatter plot of the data (again, n = 588) and a best fit linear model illustrated as a red line.  A change in fall GPA of 0.1 points corresponds to a change in spring GPA of 0.086 points (95% CI: 0.079-0.093).  This result is statistically significant (p < 0.001).  The correlation coefficient is 0.70.  The fall-spring GPA relationship of nearly 1-to-1 is important and useful.  This fit is more robust, the data hug the linear model much more tightly, and in general the predictive power of the model is higher.

So, now back to the original question about why some kids succeed and why some do not.  By any measure, the students entering Uva are academically talented.  In fact, these data show:

  • our applicant pool is tremendously strong
  • GPA is not a great predictor of first year academic performance
  • our second semester is harder than our first (since the slope of the fall-spring GPA correlation line is less than 1)
  • most of our students do well the first year (mean GPA = 3.18, median GPA = 3.20)
  • only about 56 students ended the first year with a GPA below 2.5

My theory, based upon a huge volume of conversations with student about why they struggle, is based upon grit.  My sense is that students in the low-GPA crowd are intellectually talented and capable, but are not prepared for the adversity and challenges that they face in college.  Many of these students are unable to adjust their approach to learning to suit their new environment–whatever they have been doing to achieve academic success in high school and even before…well, they just keep doing it (even though it’s not working).  They don’t have the tools to adapt and adjust, to make the hard decisions about how to succeed, and most importantly to rebound from failure.

Grit is defined as perseverance toward a long-term goal.  It characterizes how we respond to failure, how we rise to meet a challenge, how we engage with our work.  Go ahead, get your grit score.  In Paul Tough’s work, grit is just one of a set of non-cognitive skills that plays a fundamental role in childhood and lifelong success.  And I am pretty well convinced that these sorts of traits are the ones that see a student with 1500 SAT score end his or her first year of college with a 2.2 GPA.  It’s not about intellect.  It’s not about IQ or SAT or AP exams or anything else.  It’s about preparation to meet a challenge, and preparation to grow as a person through experiences and–gulp, dare we say it?–failure.  It’s about openness to experiences and having a malleable mindset.

The challenge:  what is the right intervention in higher education to cultivate grit and other key non-cognitive skills in students who clearly have the intellectual capacity to succeed?

I am the 90%

January 20, 2013 Leave a comment

Yes, I am.  I’m this 90%.  And this one.  And this one too.  (But I hope not in this one.) But I’m also in this one: the 90% of MOOC students who do not finish the course.  And I’m okay with that.  On balance, my MOOC experience has been quite positive: my first course in Fall 2012 was “Computing for Data Analysis”, taught by Roger Peng from Johns Hopkins, and it was a well-constructed and nicely-delivered course.  Not shift-the-Earth-off-its-axis great, but very serviceable and for self-motivated learners it served a nice purpose.  I am currently enrolled in “Data Analysis”, offered by Jeff Leek also from Johns Hopkins.  It’s also a well-thought-out course that (so far) has given a gentle but useful overview of doing data analysis, especially on large datasets.  Great.

But think of the pedagogical challenges associated with developing a MOOC.  Your students, perhaps over 100,000 of them, are:

  • from dozens or more different countries around the world, with different cultural views and experiences of education
  • from all age groups and levels of previous academic achievement
  • equipped with different levels of academic ambition (i.e., some want to take the course for very specific career-related reasons, some might want to simply “sit in” and observe)
  • confronted with different levels of constraints upon their time available to devote to the course
  • and so on…

Essentially, when designing a MOOC you are trying to develop an educational experience that respects and reflects all those differences listed above (and more), yet serve some segment of the student population–presumably you try to teach to the students that will finish the course–whose thirst for the course content is highest.  Moreover, you are probably modeling the course after an existing, in-person course offered at a brick-and-mortar institution somewhere, and “translating” it to the MOOC domain.

So back to why I am the 90%.  I work, have kids, engage in the community, and do all the things that lots of other people do with their time.  I am interested in the subject matter of my two courses, and it’s certainly helpful for my job, but I’d hesitate to say that it makes a discrete difference in the quality or quantity of my work.  So I’m not motivated enough to actually complete all the work in the course.  One night, I say down in bed at 10 pm to do one of the programming assignments for the Computing for Data Analysis course, and it was literally about 5 am when I realized what had happened.  I was immersed in the material, it was interesting, I was definitely learning about the R programming language, but this was no way to live.  It took me two days to recover from doing my homework.

So I am happily part of the 90%.  I learned the course material to a large degree, I can perform lots of basic functions in R and am using R right now to analyze some large-ish-scale data we collected from our students (about 1000 rows and about 40 columns).  So I continue to get smarter even though the course is over.  And it’s helpful that the course I’m in now (that I’m not planning to complete) also uses R as the computing platform.  But I am fully happy to be 2-for-2 (or is it 0-for-2?) in MOOCs.  And perhaps for this segment of the population (i.e., working professionals interested in the subject matter), this is the best we can hope for. I still wonder about the students for whom the course subject matter and skills would make a discrete difference in their life and/or employment prospects…how do we get them out of the 90%?  Is it possible that if the course really would make a discrete difference, they would be self-motivated enough to not end up in the 90%?

To be sure, not all MOOCs are created the same. Many are excellent, some are okay, and a few are not so good at all.  But this is an experiment worth doing, despite people like me in the 90%.  What we learn about delivering course content via the MOOC platform could add a lot of value to how we teach face-to-face.  The content-instruction-assessment triad of teaching and learning takes on new importance in the MOOC, and some very thoughtful people are working, right now, on compelling MOOC pedagogies.

But what can the 90% learn about teaching by taking a MOOC?  That’s a better question, and here’s what I think.  My first three observations so far are, you are saying, the obvious things that any conscientious teacher will do for his or her class (and you’d be right):

  • to think very carefully about the preparation of students in the class, and more specifically the variation in preparations especially in a large class
  • to consider how students can access help via the instructors and TAs, especially online and asynchronously
  • to develop assessments that are sensible and try to measure the things that are important

And of course we should follow some basic best practices in how to present materials, use hand-written or PPT notes, etc.  The really enlightening thing for is this:  the social constructivist part of this, including peer support, peer review for grading, and essentially group construction of knowledge and meaning around the course material is exceptionally powerful in a MOOC.  This notion (i) turns that variation in preparation into an asset by enlisted more prepared students to help and support the less prepared students (both formally and informally), and (ii) the peer review part of it absolutely falls into the category of “sensible” (i.e., scalable) and, if you are careful and deliberate in planning your exercises and assessments, will measure the right things in a meaningful way.

This is powerful, for sure.  Where MOOCs fall down as an educational endeavor might be there relative lack of interactivity and the all-important active learning strategies that we talk about so much in educational circles there days.  I think we shouldn’t be too hard on MOOCs in this regard, because on any college campus, on any given day, in any discipline, I bet we can find a face-to-face class with the most dismal, non-active, disengaging lecture approach that has ever existed since the dawn of time.  So, let’s not hold face-t0-face instruction as the gold standard here, because the abuses of face-to-face class time are many, honed by years and years of dedicated practice (ha), and so saddled by instructor inertia as to be virtually unsolvable.

But MOOCs have at least started a new conversation, or perhaps revived an old one, about what an educational environment should look like, how a course should be constructed, what assessments should look like, and what student expectations should be.  And for this, we–the 90% and the 10%–should thank them.


Nate Silver: Rock Star

November 8, 2012 Leave a comment

Nate Silver, the data-wonk-cum-blogger-cum-NYT-contributor-cum-statistical-demi-god-cum-media-darling of this week’s election…well, he got it right (compare his “forecast” with his “nowcast”).  But what, exactly, did he get right, and how did he do it?  The media and various pundits are enamored with Silver’s moxie and uncanny accuracy in predicting the election’s outcome.  But it appears to me that the big winner of this story is cold, hard, sober data analytics.  His blog is a playground of interesting, practical, well-founded analysis of data, data, and more data.  This is big data, huge data, culled from multiple sources and giving specific state-by-state snapshots of the situation on the ground over time.  This is no trivial task to synthesize all this information into a set of predictions.

Why are we so enamored when someone uses math productively?  Think of it this way:  in popular culture, when someone is good at math, people say: “wow, you must be really smart”.  But when someone is good at, say, history, people say: “wow, you must really like history, and you probably studied a lot to get to be so knowledgeable about history.”  It’s a bias, plain and simple, against the kind of basic quantitative literacy that will only become more important to this nation and the world over time.  How can we evaluate election results, pollution data, SOL outcomes, or any other quantitative information without a basic foundation in, and respect for, general quantitative literacy?

So what did Silver get right?  He looked at the whole range of polling data available over time, and worked to evaluate the quality of that data by examining potential error/bias embedded in it.  The key was that he aggregated data from multiple sources and finessed an understanding of the sources of (and magnitude of) the error in the aggregated dataset.  This is quite contrary to John Oliver’s Twitter-based, real-time approach to prognostication (starts at about 2:45 of the clip). Silver’s brand of sober analysis leads to forceful predictions like this (from his blog, 11/3/12):

To be exceptionally clear: I do not mean to imply that the polls are biased in Mr. Obama’s favor. But there is the chance that they could be biased in either direction. If they are biased in Mr. Obama’s favor, then Mr. Romney could still win; the race is close enough. If they are biased in Mr. Romney’s favor, then Mr. Obama will win by a wider-than-expected margin, but since Mr. Obama is the favorite anyway, this will not change who sleeps in the White House on Jan. 20.

My argument, rather, is this: we’ve about reached the point where if Mr. Romney wins, it can only be because the polls have been biased against him. Almost all of the chance that Mr. Romney has in the FiveThirtyEight forecast, about 16 percent to win the Electoral College, reflects this possibility.

Yes, of course: most of the arguments that the polls are necessarily biased against Mr. Romney reflect little more than wishful thinking.

It is both unfortunate and energizing to think that the general public (and the pundits in particular) might not fully appreciate how math works, how practical it can be, and why a systematic consideration of not just the mathematical operations, but also the quality of the input data, can lead to better predictions, or better policies, or better profits, or better quality of  life.  Unfortunate and frustrating, perhaps.  But it’s also an important opportunity for us, as academics and people in the science/technology/mathematics literacy world (we are, after all, in higher education), one that should energize us with the challenge that lies ahead.

A Bold Proposal:  Let’s develop a course on information literacy, required for every student at UVa, and continuously measure the outcomes and impact of that course on how students approach their academics and their life.  An educated, global citizenry requires nothing less.

Big Data

October 13, 2012 Leave a comment

I’ve been thinking a lot lately about the skill set engineers graduating in the next few years will need to prepare them for a vibrant and productive career.  I can think of very few skills that rival big data analytics in importance for the next generation of engineers.  We continue to collect data on a vast scale every day.  We need to look no further than, the repository of an astounding array of data from a huge swath of government agencies on subjects ranging from the crucial to the mundane.  Data and analytics are everywhere, in every aspect of our lives; to wit:

  • commerce: the NYT ran a fascinating article on Target and its data collection and analytics efforts on its customers
  • health: from the Nike+ system (which I’ve been using since 2007) to the Zeo sleep system, personal data collection systems are popular, growing in their diversity, and increasingly powerful
  • sport:  perhaps my favorite example, the Manchester City Football Club (England) has started an MCFC Analytics initiative, in which they release player performance data to the community, and the community is encouraged to analyze, graph, and otherwise break down the data “however you see fit”

Big data is here to stay, and everyone–engineer or not–needs basic literacy about how to access, analyze, interpret, and otherwise engage with data.  So it is incumbent upon the faculty in higher education to give students opportunities, experiences, and training around this critical skill set.

Big data hits upon several of the key student learning outcomes that educators have wrestled with for many years:

  • an ability to pose research questions and gather sufficient resources/evidence to answer those questions
  • a general comfort with uncertainty, lack of complete information, poor signal-to-noise ratio, etc.
  • the ability to conduct data analysis, especially on large data sets, using modern computer tools
  • the ability to visualize data, and use graphics to tell a persuasive story about what the data means

These basic skills are part of the new literacy for all engaged citizens–not just scientists and engineers.  And developing curricula around big data opens up some enticing new possibilities on student motivation and engagement:  students can choose to ask and answer research questions about which they care, in topical areas that are meaningful to them.  A student interested in environmental issues could analyze public data sets about pollution, air quality, or water quality.  A student passionate about economics could look at unemployment rates, pay scales, or international trade.  A student interested in energy could examine subsidies for green energy companies, consumption locally and worldwide, or performance data for various alternative energy technologies.

There’s much more to say here.  But preliminarily the point is that when it comes to education and the future of an educated citizenry, basic literacy about how to understand data is more important than ever before.