Three years ago, in a New York Times article detailing her bid to become head of the American Federation of Teachers union, Randi Weingarten boasted that despite my calls for “radical reform” to New York City’s school system, Mayor Michael Bloomberg and I had achieved only “incremental” change. It seemed like a strange thing to crow about, but she did have something of a point. New York over the past nine years has experienced what Robert Schwartz, the academic dean of Harvard’s education school, has described as “the most dramatic and thoughtful set of large-scale reforms going on anywhere in the country,” resulting in gains such as a nearly 20-point jump in graduation rates. But the city’s school system is still not remotely where it needs to be. 

That story holds more than true for the country at large. Nearly three decades after A Nation at Risk, the groundbreaking report by the National Commission on Excellence in Education, warned of “a rising tide of mediocrity that threatens our very future as a Nation and a people,” the gains we have made in improving our schools are negligible—even though we have doubled our spending (in inflation-adjusted dollars) on K–12 public education. On America’s latest exams (the National Assessment of Educational Progress), one-third or fewer of eighth-grade students were proficient in math, science, or reading. Our high-school graduation rate continues to hover just shy of 70 percent, according to a 2010 report by the Editorial Projects in Education Research Center, and many of those students who do graduate aren’t prepared for college. ACT, the respected national organization that administers college-admissions tests, recently found that 76 percent of our high-school graduates “were not adequately prepared academically for first-year college courses.” 

Video: Joel Klein explains the twisted politics of New York education in a conversation with Atlantic editor James Bennet 

While America’s students are stuck in a ditch, the rest of the world is moving ahead. The World Economic Forum ranks us 48th in math and science education. On international math tests, the United States is near the bottom of industrialized countries (the 34 members of the Organization for Economic Cooperation and Development), and we’re in the middle in science and reading. Similarly, although we used to have one of the top percentages of high-school and college graduates among the OECD countries, we’re now in the basement for high-school and the middle for college graduates. And these figures don’t take into account the leaps in educational attainment in China, Singapore, and many developing countries. 

During the first three-quarters of the 20th century, America developed an enormously successful middle class, first by making high school universal, and then, after the Second World War, by making college much more available, through the GI Bill and other scholarship programs. As a result, our educational attainment kept pace with our strong technological advancement. But that’s changed markedly since 1980, and now our technological progress is advancing more rapidly than our educational attainment. From 1960 to 1980, our supply of college graduates increased at almost 4 percent a year; since then, the increase has been about half as fast. The net effect is that we’re rapidly moving toward two Americas—a wealthy elite, and an increasingly large underclass that lacks the skills to succeed. 

This division tears at the very fabric of our society. Nevertheless, there’s little national urgency to fix its underlying causes. Unlike a bad economy, poor educational achievement creeps up on us. Right now, if you were running for office, would you be more concerned with unemployment or education? Also, unlike terrorism, an educational crisis has a different impact on the powerful than it does on most of society. Their children, who are in private schools or elite public schools, receive a decent education, so it’s hard to get them fully engaged in the broader national debate. Plus, unlike in health care, for example, where we perceive the quality of care to be good and worry instead about controlling costs and covering the uninsured, in education, despite massive increases in expenditure, we don’t see improved results. That leads too many people to suspect that poverty is destiny, that schools can make only a small difference, and that therefore we’re unable to fix this problem, regardless of its seriousness. So why try? 

If the forces behind reform seem scattered and weak, those defending the status quo—the unions, the politicians, the bureaucrats, and the vendors—are well organized and well financed. Having spent eight years trying to ignite a revolution in New York City’s schools under Bloomberg’s leadership, I am convinced that without a major realignment of political forces, we won’t get the dramatic improvements our children need. 

To comprehend the depth of the problem, consider one episode that still shocks me. Starting in 2006, under federal law, the State of New York was required to test students in grades three through eight annually in math and English. The results of those tests would enable us, for the first time, to analyze year-to-year student progress and tie it to individual teacher performance—a metric known in the field as “teacher value-added.” In essence, you hold constant other factors—where the students start from the prior year, demographics, class size, teacher length of service, and so on—and, based on test results, seek to isolate the individual teacher’s contribution to a student’s progress. Some teachers, for example, move their class forward on average a quarter-year more than expected; others, a quarter-year less. Value-added isn’t a perfect metric, but it’s surely worth considering as part of an overall teacher evaluation. 

After we developed data from this metric, we decided to factor them into the granting of tenure, an award that is made after three years and that provides virtual lifetime job security. Under state law at the time, we were free to use these data. But after the New York City teachers union, the United Federation of Teachers, objected, I proposed that the City use value-added numbers only for the top and bottom 20 percent of teachers: the top 20 percent would get positive credit; the bottom would lose credit. And even then, principals would take value-added data into account only as part of a much larger, comprehensive tenure review. Even with these limitations, the UFT said “No way,” and headed to Albany to set up a legislative roadblock. 

Seemingly overnight, a budget amendment barring the use of test data in tenure decisions materialized in the heavily Democratic State Assembly. Joe Bruno, then the Republican majority leader in the State Senate, assured me that this amendment would not pass: he controlled the majority and would make sure that it remained united in opposition. Fast-forward a few weeks: the next call I got from Senator Bruno was to say, apologetically, that several of his Republican colleagues had caved to the teachers union, which had threatened reprisals in the next election if they didn’t get on board. 

As a result, even when making a lifetime tenure commitment, under New York law you could not consider a teacher’s impact on student learning. That Kafkaesque outcome demonstrates precisely the way the system is run: for the adults. The school system doesn’t want to change, because it serves the needs of the adult stakeholders quite well, both politically and financially. 

Let’s start with the politicians. From their point of view, the school system can be enormously helpful, providing patronage hires, school-placement opportunities for connected constituents, the means to get favored community and business programs adopted and funded, and politically advantageous ties to schools and parents in their communities. 

During my maiden testimony before the State Assembly, I said that we would end patronage hires, which were notorious under the old system of 32 school districts, run by 32 school boards and 32 superintendents (a 2002 state bill granting Bloomberg mayoral control of the city’s schools abolished the 32 boards). At my mention of patronage, the legislators, like Captain Renault in Casablanca, purported to be “shocked.” Nevertheless, after the hearing, when I went to thank committee members, one took me aside and said: “Listen, they’re trying to get rid of a principal in my district who runs a Democratic club for us. If you protect him, you’ll never have a problem with me.” This kind of encounter was not rare. 

Similarly, I faced repeated requests for “constituent services,” meaning good school placements for wired constituents. After we reorganized the system and minimized the power of the 32 local superintendents—the go-to people for politicians under the past regime—a local official called me and asked, “Whom do I call for constituent services after your reorg?” I replied, “What’s that?” Impatiently, he asked, “How do I get a kid into a school when I need to?” I jokingly answered, “Oh, we must have left out that office in the reorg” (actually thinking, silly me, that the school system should use equitable rules for admission). He said, “Go fuck yourself,” and hung up. Despite our constant efforts, or because of them, this kind of political pressure—and payback if we weren’t responsive—happened at every level. Even more important, politicians can reap enormous political support from the unions representing school employees. The two national unions—the American Federation of Teachers and the National Education Association—together have some 4.7 million members, who pay hundreds of millions of dollars in national, state, and local dues, much of which is funneled to political causes. Teachers unions consistently rank among the top spenders on politics. 

Moreover, millions of union members turn out when summoned, going door-to-door, staffing phone banks, attending rallies, and the like. Teachers are extremely effective messengers to parents, community groups, faith-based groups, and elected officials, and the unions know how to deploy them well. And just as happy unions can give a politician massive clout, unhappy unions—well, just ask Eva Moskowitz, a Democrat who headed the City Council Education Committee when I became chancellor in 2002. Brilliant, savvy, ambitious, often a pain in my neck, and atypically fearless for an elected official, she was widely expected to be elected Manhattan borough president in 2005. Until, that is, she held hearings on the New York City teachers-union contract—an extraordinary document, running on for hundreds of pages, governing who can teach what and when, who can be assigned to hall-monitor or lunchroom duty and who can’t, who has to be given time off to do union work during the school day, and so on. Truth is, the contract defied parody. So when Moskowitz exposed its ridiculousness, the UFT, then headed by Randi Weingarten, made sure that Moskowitz’s run for borough president came up short. After that, other elected officials would say to me, “I agree with you, but I ain’t gonna get Eva’d.” 

In short, politicians—especially Democratic politicians—generally do what the unions want. And the unions, in turn, are very clear about what that is. They want, first, happy members, so that those who run the unions get reelected; and, second, more members, so their power, money, and influence grow. As Albert Shanker, the late, iconic head of the UFT, once pointedly put it, “When schoolchildren start paying union dues, that’s when I’ll start representing the interests of schoolchildren.” And what do the members want? Employees understandably want lifetime job security (tenure), better pay regardless of performance (seniority pay), less work (short days, long holidays, lots of sick days), and the opportunity to retire early (at, say, 55) with a good lifetime pension and full health benefits; for their part, the retirees want to make sure their benefits keep coming and grow through cost-of-living increases. The result: whether you work hard or don’t, get good results with kids or don’t, teach in a shortage area like math or special education or don’t, or in a hard-to-staff school in a poor community or not, you get paid the same, unless you’ve been around for another year, in which case you get more. Not bad for the adults. 

But it’s just disastrous for the kids in our schools. While out-of-school environment certainly affects student achievement, President Obama was on to something in 2008 when he said: “The single most important factor in determining [student] achievement is not the color of [students’] skin or where they come from. It’s not who their parents are or how much money they have. It’s who their teacher is.” Yet, rather than create a system that attracts and rewards excellent teachers—and that imposes consequences for ineffective or lazy ones—we treat all teachers as if they were identical widgets and their performance didn’t matter. 

In fact, notwithstanding union rhetoric that “tenure is merely due process,” firing a public-school teacher for non-performance is virtually impossible. In New York City, which has some 55,000 tenured teachers, we were able to fire only half a dozen or so for incompetence in a given year, even though we devoted significant resources to this effort. 

The extent of this “no one gets fired” mentality is difficult to overstate—or even adequately describe. Steven Brill wrote an eye-opening piece in The New Yorker about the “rubber rooms” in New York City, where teachers were kept, while doing no work, pending resolution of the charges against them—mostly for malfeasance, like physical abuse or embezzlement, but also for incompetence. The teachers got paid regardless. (To add insult to injury, these cases ultimately were heard by an arbitrator whom the union had to first approve.) Before we stopped this charade—unfortunately by returning many of these teachers to the classroom, as the arbitrators likely would have required—it used to cost the City about $35 million a year. 

In addition, more than 1,000 teachers get full pay while performing substitute-teacher and administrative duties because no principal wants to hire them full-time. This practice costs more than $100 million annually. 

Perhaps the most shocking example of the City’s having to pay for teachers who don’t work involves several teachers accused of sexual misconduct—including at least one who was found guilty—whom the union-approved arbitrators refuse to terminate. Although the City is required to put them back in the classroom, it understandably refuses to do so. And the union has never sued the City to have these teachers reinstated, even though it knows it could readily win. It has also never helped figure out how to get these deadbeats off the payroll, where they may remain for decades at full pay, followed by a lifetime pension. No one—and the union means no one—gets fired. 

Next, consider the consequences of the ubiquitous practice of paying the same for math and physical-education teachers. Given the other job opportunities for talented mathematicians—but not for phys-ed teachers—the same salary will attract many more of the latter than the former. It’s simple supply and demand. But when you’re short of qualified math teachers—as virtually every major urban school district is—poor kids with the greatest needs invariably get cheated, because most teachers prefer to teach highly motivated kids who live in safe communities, and whose parents will contribute private money to the school. The result: too few effective math and science teachers in high-poverty schools. 

Finally, coming on top of these other senseless policies is the remarkable way that benefits and seniority drive overall teacher compensation. It’s possible for a teacher in New York City to retire at 55 and draw down an annual pension of more than $60,000, plus lifetime health benefits for herself and her family. The pension is not subject to New York State or local taxes and goes up with cost-of-living increases. The huge value of this lifetime stream of benefits is rarely mentioned when we talk about teachers’ compensation, but the teachers are well aware of it and act rationally in response to it. What we end up with is both a form of lock-in for employees and an enormous long-term financial exposure for the taxpayers. 

The impact of the lock-in shapes the entire compensation system, because the “big” money comes only after a certain number of years—in New York City, for example, many teachers get their full pension after working 25 years, and a far smaller pension if they work for only 24 years. As a result of backloaded policies like this, after 10 years fewer than 1 percent of teachers leave the system, and after 15 years only about 0.1 percent leave. Many have candidly told me they are burned out, but they can’t afford to leave until their pension fully vests. So they go through the motions until they can retire with the total package. 

Aggravating the perverse incentive of the benefit lock-in is the nature of almost all pay increases in public education, which are either automatic if you stay another year or so, or take 30 college credits; or across-the-board percentage raises—for example, 10 percent over three years, meaning that every veteran teacher making $80,000 gets an $8,000 increase, while every beginning teacher making $40,000 gets a $4,000 increase. 

None of these pay increases makes sense. Why pay someone more for simply working another year or for taking a few courses? Starting last year, Mayor Bloomberg refused to give teachers in New York a raise, because he was facing budget cuts. But the overall pay for teachers still went up nearly 3.5 percent automatically, simply for longevity and college credits. (According to a Department of Education internal analysis, the average NYC teacher works fewer than seven hours a day for 185 days and costs the city $110,000—$71,000 in salary, $23,000 in pensions, and $16,000 in health and other benefits.) And why give all teachers making $80,000, or more, a 10 percent raise? They’re not going to leave, since they’re close to vesting their lifetime pensions. By contrast, increasing starting salaries by $8,000 (rather than $4,000) would help attract and retain better new teachers. But because of seniority, we can’t do it that way. 

Now consider the financial burden that comes with providing lifetime benefits. Given the time between first putting aside the money to fund such a “long-tail exposure” and having to begin paying it, the amount “reserved” by the employer necessarily depends on a host of imprecise assumptions—about the rate of return that the money invested in the pension fund will earn, about how long employees will live, and even about how much overtime employees will work during their last few years, which is normally included in calculations of the amount of the pension. Each dollar set aside this year to cover the ultimate pension exposure must be taken from what would otherwise be current operating dollars. 

Consequently, elected officials have had every incentive to make extraordinarily optimistic assumptions about the pension plan—or to simply underfund it—so they can put as little as possible into the reserve. Unfortunately, but predictably, that’s exactly what has happened: most states “assumed” they would get an average 8 percent return on their pension reserves, when in fact they were getting significantly less. Over the past 10 years, for example, New York City’s pension funds earned an average of just 2.5 percent. Now virtually every pension plan in America that covers teachers has huge unfunded liabilities. A recent study by the Manhattan Institute estimated the total current shortfall at close to $1 trillion. There’s only one way to pay for that: take the money from current and future operating budgets, robbing today’s children to pay tomorrow’s pensions. In NYC, for example, the portion of the overall budget set aside for education pensions went from $455 million in 2002 to $2.6 billion in 2011, most of it for teachers. Not surprisingly, retirees remain politically vigilant, and vote at much higher levels than active teachers in union elections (50 percent versus 24 percent in New York’s last UFT election). 

During my tenure, I fought to break this institutional stranglehold of defenders of the status quo. I did so because I believed that our kids are not getting the education they deserve, that we have clear examples showing dramatically better results, and that we won’t achieve those results if we just keep tinkering. Since 2007, my colleague Michelle Rhee, in Washington, D.C., has been making the same noises. The response, often from friends as well as opponents, was that we were unrealistic: complex systems don’t change easily, impatience is immature, and directly challenging the educational establishment is not a winning strategy. “You need to be more collaborative and less controversial,” we were repeatedly admonished. 

That’s bad advice. Collaboration is the elixir of the status-quo crowd. Consider one of the most cherished mantras in public education today—“We’ll never fix education until we fix poverty.” This lets the school system off the hook: “We can’t do too much with these poor kids, so don’t blame us (but give us more money).” Sure, money, a stable family, and strong values typically make educating a child easier. But we also now know that, keeping those things constant, we can get dramatically different outcomes with the same kid, based on his or her education. Texas and California, for example, have very similar demographics. Nevertheless, even though Texas spends slightly less per pupil than does California, it outperforms California on all four national tests, across demographic groups. The gap is around a year’s worth of learning. That’s big. And the gaps are even bigger when we compare similar demographic groups in large urban districts. Low-income black students in Boston or New York, for example, are several years ahead of those in Detroit or Los Angeles on the national exams. 

At the individual school level, the differences can be breathtaking. One charter school in New York City, Harlem Success Academy 1, has students who are demographically almost identical to those attending nearby community and charter schools, yet it gets entirely different results. Harlem Success has 88 percent of its students proficient in reading and 95 percent in math; six other nearby schools have an average of 31 percent proficient in reading and 39 percent in math. And according to the most-recent scores on New York State fourth-grade science tests, Success had more than 90 percent of its students at the highest (advanced) level, while the city had only 43 percent at advanced, and Success’s black students outperformed white students at more than 700 schools across the state. In fact, Success now performs at the same level as the gifted-and-talented schools in New York City—all of which have demanding admissions requirements, while Success randomly selects its students, mostly poor and minority, by lottery. 

These school-level differences ultimately reflect the effectiveness of a child’s particular teachers. Eric Hanushek, an economist at Stanford, has shown that, while some teachers get a year and a half’s worth of learning into a year, others get in only half a year’s worth of learning with essentially the same students. Imagine the cumulative impact of the best teachers over 13 years of elementary and secondary education. Indeed, even if California raised its performance to Texas’s level, Detroit to Boston’s, the neighborhood schools in Harlem to Harlem Success’s—that is to say, if our least effective teachers performed at the level of our most effective—the impact would be seismic. 

Critics are strangely eager to discredit these differences. Writing last year in The New York Review of Books, the educational historian Diane Ravitch argued that schools like Harlem Success aren’t the answer, because, as a group, charter schools in the U.S. don’t outperform public schools. To make her case, Ravitch relied on a study by Margaret Raymond at Stanford; but curiously, Ravitch failed to mention that Raymond applied precisely the same analysis to New York City (where the school district was atypically supportive of charters), and found that charter schools there were getting significantly better reading and math results with their students than were comparable traditional public schools. And even Ravitch had to acknowledge that some charter schools are getting “amazing results.” If that’s the case, then instead of relying on the kind of group-think that pits charter schools against non-charter schools, shouldn’t we be asking why some schools get much better results, and focus on how we can replicate them? 

Several recent developments offer some hope. In the past year, 42 states, as well as the District of Columbia and the U.S. Virgin Islands, have agreed to adopt a new set of highly demanding core standards in English and math to replace the current state-by-state standards. The Obama administration has also granted two consortia of states $330 million to design tests aligned with these new standards. As a result, we’ll have a more realistic sense of how our kids are performing, we’ll be able to compare kids in one jurisdiction with those in another, and, most important, we’ll know that kids who graduate from high school will actually be prepared for college. 

But we still won’t get to where we need to go unless we’re prepared to do three difficult, but essential, things: rebuild our entire K–12 system on a platform of accountability; attract more top-flight recruits into teaching; and use technology very differently to improve instruction. 

Surprisingly enough, the best case for greater accountability was made by Albert Shanker, four years before he died, in his capacity as the leader of the American Federation of Teachers. In a truly remarkable speech to the 1993 Pew Forum on Education Reform, which I’ve never seen quoted by any teachers-union official since, Shanker said: 

The key is that unless there is accountability, we will never get the right system. As long as there are no consequences if kids or adults don’t perform, as long as the discussion is not about education and student outcomes, then we’re playing a game as to who has the power.

Two points are critical here. First, Shanker makes clear that accountability needs to be measured by “student outcomes,” which he goes on to explain must be based on progress on standardized tests. And second, he calls out the fundamental truth about the system: because it’s not anchored to outcomes, it ends up being about “who has the power,” which can then be used to serve other agendas—such as better pay, political support, or vendor contracts. 

Accountability, in most industries or professions, usually takes two forms. First and foremost, markets impose accountability: if people don’t choose the goods or services you’re offering, you go out of business. Second, high-performing companies develop internal accountability requirements keyed to market-based demands. 

Public education lacks both kinds of accountability. It is essentially a government-run monopoly. Whether a school does well or poorly, it will get the students it needs to stay in business, because most kids have no other choice. And that, in turn, creates no incentive for better performance, greater efficiency, or more innovation—all things as necessary in public education as they are in any other field. 

A full-scale transition from a government-run monopoly to a competitive marketplace won’t happen quickly. But that is no reason not to begin introducing more competition. Many middle-class families have plenty of choice (even beyond private schools): they can move to another neighborhood, or are well-connected enough to navigate the system. Those families who are least powerful, however, usually get one choice: their neighborhood school. That has to change. 

In the lower grades, we should make sure that every student has at least one alternative—and preferably several—to her neighborhood school. We implemented this strategy by opening more than 100 charter schools in high-poverty communities. Tellingly, almost 40,000 families chose these new schools, and another 40,000 are on waiting lists. The traditional schools, as well as their employees and the unions, are screaming bloody murder, something vividly depicted in The Lottery, a recent documentary that shows community agitators brought in by the union to oppose giving public-school space to the Harlem Success network. But this kind of push-back is actually a good sign: it means that the monopolists are beginning to feel the effects of competition. 

At the middle- and high-school levels, where students are more mobile, we can also create community-based choice systems, or even citywide choice systems. In New York City, for example, high-school students now have citywide choice (with some geographic priority), and schools know they have to recruit—and compete for—students. 

To support effective choice, moreover, we need to provide real funding equity: the money must be for the child, not the school. So if Juan goes to PS 11, which gets $20,000 as a result, then that same $20,000 must go to a KIPP charter school if Juan decides to go there. Similarly, capital funds, or space within a school building, must also follow the child—either to PS 11 or to KIPP—on equitable terms. 

Unfortunately, the likelihood of rapidly expanding choices remains small. Witness, for example, those 40,000 families wait-listed for charter schools in New York City. By the time the City opens another 100 schools to meet that demand, at least another 40,000 families will likely be waiting. And now that the union and its allies have seen the smashing impact of the first 100 charter schools, they won’t make it any easier to open the next 100. 

That’s why internal accountability along the lines that Shanker discussed is critical. School districts need a system to fairly evaluate the effect of schools and teachers on kids, which is the best proxy we have for assessing “consumer preference” in a largely monopolistic system. Shanker also had the right idea about how to measure outcomes: by looking at student progress on apples-to-apples metrics, rather than at whether students do well or poorly against an absolute, static index. On a four-point scale, for example, a teacher deserves credit for moving a kid from a 1 to a 2 and should lose credit for letting another kid fall from a 4 to a 3, even though a 3 is better than a 2 in an absolute sense. Some kids come to school way ahead of others, and giving the school or teacher credit for that makes no sense. But if schools or teachers have essentially the same kids, with the same challenges, and the same starting performance levels, it’s pretty easy to measure which are helping the kids make progress and which aren’t. 

Finally, as Shanker emphasized, meaningful teacher accountability means major consequences for student outcomes. Those teachers and principals whose students do well should get substantial merit pay; those who don’t should be fired. Similarly, schools that do poorly should be replaced. Without real consequences tied to performance, the results won’t significantly change. Again, resistance to this kind of accountability is always fierce. In New York, we closed many large, overwhelmingly minority high schools that were posting abysmal graduation rates—some even below 40 percent—and replaced them with new, small high schools. Although research showed that the new schools were getting significantly better results, I wasn’t surprised when the teachers union sued us to block future closures—they want to protect their members. But I was shocked when the NAACP joined the suit. How could it defend schools that were consistently graduating fewer than half their African American children? 

Despite the setbacks, we are seeing progress. In response to President Obama’s $4.3 billion Race to the Top Fund, which requires states to compete for big federal grants, and rewards accountability systems that measure whether teachers add value, several states—including Colorado, Florida, Louisiana, New Mexico, and Ohio—have enacted legislation moving in this direction. Under Michelle Rhee’s leadership, Washington, D.C., adopted the best of these systems with the agreement of its local and national teachers unions, including the union headed by Randi Weingarten. The District was authorized to award substantial merit pay (resulting in salaries of up to $130,000) and to fire teachers who were not performing well. Rhee fired more than 200 of them. 

But although Weingarten’s union had agreed to the contract, it reportedly spent $1 million and mobilized huge numbers of volunteers to defeat Washington’s mayor, Adrian Fenty, when he was up for reelection two months later. That intervention surely sent a message to other reformers throughout the country: we unions talk reform, but firing incompetent teachers will never be a real part of that. 

The second big thing we need to change is the people we attract into teaching. When McKinsey and Company compared educational performance around the world, it came to the seemingly obvious, yet often disputed, conclusion that “the quality of an education system cannot exceed the quality of its teachers”: 

The top-performing school systems [internationally] attract more able people into the teaching profession, leading to better student outcomes … The top-performing systems we studied recruit their teachers from the top third of each cohort [that graduates] from their school system … Conversely, lower-performing school systems rarely attract the right people into teaching. The New Commission on the Skills of the American Workforce observes that, “We are now recruiting our teachers from the bottom third of high-school students going to college.”

By recruiting teachers mostly from the middle and bottom of their college classes, as America has done for decades now, not only did we not get the talent we needed, but we also fostered a culture where excellence and merit don’t matter. 

A rational compensation scheme is critical to fixing this core human-capital weakness: rather than just pay for longevity and lifetime benefits, we must reward excellence and enable the system to meet its needs. If, going forward, we eliminated all the automatic raises and promises of huge lifetime benefits, we’d have an enormous amount of money to devote to merit pay, hardship-assignment incentives, and recruiting in subjects where we have shortages. If we could front-load compensation, new teachers could get as much as $80,000 by year three or four. This would make a huge difference. If you have any doubt, just ask the talented, ambitious young teachers who come through Teach for America or comparable programs. Many leave well before they peak, even though they like the work, because their pay remains quite low in the early years—up to about $55,000—and they are unwilling to commit to staying around for 25 years to cash in on the back-loaded pay structure. 

I once proposed a portion of this—simply eliminating the lifetime, defined-benefit pension, monetizing the savings, and then paying it to teachers in their early years—in a conversation with union officials. I was prepared to give each new teacher a choice between the current pay scale (with the existing pension) and this new pay proposal. Although no teacher would have been compelled to switch, the UFT rejected the idea as “anti-union.” But we have evidence to show that these monetary incentives can work. In Washington, D.C., Michelle Rhee negotiated a merit-based compensation system—where teachers could get paid much more in the early years. As a result, it appears, significant numbers of teachers from D.C.’s charter schools apply to teach in its traditional public schools. Only money can explain that. 

Of course, another way to attract and retain very effective teachers would be to create more schools that work. In my experience, many of the best public-school teachers apply to high-functioning charter schools, even though they usually give up job security, and lifetime health care and pensions, while generally getting a similar or slightly higher salary (although often augmented by modest merit pay). They go because they want to be part of a successful school, where teachers are treated like professionals and not subjected to endless administrative and union micromanagement. 

Last, to shake up the system, we must change how we use technology to deliver instruction. (This is what I’m now seeking to do at News Corporation.) The present resistance to innovation is breathtaking. Consider this story: When we replaced many large, failing high schools with more, much smaller schools, many of the new schools had only a handful of kids who wanted to take rigorous Advanced Placement courses, which can earn students college credit. Several good online programs teach the necessary course content. But in New York state, you cannot get high-school credit unless you’re taught by a live teacher (a requirement referred to as “seat time”), and these small schools didn’t have enough students to bring in an AP teacher. I approached our State Education Department in Albany, which had the authority to waive the seat-time requirement: if a kid could get college credit for passing an online AP course, surely she should be able to get high-school credit as well. 

As soon as the UFT heard that we had requested a waiver from the state, it faxed us a letter saying, “The elimination of seat-time requirements needs to be negotiated,” making clear that if we tried to proceed, this would be war. You see, if we opened the door to online AP courses, maybe we’d end up needing fewer teachers, and that wouldn’t be good for union membership, dues, or power. I got nowhere. 

But one of the best things we could do is hire fewer teachers and pay more to the ones we hire. And, as in any other field, technology can help get us there. If you have 5,000 math teachers, many of whom are underperforming, significantly improving overall quality is nearly impossible. But if you get the best math professors in the world—who are great teachers and who deeply understand math—and match them with great software developers, they can create sophisticated interactive programs that engage kids and empower teachers. Why not start with such a program and then let teachers supplement it differently, depending on the progress of each student? 

That’s a whole lot easier than trying to teach the same math lesson to 30 kids, some of whom are getting it quickly and some of whom aren’t getting it at all. We now have multiple ways to teach the same lessons. As a result, we can tailor both the means and the pacing to each student. We can use digital games where kids progress based on solving increasingly difficult math problems, virtual classes that kids can take online, and tutors whom kids can work with online, as well as, of course, teachers working with large or small groups in person. The possibilities are enormous. We should be trying them all and constantly improving how we do the work. That’s exactly what New York City is doing in a pilot program called the School of One, which was designed to move from the classroom as the locus of instruction to the individual student as the focus of instruction. 

More broadly, we need to foster a fundamental shift from a top-down, one-size-fits-all culture—mandated class-size reduction, after-school programs, and the like—to a culture that supports innovation. In New York City, we set out to change these preexisting dynamics by allowing educators and community groups—rather than the central bureaucracy—to design and run new schools to replace the failing ones. The result was a lot of innovation. For example, New York City is now piloting something called the Generation School, which uses staff time very differently and thus extends the school day and year significantly. Last year, the City also opened something called the New American Academy, where four teachers are collectively responsible for educating 60 kids, and they stay with those children from kindergarten through the fifth grade. The teachers are categorized as Master, Partner, Associate, and Apprentice, and they are paid very differently and get promoted from one level to the next based on performance as well as peer and supervisory review. 

Change is possible. In New York City, it took a mayor willing to assume control over the system and risk significant political capital. It required time—Mayor Bloomberg and I had more than eight years together, while most urban superintendents serve for about three and a half years. It required taking risks, knowing that not every change will work out and that your critics will focus mercilessly on those that don’t. But most of all, it required building community and political support. Toward the end of my tenure, we were engaged in an enormous fight to lift the state-imposed cap on our number of charter schools—an initiative the teachers unions strongly opposed precisely because our expansion of charter schools had been so successful. In fact, six months earlier, a similar effort had gone down to defeat at the unions’ hands. But this time, the families with kids in charter schools and our allies in the community were prepared to help us fight. Philanthropic and business interests raised millions to support the mobilization effort, run ads, and hire lobbyists. We prevailed, and the cap was raised substantially. 

Sadly, that kind of success is still exceptional. In the three decades since A Nation at Risk came out, many have echoed its cries of alarm, but few have heeded its calls for bold change. Indeed, in his 1993 Pew Forum speech, Al Shanker spoke in shockingly candid terms: 

We are at the point that the auto industry was at a few years ago. They could see they were losing market share every year and still not believe that it really had anything to do with the quality of the product I think we will get—and deserve—the end of public education through some sort of privatization scheme if we don’t behave differently. Unfortunately, very few people really believe that yet. They talk about it, and they don’t like it, but they’re not ready to change and stop doing the things that brought us to this point.

Time is running out. Without political leadership willing to take risks and build support for “radical reform,” and without a citizenry willing to insist on those reforms, our schools will continue to decline. And just as it was with Detroit, the global marketplace will be very unforgiving to a populace that doesn’t have the skills it demands. McKinsey estimates that the benefits of bringing our educational levels up to those of the highest-performing countries would have raised our gross domestic product by about $2 trillion in 2008. By the same token, every year we fail to close that gap is like living with the equivalent of a permanent national recession. Shocking as that may sound, the costs in human terms, to our nation and to the kind of people we aspire to become, will be even greater.