The Path to the Final Four

It's time to gear up for the Greatest Show on Earth. There's a lot to process: historical trends and computer rankings and figuring out what Seth Greenberg said. Bracket Which Cinderellas will pull off the upsets. What top seed will fall flat on its face?

Most importantly, what does it take to reach the Final Four?

That last one might be particularly pressing for Duke fans --
how come Duke made the Final Four seven times in nine years from 1986 to 1994 but only once in nine years from 2005 to 2013. Did those early teams have some special something Duke's more recent teams lacked?

We'll get Duke-specific later. For now, an evaluation of the past 29 NCAA tournaments (since 1985, the year the tourney went to 64 teams) suggests the most important "something" is probably a factor most people haven't spent all that much time thinking about. Perhaps because the teams themselves have no control over it; possibly because it isn't knowable in advance. So what is this magic factor that has so much control over which teams makes the Final Four and which don't?

Who the teams play.

Not who they're supposed to play, but the actual path facing a team on its journey to the promised land. For example, last season the expected path for 2nd seeded Duke was to play the following seeds in the following order: 15-7-3-1. And that's exactly the seeds Duke played. Similarly, 1-seeds face an expected path of 16-8-4-1. And assuming for the moment that 8-seeds and 9-seeds are equivalent for our current purposes, that's the exact path three of last season's four 1-seeds took (though they each ended the trip at a different point). The only 1-seed that took a different path? That would be Louisville, which played 12-seed Oregon instead of 4-seed St. Louis in the Sweet 16. Louisville also happened to be the only 1-seed to make the Final Four. Coincidence?

Maybe. But the fact is that top-four seeds (teams seeded 1, 2, 3, or 4) have run "chalk" (their expected path) a total of 212 times since 1985 and have made the Final Four only 17 of those times (8.0%). When those same teams have played at least one game against earlier upset winners ("non-chalk"), they've made the Final Four 82 out of 252 chances (32.5%). Four times more likely would be a pretty big coincidence.

Let's break it down further. Here's how the different seeds have fared since 1985 (with 5-seeds thrown in for good measure):

Seed Chalk Played Chalk Final Fours Chalk % Success Non-Chalk Played Non-Chalk Final Fours Non-Chalk % Success
1 (vs. 8-seeds only) 28 8 28.6% 88 39 44.3%
1 (8/9 seeds equivalent) 48 12 25.0% 68 35 51.5%
2 43 4 9.3% 73 21 28.8%
3 65 3 4.6% 51 11 21.6%
4 76 2 2.6% 40 11 27.5%
5 97 0 0.0% 19 6 31.6%

In fairness, the above figures include first-round upsets, because technically, when a 14-seed beats a 3, the 3-seed played chalk. So, here's what the table looks like for teams that managed to beat their first round opponent:

Seed Chalk Played Chalk Final Fours Chalk % Success Non-Chalk Played Non-Chalk Final Fours Non-Chalk % Success
1 (vs. 8-seeds only) 28 8 28.6% 88 39 44.3%
1 (8/9 seeds equivalent) 48 12 25.0% 68 35 51.5%
2 36 4 11.1% 73 21 28.8%
3 48 3 6.3% 51 11 21.6%
4 51 2 3.9% 40 11 27.5%
5 56 0 0.0% 19 6 31.6%

Slightly different percentages, but the song remains the same. Even without counting first round upsets, top seeds are twice as likely to make the Final Four if one of their expected opponents falls prey to an upset. 2-seeds are almost three times more likely, 3-seeds almost five times more likely, and 4-seeds more than six times more likely. And no 5-seed has ever made the Final Four through a chalk path.

We tend to think of upsets as huge surprises. We know they do happen, of course, but in the NCAA tournament we pretty much always expect the favorite to win, or if it doesn't we think the favorite was flawed, not as good as advertised. Except that's not the way things work, either statistically or in real life.

If a team is favored to beat another team nine times out of ten, while the favorite could certainly win ten games in a row, the expectation would be that team would lose one game in a ten game series. That one could be the first game in the series, the last game, or any game in between. Or it may not happen at all. Put another way, if Team A had a 90% chance to beat team B, and the teams played ten games, the probability that Team A would win all ten games would be: (0.9)*(0.9)*(0.9)*(0.9)*(0.9)*(0.9)*(0.9)*(0.9)*(0.9)*(0.9), which equals 34.9%. The odds would be nearly 2 to 1 against Team A winning all ten.

In a one-and-done situation, who knows if and when the "one" will come up?

A more pertinent example would be to apply this idea to the NCAA tournament. Using Ken Pomeroy's rating system to determine the probabilities, last season Duke had the following chance to make the Elite Eight:

0.9358 (chance of beating Albany) * 0.6427 (chance of beating Creighton) * 0.5562 (chance of beating Michigan State).

Multiplying it out, Duke's probability of winning those three games came to 33.45%. Again, a 2 to 1 chance against (although fortunately Duke beat the odds).

On the other side of the 2013 Midwest bracket, Louisville had a 63.5% chance to reach the Elite Eight, thanks in large part to a weaker set of expected opponents. Then Louisville got another boost when 12-seed Oregon beat the region's 4-seed and 5-seed to face Louisville in the Sweet 16, upping Louisville's Elite Eight chances to 70.1%. Better than 2 to 1 in favor. If you'd switched Duke and Louisville, Duke's chances against Louisville's opponents would have been 51.6%, while Louisville's chances against Duke's opponents would have been 54.3%. Essentially two coin flips. The actual path matters.

Employing the same method using the last five years of pre-Tournament Pomeroy data, averaging for each seed, here are the expected probabilities of each top-8 seed winning its first game:

1-seed: 96.2% (actual performance over 29 years: 100%)
2-seed: 92.1% (actual over 29 years: 94.0%)
3-seed: 81.7% (actual over 29 years: 85.3%)
4-seed: 79.5% (actual over 29 years: 78.5%)
5-seed: 64.6% (actual over 29 years: 64.7%)
6-seed: 58.2% (actual over 29 years: 66.4%)
7-seed: 54.7% (actual over 29 years: 60.3%)
8-seed: 56.1% (actual over 29 years: 48.3%)

So when people joke about how 12-seeds always beat 5-seeds, the answer is they beat them almost exactly the amount you'd expect based on the probabilities. If anything (other than 9s beating 8s), we get slightly fewer upsets than we should.

Expanding to all games in a seed's expected path (using the same five years of Pomeroy pre-Tournament data), the chance of each top-4 seed to advance in the tournament is as follows:

1-seed: 73.4% chance of making Sweet 16; 48.9% chance of making Elite Eight; 30.5% chance of making Final Four;
2-seed: 63.4% chance of making Sweet 16; 36.5% chance of making Elite Eight; 15.4% chance of making Final Four;
3-seed: 49.6% chance of making Sweet 16; 22.7% chance of making Elite Eight; 8.2% chance of making Final Four;
4-seed: 47.1% chance of making Sweet 16; 16.6% chance of making Elite Eight; 8.0% chance of making Final Four;

Even for the best teams, the odds are against them. The team in the last five years with the highest probability against its expected path to reach the Final Four was 2010 Duke, with a 47.1% chance, but even that team was more likely than not to miss out on the Final Four. Which is why the early round upsets are so important, because like Louisville last year, playing a 12 instead of a 4 can really boost your odds.

But does it really work like that in real life? Let's take a closer look:

AFTER FIRST ROUND

Seed After Facing # games # Final Fours Pct.
1 8 56 22 39.2%
1 9 60 25 41.7%
2 7 67 17 25.3%
2 10 42 8 19.0%
3 6 63 8 12.7%
3 11 36 6 16.7%
4 5 61 7 11.5%
4 12 30 6 20.0%

AFTER SECOND ROUND

Seed After Facing # games # Final Fours Pct.
1 4 43 17 39.5%
1 5 35 15 42.9%
1 >5 23 15 65.2%
2 3 36 10 27.8%
2 6 28 9 32.1%
2 >6 11 6 54.5%
3 2 37 8 21.6%
3 >2 23 6 26.1%
4 1 43 10 23.3%
4 >1 8 3 37.5%

AFTER THIRD ROUND

Seed After Facing # games # Final Fours Pct.
1 2 39 19 48.7%
1 3 20 12 60.0%
1 >3 21 16 76.2%
2 1 39 20 51.3%
2 >1 15 5 33.3%
3 1 20 8 40.0%
3 >1 10 6 60.0%
4 2 or 3 11 7 63.6%
4 >3 7 6 85.7%

In almost every case, having your expected path derailed by an upset winner increases your chances to reach the Final Four. And not surprisingly, the bigger the upset, the more your odds go up. Not really rocket science, is it?

So that's all well and good, but what about Duke? Can we really explain 7 Final Fours in 9 years vs. 1 Final Four in 9 years based on paths and "chalk"?

Maybe we can.

From 1985 to 1994, Duke played "chalk" only twice. The 1992 team was dominant enough to get through the chalk path to the Final Four (although it took a Christian Laettner miracle to do it) but the 1993 team missed the Final Four. That means the team played 8 "non-chalk" paths in the time period. Not counting the 1987 team that wasn't a top-four seed, the team made the Final Four through 6 of those 7 easier paths. From 1995 to 2004, Duke failed to make the tourney once and once wasn't a top-four seed, but all 8 of the other years were non-chalk paths, leading to 3 Final Fours. Playing chalk only twice in 17 opportunities was remarkably lucky.

That luck seemed to run out in the 2005 to 2013 period. Not counting the 2007 team that wasn't a top-four seed, the team played 5 chalk paths out of 8, and failed to reach the Final Four through any of them. Of the 3 non-chalk paths in the period, Duke made one Final Four. And two of the non-chalk teams were only non-chalk because Duke played a 5-seed instead of a 4-seed.

Overall, Duke has played 7 chalk paths and reached the Final Four once out of 7 (14.3%) and played 18 non-chalk paths and made the Final Four in 10 of those (55.6%). How does that compare to other schools?

Counting teams that have either been a top-four seed five or more times or have made a Final Four from a top-four seed, such teams have performed as follows:

Chalk paths*: 176; 21 reached Final Four (11.9%)
Non-chalk paths: 198; 77 reached Final Four (38.9%)

* - For the purposes of this subsection, we're counting 8-seeds and 9-seeds as equivalent.

Duke slightly outperformed such teams along chalk paths and significantly outperformed them along non-chalk paths. It's also worth noting that of the 21 teams that have gotten through chalk paths to the Final Four, six of those teams won the national championship (including Duke's 1992 team), and a seventh team, 1985 Georgetown was totally dominant until it ran into Villanova and perhaps the biggest upset in the history of the NCAA tournament. Maybe some teams are so much better than everyone else that the chalk doesn't bother them? Who knows?

The above grid counted your average Joe. How's it look if we only take the "elite"? Well, counting teams that have had a 4-seed or better in 10 or more tournaments, the performance grid looks like this:

Chalk paths: 116; 15 reached Final Four (12.9%)
Non-chalk paths: 132; 57 reached Final Four (43.2%)

Not very different, relative to Duke. Here's a detailed grid of the 17 "elite" teams that comprise the latter grid:

Team Chalk Played Chalk Final Fours Chalk % Success Non-Chalk Played Non-Chalk Final Fours Non-Chalk % Success
Duke 7 1 14.3% 18 10 55.6%
Arizona 8 1 12.5% 8 3 37.5%
Connecticut 2 0 0.0% 11 4 36.4%
Georgetown 6 1 16.7% 5 1 20.0%
Illinois 9 1 11.1% 2 1 50.0%
Indiana 8 1 12.5% 2 1 50.0%
Kansas 14 2 14.3% 10 5 50.0%
Kentucky 5 3 60.0% 13 3 23.1%
Louisville 5 0 0.0% 7 4 57.1%
Michigan 5 0 0.0% 5 3 60.0%
Michigan State** 10 1 10.0% 5 5 100.0%
North Carolina 10 3 30.0% 11 5 45.4%
Ohio State 5 0 0.0% 7 3 42.9%
Oklahoma 6 0 0.0% 8 2 25.0%
Pittsburgh 7 0 0.0% 3 0 0.0%
Syracuse 6 0 0.0% 10 4 40.0%
UCLA 3 1 33.3% 7 3 42.9%

** - Michigan State has made two Final Fours from a 5-seed, and has been seeded 5th half a dozen times over the years, so the Spartans' totals in this grid include the team's appearances as a 5-seed, while the other teams in the grid reflect 1- through 4-seeds only.

Only two of the teams perform significantly better than Duke against chalk. The fact that those teams are Kentucky and North Carolina may possibly cause some angst among Duke fans, but at that level it's a pretty small sample.

What's it all mean? Well, next time you're tempted to evaluate a team's entire season based on its NCAA tournament performance, remember that a great deal of that showing was dictated by circumstances entirely outside the team's and coach's control. Sure, basketball isn't played in a calculator, and the teams still have to win (or lose) the games. But the fact remains it's a lot easier to win when the odds are stacked in your favor, and a lot harder when they aren't.