Anderson & Hester vs. Sagarin

[donovan]

Seems to me that a rankings find true verification when games are played. I have a ranking and put Team XYZ above team ABC, when these teams meet, team XYZ should win. If that happens, my listing was verified by the actual contest. (Now I know there are those that will make conversation infinitum why ABC is a better team, even though they lost...but, that is another issue...one best addressed in a office where you lie on a couch in a transitory state)

Because rankings are verified as to their accuracy by the contests then the ranking system that has the highest percentage of correct games predicted, in my opinion, becomes the most accurate ranking. You find that ranking system, right here on this website.

[billybud]

How "good" a ranking algorithmn is can be reflected by its accuracy...

If a system is a more accurate predictor than another system, and consistantly so, then it is a better system for ranking the relative strengths. As I have posted before, there are awards handed out yearly (for past seven seasons) for football system ranking accuracy..........and on many parameters of measurement, both predictive and retrodictive.

From 2007 awards...cut/paste

MOST STRAIGHT UP WINNERS (Entire Season)

2007 Winner: StatFox

(And you might want to call it a disappointing season for computer ratings as StatFox
was the only system to do better than the Oddsmakers.)


2006 Winner: Thompson SPRS, system average, Sagarin predictive, & Sagarin overall
2005 Winner: Pigskin Index
2004 Winner: Jeff Sagarin's Overall Rating
2003 Winner: Edward Kambour & Jeff Sagarin's Overall Rating
2002 Winner: ARGH Power Ratings
2001 Winner: Massey Ratings
2000 Winner: ARGH Power Ratings
1999 Winner: Vegas line

MOST ACCURATE PREDICTOR (Entire Season)

2007 Winner: The Vegas line

2006 Winner: The Vegas Line
2005 Winner: The Vegas Line
2004 Winner: The Vegas Line
2003 Winner: The Vegas Line
2002 Winner: The Vegas Line
2001 Winner: The Vegas Line
2000 Winner: The Vegas Line
1999 Winner: The Vegas Line

SMALLEST DEVIATION FROM ACTUAL GAME SCORES (Second Half of Season)

2007 winner Edward Kambour

2006 Winner: Computer Adjusted Line
2005 Winner: Vegas Line
2004 Winner: Ken Ashby AccuRatings
2003 Winner: Vegas Line (opening)
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line

MOST ACCURATE PREDICTOR (Second Half of Season)

2007 Winner: Atomic Football

2006 Winner: Vegas Line
2005 Winner: Vegas Line
2004 Winner: Ashby Accuratings
2003 Winner: Vegas Line
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line

MOST RETRODICTIVE WINS - Complex System

2007 Winner: The Sports Report - SLOTS

2006 Winner: The Sports Report - SLOTS
2005 Winner: The Sports Report - SLOTS
2004 Winner: The Sports Report - SLOTS
2003 Winner: The Sports Report - SLOTS
2002 Winner: The Sports Report - SLOTS
2001 Winner: CPA Rankings
2000 Winner: CPA Rankings

MOST RETRODICTIVE WINS - Simple System

2007 Winner: Jeff Sagarin

2006 Winner: The SuperList
2005 Winner: Jeff Sagarin, Warren Claassen
2004 Winner: Logistic Regression
2003 Winner: Anderson/Hester
2002 Winner: Logistic Regression
2001 WInner: System Average

SMALLEST RETRODICTIVE MEAN ERROR - Complex System

2007 Winner: Least Square Regression with team HFA

2006 Winner: Least Square Regression with team HFA
2005 Winner: Least Square Regression with team HFA
2004 Winner: Least Square Regression with team HFA
2003 Winner: The Sports Report - SLOTS
2002 Winner: The average of all systems.
2001 Winner: Edward Kambour Football Ratings
2000 Winner: CPA Rankings

SMALLEST RETRODICTIVE MEAN ERROR - Simple System

2007 Winner: Jeff Sagarin Predictive

2006 Winner: Jeff Sagarin Predictive
2005 Winner: Jeff Sagarin Predictive
2004 Winner: Least Squares Regression
2003 Winner: Least Squares Regression

etc..etc.

[Spence]

You can look at who won in any particular year, but if you look at overall accuracy they are all within 3-5% of each other. That would probably be very close to the margin of error so they are all basically the same over time. Why hasn't anyone broken from the pack should be the question you are asking. In answering that question you can understand what needs to be done to improve the overall system. It is an easy fix, but it requires a major monetary shift and that is probably the reason it will never happen.

[billybud]

Seems to me that I see the same names over the years...I would say that those that keep showing up as "winners'", with 56 + systems being rated, are more accurate than systems that never show up...

I surely would pay attention to Sagarin...his system is highly ranked for accuracy...I would pay attention to the Vegas Line..it is a real tell tale.

And, as a caveat...the best predictor system in college ball last year still whiffed on almost a quarter of the games.

[Spence]

And, as a caveat...the best predictor system in college ball last year still whiffed on almost a quarter of the games.

And that is my point. Not enough information. When the best that is out is 75%, that isn't great. If you go into the hospital for an operation do you want a doctor that is correct, at best, 75% of the time or would you rather have the one who recovers his patient in the 90's? I'm not saying you shouldn't pay attention to the computer rankings, most of them are more accurate on average then their human counterparts. They are still flawed, though, because they operate at such a low percentage of accuracy. There are not many businesses that you can operate under 75% production and compete. Most want production in the high 90's. The computers will never get into the high 90's because of the amount of unknowns, but if the schedules were set up in a fashion that everyone had to play, at least, four games with their peers from other conferences the computer rankings could bring their numbers into the high 80's or low 90's. If they could operate at those levels we would have a system that could give you a legit national champ. It wouldn't stop the controversy, but it would be highly accurate in any case.

It will never happen because college football wants the controversy. It drives the machine.

[CFP Admin]

You missed one:

SMALLEST AVERAGE GAME BIAS (Entire Season)

2007 Winner: Thompson SPRS
2006 Winner: Congrove Computer Rankings
2005 Winner: Computer Adjusted Line
2004 Winner: CPA Rankings
2003 Winner: Edward Kambour
2002 Winner: Gupta Power Ratings
2001 Winner: Flyman
2000 Winner: ARGH Power Ratings
1999 Winner: Jeff Sagarin

Also: What's more important is year-over-year consistency.
Here is Congrove vs. the BCS rankers, based on Pct. correct
2007 (out of 60 ranked, including Vegas Line and Line Updated)
16- Dave Congrove 0.72753
42 - Anderson/Hester 0.70627
46 - Sagarin Elo 0.70084
49 - Massey BCS 0.69444
50 - Billingsly 0.69242
51 - Colley Rankings 0.69014
53 - Wolfe 0.68647

2006 (out of 56)
34 - Dave Congrove 0.72969
37 - Billingsly 0.72829
40 - Sagarin Elo 0.72269
46 - Colley Rankings 0.71154
47 - Anderson/Hester 0.70879
48 - Wolfe 0.70879
53 - Massey BCS 0.69231

2005 (out of 55)
12 - Dave Congrove 0.71601
44 - Sagarin Elo 0.68127
45 - Billingsly 0.67674
46 - Colley Rankings 0.67593
48 - Wolfe 0.66667
49 - Massey BCS 0.66667
51 - Anderson/Hester 0.66047

2004 (out of 48)
3 - Colley Rankings 0.74522
4 - Massey BCS 0.74427
7 - Anderson/Hester 0.74204
15 - Sagarin Elo 0.73323
21 - Dave Congrove 0.72923
27 - Billingsly 0.72397
31 - Wolfe 0.72293

2003 (out of 50)
8 - Dave Congrove 0.74928
34 - Sagarin Elo 0.72636
39 - Billingsly 0.71920
41 - Wolfe 0.71563
44 - Massey BCS 0.70796
47 - Anderson/Hester 0.69976
48 - Colley Rankings 0.69504

Congrove was not included prior to 2003.

[CFP Admin]

Here's Congrove against the spread vs. the BCS

2007
10 - Dave Congrove 0.51571
45 - Sagarin Elo 0.48429
49 - Billingsly 0.48286
55 - Anderson/Hester 0.46980
56 - Colley Rankings 0.46705
57 - Wolfe 0.46644
59 - Massey BCS 0.44654


2006
11 - Billingsly 0.53094
17 - Dave Congrove 0.52086
41 - Massey BCS 0.50000
45 - Colley Rankings 0.49721
48 - Wolfe 0.49441
51 - Sagarin Elo 0.48994
52 - Anderson/Hester 0.48883


2005
6 - Massey BCS 0.50955
9 - Anderson/Hester 0.50240
11 - Dave Congrove 0.50164
13 - Billingsly 0.49845
16 - Wolfe 0.49682
30 - Colley Rankings 0.48726
32 - Sagarin Elo 0.48372


2004
2 - Wolfe 0.54662
5 - Colley Rankings 0.53376
10 - Sagarin Elo 0.52315
11 - Massey BCS 0.52308
21 - Anderson/Hester 0.51125
29 - Billingsly 0.49666
40 - Dave Congrove 0.48525 ouch!


2003
4 - Dave Congrove 0.51618
25 - Massey BCS 0.48649
36 - Sagarin Elo 0.47877
40 - Billingsly 0.46559
43 - Colley Rankings 0.46377
46 - Wolfe 0.45860
47 - Anderson/Hester 0.45411


In my view, there is a bit of a flaw in this measuring system that could positvely/negatively affect everyone. The flaw is that the predictions are measured against the Final Line when, in my opinion, they should be measured within 24-48 hours of the opening line. The line changes in reaction to the betting, and the betting is influenced by all of the information put out there. So, in essence, the line has the opportunity to cheat. This is why we typically measure the computer's ATS percentage against a Tuesday or Wednesday line, and use the line most-often quoted when comparing mutlple Vegas sportsbooks. In other words, we measure the results as though we actually placed bets on one day of the week.

Disclaimer: We do not bet on college football, and do not encourage anyone to use this information for such purpose.

[CFP Admin]

I surely would pay attention to Sagarin...his system is highly ranked for accuracy

oh really...... The figures above say otherwise. Sagarin is good for retrodictive (looking backwards), but isn't worth a hoot for predictive.

...I would pay attention to the Vegas Line..it is a real tell tale.

It has the luxury of being a moving target, reacting to its competitors.

And, as a caveat...the best predictor system in college ball last year still whiffed on almost a quarter of the games.

And humans don't come anywhere near that close.

[billybud]

I understand the site administration's personal interest in this subject...and I mean you no disrespect Mr. C...

oh really...... The figures above say otherwise. Sagarin is good for retrodictive (looking backwards), but isn't worth a hoot for predictive.

I see Sagarin listed as winner in MOST STRAIGHT UP WINNERS (Entire Season) in 2006 and 2004...that is predictive.

So Congrove is good vs the spread ( a gambler's friend) but Sagarin seems to be better at predicting outright winners.

Different measures...if I was a gambler betting against the spread, I'd want to have the guy who called the spread right more often...if I wanted to know who was going to win the game..I'd want the guy who called the winner more often..

I think that diferent sysytems have individual strengths....I am no expert, but I do try to follow the systems...

[CFP Admin]

I understand the site administration's personal interest in this subject...and I mean you no disrespect Mr. C...

oh really...... The figures above say otherwise. Sagarin is good for retrodictive (looking backwards), but isn't worth a hoot for predictive.

I see Sagarin listed as winner in MOST STRAIGHT UP WINNERS (Entire Season) in 2006 and 2004...that is predictive.

So Congrove is good vs the spread ( a gambler's friend) but Sagarin seems to be better at predicting outright winners.

Why would I/we feel disrespected? What I compared are the BCS Rankers that each system uses. As indicated, CCR has historically proven to be more accurate.

[CFP Admin]

I understand the site administration's personal interest in this subject...and I mean you no disrespect Mr. C...

I see Sagarin listed as winner in MOST STRAIGHT UP WINNERS (Entire Season) in 2006 and 2004...that is predictive

.

I guess I have to stand corrected on my assertion that "Sagarin is good for retrodictive (looking backwards), but isn't worth a hoot for predictive". His "predictive" method compares well. But to clarify, I was pointing out that his BCS Rankings (Sagarin ELO) are not worth a hoot for predictive. Unfortunately, his BCS rankings are typically the least accurate of his three versions.

Let's compare CCR directly with Sagarin in the category you outlined, and I will show ALL of his various rankings.

MOST STRAIGHT UP WINNERS - 2007
15 - Dave Congrove
26 - Sagarin Predictive
32 - Sagarin
42 - Sagarin Elo


MOST STRAIGHT UP WINNERS - 2006
3 - Sagarin Predictive (trails only the "System Average" and "System Median"
4 - Sagarin
29 - Dave Congrove
33 - Sagarin Elo


MOST STRAIGHT UP WINNERS - 2005
9 - Dave Congrove
30 - Sagarin Predictive
32 - Sagarin
39 - Sagarin Elo


MOST STRAIGHT UP WINNERS - 2004
1 - Sagarin
11 - Sagarin Elo
13 - Sagarin Predictive
21 - Dave Congrove


MOST STRAIGHT UP WINNERS - 2003
2 - Sagarin
5 - Sagarin Predictive
6 - Dave Congrove
24 - Sagarin Elo


Comparing his "Predictive" rankings heads-up with CCR, the CCR wins 2 out of 5 seasons.
Sagarin Predictive average finish over the last 5 years is 15.4
Congrove average finish over the last 5 years is 16.0

Sagarin Predictive Total SU Winners (2003-007): 2,519
CCR Total SU Winners (2003-007): 2,510

Comparing CCR to all Sagarin rankings, CCR beats Sagarin cumulative 8 times, Sagarin beats CCR a cumulative 7 times.

[billybud]

Yeah...Sagarin states that his "BCS" ratings are not as predictive as his "predictor"...I think that the BCS missed the boat by eliminating a predictor like scoring differential.