'04 FBR Rankings 03-27-12

First, great work. I feel you pain....primarily in the data. Check out the 05 forum with the thread of: statistical rankings.

In the spirit of getting better results, let's get cerebral. First I acknowledge that no system is perfect and everyone strives to tweak to predict the future.

Why use tiers? My feeling is that tiers give a greater chance to ignore the true differential of the teams.
Your site says "We establish ranking Tiers in the FBR that give greater weighting to games played against stronger opponents." Why can't the weighting be the true difference of the teams.... which is the difference in the score of the game? Why water it down? Why try to correct actuals?

The FBR solution you use was created for American College Football. Suspect the developer implemented the weighting because they could not get enough statistical data in a college football season. College football teams average about 10 games a year (8 conference, 2 out of conference). Our academy soccer teams are averaging about 30 games a year (league and tournament). 3x more data and multiple tournaments that really help in normalizing the strength of schedule. Just don't think you need the tiers with all the games and cross play.

If the #1 team plays team #10 team. How much should team #1 wins by....statistically speaking? Hard to say with a more subjective weighting system. The 80/20 is more of a guess too....and it is applied across all teams/tiers (socialism?). You assign a value (eg 52, 26, 13...) for a win? You know the difference from the score though. The 40/10 on the weighted wins. How do you get 40/10? Applies to each age level the same also?

On the Most Weighted Wins, seems like the team that play less games are penalized. That concept is a bit like the one that gotsoccer uses ....and that is garbage (imo).....check out the rankings on their site. Not terribly reflective of the actual landscape if you are familiar with big girl soccer (aka select).

I don't want to do other age groups because data is a pain. PM me the data, I'll run it through my model, and send you the results without posting. See how close (or not) they are. We could track the last few weeks of the year to see which is more accurate. I've got no dog in the fight....just looking for the best mousetrap.

soccer.stats.05 wrote:

Why use tiers? My feeling is that tiers give a greater chance to ignore the true differential of the teams.
Your site says "We establish ranking Tiers in the FBR that give greater weighting to games played against stronger opponents." Why can't the weighting be the true difference of the teams.... which is the difference in the score of the game? Why water it down? Why try to correct actuals?

The FBR solution you use was created for American College Football. Suspect the developer implemented the weighting because they could not get enough statistical data in a college football season. College football teams average about 10 games a year (8 conference, 2 out of conference). Our academy soccer teams are averaging about 30 games a year (league and tournament). 3x more data and multiple tournaments that really help in normalizing the strength of schedule. Just don't think you need the tiers with all the games and cross play.

If the #1 team plays team #10 team. How much should team #1 wins by....statistically speaking? Hard to say with a more subjective weighting system. The 80/20 is more of a guess too....and it is applied across all teams/tiers (socialism?). You assign a value (eg 52, 26, 13...) for a win? You know the difference from the score though. The 40/10 on the weighted wins. How do you get 40/10? Applies to each age level the same also?

On the Most Weighted Wins, seems like the team that play less games are penalized. That concept is a bit like the one that gotsoccer uses ....and that is garbage (imo).....check out the rankings on their site. Not terribly reflective of the actual landscape if you are familiar with big girl soccer (aka select).

I don't want to do other age groups because data is a pain. PM me the data, I'll run it through my model, and send you the results without posting. See how close (or not) they are. We could track the last few weeks of the year to see which is more accurate. I've got no dog in the fight....just looking for the best mousetrap.

I'm an engineer. Know enough about statistics to get into plenty of trouble, but am admittedly not a statistician by nature or training.

The weighting tiers and the most weighted wins are just one approach to handle teams that play unbalanced schedules, both in term of # of games played, and in terms of relative strength of opponents.

Sorry, but I have no desire to get into a debate on the statistical validity of the FBR methodology. It was not developed based on any kind rigorous statistical analysis, rather the FBR methodology was "tuned" over time with empirical data. What I can say, is that I use the exact same methodology, weighting factors, and calculation formulae for the '01's, '02's, and '04's (and will for the '03's if I ever get around to them).

At the end of the day, I look at the results and say "can I defend these rankings based on side by side comparison of the actual game results". I wouldn't publish them if I couldn't.

With that said, if you think you can develop a better mouse trap, be my guest. I welcome it.

Tell you what... I'll work on sending you my data for the '02's. Put your system together and let's run them side-by-side for the '02's leading all of the way up to QT this summer and let's see which system does a better job at predicting the teams that end up qualifying for Lake Highlands. Maybe even put a friendly wager on it?

FBR was ~90% accurate at predicting the '01's for Lake Highlands last year.

That was a good question and a good answer, imo. Throw your hand in there stats05. Be interesting to see. Also I work for an actuarial company. I could get the nerds working on a third posting of the scores of games yet to be played. It would save us a lot of gas money

soccer.stats.05 wrote:First, great work. I feel you pain....primarily in the data. Check out the 05 forum with the thread of: statistical rankings.

In the spirit of getting better results, let's get cerebral. First I acknowledge that no system is perfect and everyone strives to tweak to predict the future.

Why use tiers? My feeling is that tiers give a greater chance to ignore the true differential of the teams.
Your site says "We establish ranking Tiers in the FBR that give greater weighting to games played against stronger opponents." Why can't the weighting be the true difference of the teams.... which is the difference in the score of the game? Why water it down? Why try to correct actuals?

The FBR solution you use was created for American College Football. Suspect the developer implemented the weighting because they could not get enough statistical data in a college football season. College football teams average about 10 games a year (8 conference, 2 out of conference). Our academy soccer teams are averaging about 30 games a year (league and tournament). 3x more data and multiple tournaments that really help in normalizing the strength of schedule. Just don't think you need the tiers with all the games and cross play.

If the #1 team plays team #10 team. How much should team #1 wins by....statistically speaking? Hard to say with a more subjective weighting system. The 80/20 is more of a guess too....and it is applied across all teams/tiers (socialism?). You assign a value (eg 52, 26, 13...) for a win? You know the difference from the score though. The 40/10 on the weighted wins. How do you get 40/10? Applies to each age level the same also?

On the Most Weighted Wins, seems like the team that play less games are penalized. That concept is a bit like the one that gotsoccer uses ....and that is garbage (imo).....check out the rankings on their site. Not terribly reflective of the actual landscape if you are familiar with big girl soccer (aka select).

I don't want to do other age groups because data is a pain. PM me the data, I'll run it through my model, and send you the results without posting. See how close (or not) they are. We could track the last few weeks of the year to see which is more accurate. I've got no dog in the fight....just looking for the best mousetrap.

Where were you when fsd did his rankings ...that were way off sounds like you do have a dog in this fight, and now that the rankings are 100% more accurate I dont think you like where your team is ranked so just shut it and go post on the 05's section

It's never going to be a perfect system. Heck, college football has tweaked the BCS mutiple times and it's still flawed. In the 03s, we will have the FBR, Power Rankings, and inevitably Human Rankings. I've found FBR to be the most reliable of those three, yet all have value. Sure, there may be a better 'mousetrap' and perhaps that is a combination of the 3 ranking systems or perhaps it's the system soccer.stats.05 speaks of. But, for now, I support bwgophers and all of the effort he has put into the FBR. Thank you and please keep up the good work.

Heck I am just thankful I don't have to do them anymore! Thank you bwgophers

I don't want to give people the wrong impression.

I really do welcome anyone who wants to put together their own ranking system and while I love having a little friendly competition over something like who's ranking system predicts the QT outcome better, I don't think that anytime in the past 12-18 months, have I ever come on these boards and claimed that FBR was better than another ranking system, just different.

Outside of mslater, who worked closely with me mold FBR into it's current format (which is unchanged from Jan 2011), there really isn't anyone who can vouch for the amount of time, effort, and care that I put into developing FBR into it's current form, checking, re-checking, and re-checking again, the output to make sure it made sense and was defensible. Believe me when I say that Slata-Hata NEVER let me off the hook, and frankly was a bigger bulldog than anyone else on these forums in making sure that FBR put out accurate, credible, and defensible results. We had a couple of absolute knock-down, drag-outs last year, and frankly, FBR is better today because of it.

So when I say that I have no interest in revisiting the fundamentals behind FBR, it's because:

1) I truly feel that I have done due diligence on testing the system out
2) While there can always be improvements made, FBR is at a point of diminishing returns. Any improvements to FBR would be incremental at this point, while the effort required realize that incremental improvement would be substantial. Just not worth it IMO.

I welcome anyone who wants to bring a different approach and or system to the table, and I welcome anyone who wants to go out and purchase FBR and put in the effort to see if they can improve on what I am publishing.

I'm just at a point where I'm confident in what I have, and frankly don't see any real benefit to putting in the effort required to make it marginally better.

Sure, let's have a bake off. Should be fun.

All my data is posted publicly, but I will PM you the 05 data also. The raw data is on the 'Confirmed Games' tab here:
https://docs.google.com/spreadsheet/ccc?key=0AvCcWaU-Tr54dEVhMWVGZjBRQlJmdDNxczVreVhBQlE#gid=2

Send me the raw data for 01s, 02s, 04s. I'll provide the ratings and the goal differential between the teams. Then for next week let’s see how the models do by predicting the winner and goal differential of each game. We should have about 60-80 games in a weekend to bounce off the models using the four different age groups.

My trading currency is a beer. How about the loser buys a few rounds of beer at an agreed upon watering hole?

soccer.stats.05 wrote:Sure, let's have a bake off. Should be fun.

All my data is posted publicly, but I will PM you the 05 data also. The raw data is on the 'Confirmed Games' tab here:
https://docs.google.com/spreadsheet/ccc?key=0AvCcWaU-Tr54dEVhMWVGZjBRQlJmdDNxczVreVhBQlE#gid=2

Send me the raw data for 01s, 02s, 04s. I'll provide the ratings and the goal differential between the teams. Then for next week let’s see how the models do by predicting the winner and goal differential of each game. We should have about 60-80 games in a weekend to bounce off the models using the four different age groups.

My trading currency is a beer. How about the loser buys a few rounds of beer at an agreed upon watering hole?

First off... your currency is definitely accepted in my neck of the woods!

I sent the '02 data yesterday to the e-mail address listed on your '05 statistical rankings post. I will try to get the '01 and '04 data out to you, but honestly, this week is NOT good for me from a free time standpoint, so I will make no promises.

I did pit FBR against the "human" prognosticators in the '01 score prognostication game last spring. The final results are in this thread: http://www.txsoccer.net/t6673-01-prognostication-results-finally

My alter-ego "Watson" is the FBR and it fared pretty well. If you read down the thread, there is a description of the simple formula that I used to predict the scores using only GS and GA data, incorporating adjustments for average goal differential in games played between teams from different ranking tiers.

Unfortunately, the FBR software doesn't lend itself to easily making those calculations, so coming up with predicted scores involves a lot of manual effort on my side. I definitely won't be able to do something like that this week, but possibly the week of 4/14 (also, I'm pretty sure there aren't any '01 Lake Highlands or Plano Premier games this week)

bwgophers wrote:

soccer.stats.05 wrote:Sure, let's have a bake off. Should be fun.

All my data is posted publicly, but I will PM you the 05 data also. The raw data is on the 'Confirmed Games' tab here:
https://docs.google.com/spreadsheet/ccc?key=0AvCcWaU-Tr54dEVhMWVGZjBRQlJmdDNxczVreVhBQlE#gid=2

Send me the raw data for 01s, 02s, 04s. I'll provide the ratings and the goal differential between the teams. Then for next week let’s see how the models do by predicting the winner and goal differential of each game. We should have about 60-80 games in a weekend to bounce off the models using the four different age groups.

My trading currency is a beer. How about the loser buys a few rounds of beer at an agreed upon watering hole?

First off... your currency is definitely accepted in my neck of the woods!

I sent the '02 data yesterday to the e-mail address listed on your '05 statistical rankings post. I will try to get the '01 and '04 data out to you, but honestly, this week is NOT good for me from a free time standpoint, so I will make no promises.

I did pit FBR against the "human" prognosticators in the '01 score prognostication game last spring. The final results are in this thread: http://www.txsoccer.net/t6673-01-prognostication-results-finally

My alter-ego "Watson" is the FBR and it fared pretty well. If you read down the thread, there is a description of the simple formula that I used to predict the scores using only GS and GA data, incorporating adjustments for average goal differential in games played between teams from different ranking tiers.

Unfortunately, the FBR software doesn't lend itself to easily making those calculations, so coming up with predicted scores involves a lot of manual effort on my side. I definitely won't be able to do something like that this week, but possibly the week of 4/14 (also, I'm pretty sure there aren't any '01 Lake Highlands or Plano Premier games this week)

I will buy a round or two as well. Count me in ( beer drinking that is)!

Thanks for the 02 data. I ran the numbers (below). I'd like to include the Fall games/tournaments as well so if you can forward that data, I will include in the calculations.

02 Statistical Rankings
https://docs.google.com/spreadsheet/pub?key=0AkQprsT3s1EQdGs0dFFVNFdTb2p2YmU0cEZGWFRnR2c&output=html

The difference between the teams ratings is the expected goal differential if they played each other.

Did someone say beer? However this comes out... I will buy a round for the contestants.

I've got round two ....including a premium beer for whichever system ranks us higher....I mean more accurately.

Rookie wrote:I've got round two ....including a premium beer for whichever system ranks us higher....I mean more accurately.

That's all I drink!