Initial Ratings for USATT

[Pnachtwey]

The USATT had a chance to adopt the RC System many years ago but chose not to after their board of directors studied both systems.

So? Why didn't they change? There could be other reasons like not invented here rather than the merits

The RC System appears to work good when one class of players like professionals are grouped together.

Why? Where did you get this information and hopefully you simply don't quote some other person that just says so without proof.

When you have a wide range skill level of players like amateurs who play on an inconsistent basis, there are wild fluctuations in calculations, so it appears that it doesn't work very good.

Why? What is a wide range? Show me.

 

[Loopadoop]

So? Why didn't they change? There could be other reasons like not invented here rather than the merits


Why? Where did you get this information and hopefully you simply don't quote some other person that just says so without proof.


Why? What is a wide range? Show me.

Wide range - from beginner to master players or 0 - 2500

Go to a USA tournament processed by RC System

Look at changes for playing opponents for each players rating change and each players rating point exchanges By players within 200 ratings of each other, say 1600 and 1800, would be similar to pro when both players are in the same Class, seems ok

Then do the same for players with a wide difference of ratings like a 1100 and a 1600, wild rating calculation fluctuations.

 

[Pnachtwey]

Wide range - from beginner to master players or 0 - 2500

Go to a USA tournament processed by RC System

Look at changes for playing opponents for each players rating change and each players rating point exchanges By players within 200 ratings of each other, say 1600 and 1800, would be similar to pro when both players are in the same Class, seems ok

Then do the same for players with a wide difference of ratings like a 1100 and a 1600, wild rating calculation fluctuations.

You would expect the players ratings to change more if a 1100 beat a 1600 that if a 1600 beat a 1800. Likewise, a 1100 losing to a 1600 would have barely any effect if any.

You haven't made your case yet.

 

[Loopadoop]

You would expect the players ratings to change more if a 1100 beat a 1600 that if a 1600 beat a 1800. Likewise, a 1100 losing to a 1600 would have barely any effect if any.

You haven't made your case yet.

Did you see the inequalities of rating changes from one match, winning player gets match points but the losing player doesn't lose the same number of points, the net differences can be up to 300 points in some instances.

Everyone can go check this out for themselves at RC System.

For local tournaments in my area, even though they use the RC System for processing, they still USATT ratings for tournament seedings.

Edit Addition

One Example of RC for Texas Tournament

2015 Austin Spring Classic Round Robin 4/25/15 Detailed Report

Nicolae Tomescu lost to Brian Doherty

Tomescu 1300 Points lost -197
Doherty 1040 Points gain + 65
Net Point exchange diff -132

http://www.ratingscentral.com/EventDetail.php?EventID=17351#P81543

End Edit

 

[pgpg]

I think you do not fully understand the example you are quoting: someone with more established match record (1040+-97) beat player with estimated rating (1300+-200) - so you expect more uncertain number to move more. Note that he ended up with 975 rating afterwards.

P.S. Where I agree with you - individual matches gain/loss and data presented in RC are very hard to decipher.

 

[Pnachtwey]

The USATT system is a zero sum system. What one person wins is taken from another. The USATT has these fudge factors and adjustments that will accelerate the ratings change for newcomers if they win. The RC system scales the change in points by the standard deviation of the player as pgpg pointed out. The RC system is not exactly a zero sum system because of the difference in standard deviations or variance. I have seen the equation for how the ratings are updated some where. If I remember right it was there were some ratios of the one player's divided by the sum of both player's variance used as a scale factor. The RC system simply uses a more statistical way of updating ratings than having multiple iterations of fudging that can't be mathematically justified.

 

[Loopadoop]

RC - Same Tournament - Another Example

William Turpin beat Chris Weekes

William Turpin 1032 gains +4
Chris Weekes 431 loses -65
Net Points Exchange Diff -61

Explain that - lower rated player loses so many points

 

[Pnachtwey]

RC - Same Tournament - Another Example

William Turpin beat Chris Weekes

William Turpin 1032 gains +4
Chris Weekes 431 loses -65
Net Points Exchange Diff -61

Explain that - lower rated player loses so many points

You keep on ignoring the player's standard deviation. Weekes' standard deviation was very high. That is why he lost so many points. I said that the RC system is not a zero sum rating system. I thought I explained that above. As the standard deviations gets smaller and smaller the amount that is won or lost gets smaller.

 

[Loopadoop]

You keep on ignoring the player's standard deviation. Weekes' standard deviation was very high. That is why he lost so many points. I said that the RC system is not a zero sum rating system. I thought I explained that above. As the standard deviations gets smaller and smaller the amount that is won or lost gets smaller.

So, the RC System doesn't pass the common sense test.

It doesn't appear to work when there is a wide skill level range of 0 - 3000. The pure mathematical approach theoretically should work if you have one skill level class like professionals or even amateurs in the same class.

A 1032 rated player is 1-2 levels of skill above the 431 rated player. If the 431 rated loses the match to 1032 rated He shouldn't lose 65 rating points. His skill did not show a decline in playing level from that match.

 

[pgpg]

If I learned anything from RC system, it's that making sense out of individual match point changes is practically impossible, at least in the way they are presented. I would not be surprised if it's a feature and not a bug, since RC approach is heavily Bayesian in nature and isolating contribution of individual win/loss is probably quite challenging. May be it even partially explains why it is not more widely adopted, since your typical player does not really understand how maximum likelihood estimators work.

Instead - look at the overall rating change based on all wins/losses - do you have a problem with any of outcomes?

 

[Pnachtwey]

A 1032 rated player is 1-2 levels of skill above the 431 rated player. If the 431 rated loses the match to 1032 rated He shouldn't lose 65 rating points. His skill did not show a decline in playing level from that match.

You are assuming that the 431 rated player's rating is accurate in the first place. The RC system does not because of the large standard deviation. This could have been Chris Weekes' second tournament. I don't see a problem given the large standard deviation. What I would question is why the standard deviation is that wide to begin with. That standard deviation
See this
http://www.ratingscentral.com/HowItWorks.php
Look at the temporal update graph. What is saying is that it looks like that player has a rating of 1350 one year ago and 1350 now but the probability of having the exact rating of 1350 a year ago was about 0.009 and the probability of have the exact rating of 1350 after one year of not playing is about 0.0048. What a player's standard deviation is 243 the curve is very wide and not very tall. I could do the math and find that the probability of the Weeks rating is actually 431 is very small and since the curve is almost flat the probability of Weekes' rating is 431 is just and insignificant fraction higher than any other rating he could have. Just about any rating would work and that is why Weekes rating can move around a lot.

Yes, the RC rating system passes the common sense test IF you understand probabilities and math. The errors are due to users/tournament directors.

 

[Pnachtwey]

Someone on another TT forum brought up this same example as loopadoop. Apparently they can't get the concept that someone with a rating of 431 can actual have a probability of that rating being accurate is less than 1% and the the difference difference between Weeke's rating of 431 and his rating of 366 is much less than 1%. I can calculate the exact value tomorrow.

The probability that Weekes actually has a rating of 431 is very small. The bell curve may be centered on 431 but probability of actually being exactly 431 is less than 1% because the bell curve is very wide and flat.

You would think that the other forum that claims to have many PhDs and such would have someone that could explain things to the forum but no.

 

[pgpg]

That someone, LUCKYLOOP, is Loopadoop here (he brings up USA Hobby TT coalition in every place). I forgot his UID on OOAK, but he's there too.

P.S. Arguing with anonymous people on TT forum about Bayesian statistics is not my concept of fun. And yes, I do have PhD.

 

[Loopadoop]

RC Bayesian Theory ?

How do know it was applied accurately ? Did anyone check the work ? Can you make different assumptions of the variables involved ? There is no public user check and balance, supporters defend by saying it is based on probability.

 

[pgpg]

RC Bayesian Theory ?

How do know it was applied accurately ? Did anyone check the work ? Can you make different assumptions of the variables involved ? There is no public user check and balance, supporters defend by saying it is based on probability.

Detailed info can be found here, with gory model details in the references at the bottom of the page (more academic paper is the first link (published in pretty solid refereed journal, if it matters)). It spells out what it is doing and why (and also explains a bit about how results are presented). Feel free to critique methodology. I don't have visibility to RC code, so can not comment on actual implementation.

http://www.ratingscentral.com/HowItWorks.php

By the way, USATT system is based on probability as well, so I'm not sure what's your point. The reason I say I like RatingsCentral system better is because it attempts to incorporate uncertainty into ratings process. Not perfect, but I view it as an improvement. You might disagree - that's fine.

 

[Pnachtwey]

RC Bayesian Theory ?

How do know it was applied accurately ? Did anyone check the work?

I didn't check the implementation but the formula on page 199 of this
http://www.ratingscentral.com/Doc/NewTTRS.pdf
makes sense to me. It is basically a double integral that takes into account the probability of each possible rating could have and then the probability that matches the real outcome of win or loss.

Can you make different assumptions of the variables involved ?

The big assumption is the temporal update. That is rate of change in uncertainty or the variance. The variance increases by 70^2 per year. This is just a scaling issue. It could have been 60^2 per year but the ratings would change less.

There is no public user check and balance, supporters defend by saying it is based on probability.

I understand the equation on page 199.

There isn't really anything tricky but it does require more number crunching to do the double integral. It wouldn't have been possible without modern PCs.

I did the math for Chris Weekes' probability distribution. Given the numbers the probability that Chris Weekes' performance is actually 431 is 0.0016. The probability that Chris Weekes' performance was actually 431-65 at the beginning is 0.0015. If I plotted the normal distribution for Chris Weekes it would look like a flat line at the bottom of the graph if I scaled the graph to plot the probability with a scale of 0-1 for the y axis. The standard deviation is very big.

Loopadoop, given Weekes standard deviation of 243 the results don't surprise me. However, you seem to have ignored the point that I made above. Why is Weekes' standard deviation so big? If the standard deviation is 243 then the variance is 243^2=59049. If the variance is supposed to increase by 70^2 a year that would mean his 431 rating is 12 years old. That 431 rating is very old and not that significant to the RC algorithm.

There is nothing wrong with the RC system but I know of disasters in how it was used by untrained directors.
Since pgpg and loopadoop are on another TT forum then must know of Eric Fountain. He plays in the Blitz Tavern Thursday league.
http://www.ratingscentral.com/EventSummary.php?EventID=19651
People that have watched Eric play know he is much better than 1355. He is a solid 1650 USATT player. The RC ratings in the Portland area are depressed because the tournament directors started most players with ratings that are far too low.
Within a club the RC ratings are significant because only the difference in ratings mean anything but ratings from different clubs or areas cannot be compared unless there is the ability to play outside clubs so that any imbalance will even itself out but what club would what to play a Portland club if they know their ratings will be brought down?

It isn't the numbers or formulas that are the problem. It is the people.

 

[pgpg]

RC and USATT do not have the same scale and can not be compared directly. I heard several times a rule of thumb is that RC+300 ~ USATT, which seems to be true in case of Eric and few others I know with well-established ratings in both systems.

But yeah, setting initial prior too low/high will mess RC up (absolute scale, that is) for a close system.

 

[Pnachtwey]

RC and USATT do not have the same scale and can not be compared directly.

The "scales" are very close. If I fit the same function that RC uses to the USATT ratings table the difference is small.

I heard several times a rule of thumb is that RC+300 ~ USATT, which seems to be true in case of Eric and few others I know with well-established ratings in both systems.

If the scale is different then why didn't the people that told you RC+300~USATT. My first coach had a RC rating that was almost 400 points lower than his USATT rating. The RC system doesn't award a boost of accelerator rating points like the USATT system does. These accelerator points increase the average.

But yeah, setting initial prior too low/high will mess RC up (absolute scale, that is) for a close system.

That is exactly what happened in Portland.