Gen 8 Smogon University Usage Statistics Discussion Thread

Status
Not open for further replies.

Marty

Always more to find
is a Site Content Manageris a Battle Simulator Administratoris a Programmeris a Member of Senior Staffis a Community Contributoris a Top Researcheris a Top Tiering Contributor
Research Leader
Credit to Antar for the original version of this FAQ.

Frequently Asked Questions
  1. Where are the stats?
    Stats can be found at https://smogon.com/stats/ From there, navigate to the desired month.
  2. Why aren't the stats up for the month yet?
    Processing currently takes around 44 hours; ~25 hours to process the raw logs and ~19 hours to process the files found in the link above. I pre-process the first 25-29 days of any given month before the month is over to get stats out as soon as possible, and so far this has resulted in usage stats out on the 1st of every month. If for some reason there's a problem with the first step, it will take 44 hours to re-process everything so stats would only come out on the 3rd or later. Please be patient if this happens!
  3. Where are the moveset stats / metagame stats / lead stats / monotype stats / chaos stats since last month?
    Respectively, in the "moveset," "metagame," "lead," "monotype" and "chaos" subfolders of each month.
  4. What do "Usage %", "Raw", and "Real" mean?
    Code:
     + ---- + ------------------ + --------- + ------ + ------- + ------ + ------- +
    | Rank | Pokemon | Usage % | Raw | % | Real | % |
    + ---- + ------------------ + --------- + ------ + ------- + ------ + ------- +
    Antar said:
    Usage % : Weighted
    Raw: Unweighted
    Real: Only counts the Pokemon which actually appear in battle (Doubles not supported)

    The reason for the name "real" is historic--back when I first took over the stats and then the running of PO, only the Pokemon that appeared in battle were recorded in the logs, so there was no way to actually *get* the full team stats. When I modified PO to generate logs with full team info in them, we were left with a decision regarding which stats to use, and the argument was that counting only Pokemon appearing in battle was somewhat more legit, because that corresponded to actual, or "real" usage (that argument lost out in the end).
  5. How are usage stats weighted?
    Every player on Pokemon Showdown has a skill rating for each metagame they participate in. This rating--which is different from your ladder score--is calculated using an algorithm called Glicko and consists of an estimated skill value R and an uncertainty in that estimate RD. Based on these two values, we calculate the likelihood that a given player has a "true" skill value above a certain baseline (the conventional baseline was 1500, corresponding to the "average" player). For more about ratings, read Everything You Ever Wanted to Know About Ratings. For more about weightings, read Weighted Stats FAQ.
  6. How are tiers determined from usage?
    For the foreseeable future, all usage will be based on the previous month using the 3-month cutoff of 4.52% to accelerate accurate tier formation.

    During the regular 3-month cycle, month 1 has quick drop cutoff of 1.53%, month 2 a quick drop cutoff of 2.28%, and month 3 the new standard of 4.52% for drops and rises. (Unrounded these cutoffs are 0.01528524706, 0.02284003156, and 0.04515839608, respectively.) Month 2 and month 3 will be averaged evenly with the other month(s) in the cycle.

    As for which stats are used to determine the tiers, we're currently using a baseline of 1695 for OU and Doubles OU, and 1630 for all other tiers.
  7. What's the deal with the filenames?
    You'll notice that for each tier and type of analysis, there are a bunch of of different files, most with names like gen8uu-1630.txt. The first part of the filename is the tier, the second part is the weighting baseline. Also note that a baseline of 0 means that the stats are basically unweighted.
  8. How should I think about Baseline-0 vs. 1500 vs. 1630/1695 vs. 1760/1825 stats?
    • Baseline-0 (unweighted) stats represent everything in the format, no matter how lulzy the player or team. This is what you'd expect to encounter if we stopped doing matchmaking.
    • 1500 stats represents what the average player in the metagame sees. Since Showdown's playerbase is more than just Smogonites, this is considerably "below" what the average person reading this thread sees.
    • 1630 (1695 for OU) stats represent "standard" stats, what the typical competitive player should see and be prepared for.
    • 1760 (1825 for OU) stats represent "1337" stats, what the best-of-the-best in the metagame are doing. To some extent, this is what all players should strive to be doing, but there are some Pokemon and strategies that are difficult to pull off and might require a greater amount of skill than the typical competitive player possesses.
  9. Why are the OU stats for 1695 and 1825 instead of for 1630 and 1760?
    OU, aka "Standard," is, well, our standard tier. It sees more battles than any other format and has the largest playerbase (second only to randbats). It also has the smallest fraction of "competitive players" of all non-random formats, due to its prominence and easy accesibility. Since our rating systems are percentile-based (that is, a rating of x roughly corresponds to being better than y% of the ladder, rather than indicating that the player is the nth best in the metagame), that means that it's a lot easier to get a rating of 1630 in OU than it is in UU or LC. Because of that, and because OU has a larger pool of battles to work with, we can up our baseline to 1695 for the "standard" stats. Similarly, while 1760 is the usual value we use for "elite" stats (the best of the best), the number that works better for OU is 1825.
  10. What's the best way to make use of the moveset stats?
    • If you're trying to figure out what's good in a tier (in terms of movesets), 1760/1825 is probably the way to go, since that tells you what the very top players use on their Pokemon.
    • If you want to determine what the likelihood is that your opponent's Pokemon carries X move or Y item, consult the moveset stats closest to your own Glicko R rating.
    • If you're having trouble dealing with a certain Pokemon and are looking for checks/counters, consult the 1500 (or even possibly the 0) stats: the lack of "1337"ness is vastly preferred to the sheer lack of data you encounter when you get that high.
  11. Can you make an analysis of the win rate for a Pokemon / team type?
    W/L ratio is a horrible metric on PS, because the ladder tries to pair players of similar skill levels. In a perfect world, everyone's W/L ratio would be 50/50, regardless of skill, because you'd always be playing people on your own level. You also don't want to use winrate or average rating or anything like that to measure a Pokemon's effectiveness, because then all you're measuring is how popular the mon is among n00bs. You might want to read into this a bit: http://www.smogon.com/forums/thread...re-of-how-far-a-pokemon-can-take-you.3546373/
  12. Can I perform my own analyses?
    If you have background with a programming language that can parse json, take a look in the "chaos" folder of each month's stats. Those files contain all the information used to generate the moveset statistics.
  13. Why did this Pokemon drop multiple tiers at once?
    The Pokemon had previously risen multiple tiers at once earlier during that generation. If nothing major changed since then (for example, a game release), it drops back down to where it came from. See also the Policy Review thread on drastic tier shifts.
 

Jaajgko

I will disband the soccer club
is a Tiering Contributor Alumnus
Pretty surprising that Zamazenta-Crowned sees that much usage I don't think it brings a lot of utility to teams that Zacian-Crowned already does and Choice Band / Choice Scarf Zamazenta seems overall more interesting, but I guess part of it is the hype surrounding the new legendaries.
 

Jaajgko

I will disband the soccer club
is a Tiering Contributor Alumnus
^ very true, having one thread for announcements and another for discussion would be good, because atm it's one for both which makes it hard to find the actual stats among the metagame discussion. Make one of them admin only and the other free for replies. I'd recommend this one is the marty-only one and the other is for discussion, and both get renamed.
 
So, what exactly is the point of this thread? Everyone else is using the other one.
The previous threads from Antar in gen 7 used to have the following note in the op:
NOTE: DISCUSSION IN THIS THREAD WILL BE LIMITED TO STATISTICS CALCULATIONS, CLARIFICATIONS AND OVERALL TRENDS. DISCUSSIONS OF INDIVIDUAL POKEMON WILL BE DELETED (each Pokemon has its own thread--discuss there). POSTS THAT SIMPLY QUOTE OR REFERENCE STATISTICS WITH PERSONAL COMMENTARY WILL BE DELETED. POSTS DISCUSSING HYPOTHETICAL LOWER TIERS WILL BE DELETED.
 

Champion Leon

Banned deucer.
The previous threads from Antar in gen 7 used to have the following note in the op:
For BH Gen 8:
Hm. In either case I am very surprised, but happy, to see Darmanitan-Zen Galar make it in the top 30 in all 4 rankings. I see the overall trend in the need for Steel and Dragon coverage is higher this generation, and with its STAB covering both, I am happy to see it get put to good use.

A major trend I see is the either Offense or Defense focus, as besides the cover dogs, it’s rare to see a balanced Pokemon like Type: Null, etc. It seems people were very aware of the biggest threats and knew right away how to counter them, but didn’t realize Dusclops is 8.25% bulkier than Corsola-Galar when they both hold Eviolite, as Coreola has a much higher usage than Dusclops.

It was only a 2 week ladder, but interesting nevertheless.
 
Is the next uu banlist going to be based on december stats only or an average of november and december stats (since dynamax ban happened I'm not sure how relevant november stats are now) ?

Also will it still be the past 3 months involved in computing tier shifts this gen ? The changes for gen 8 tiering mention removing weighted average but don't give us the duration.
 

Marty

Always more to find
is a Site Content Manageris a Battle Simulator Administratoris a Programmeris a Member of Senior Staffis a Community Contributoris a Top Researcheris a Top Tiering Contributor
Research Leader
Is the next uu banlist going to be based on december stats only or an average of november and december stats (since dynamax ban happened I'm not sure how relevant november stats are now) ?

Also will it still be the past 3 months involved in computing tier shifts this gen ? The changes for gen 8 tiering mention removing weighted average but don't give us the duration.
Until every lower tier is formed, all usage will be based on the previous month using the 3-month cutoff of 4.52% to accelerate accurate tier formation. When PU leaves Beta in May we'll start up the usual 3-month shift schedule again, where month 1 will have a quick drop cutoff of 1.53%, month 2 a quick drop cutoff of 2.28%, and month 3 the new standard of 4.52% for drops and rises. Month 2 and month 3 will be averaged evenly with the other month(s) in the cycle, instead of the old 20:4 and 20:3:1, respectively.
 
How is the release of pokemon home going to be handled for tiering ? Is it going to be ignored and we'll have tiers based on things being available only for half of the month ?
 

Champion Leon

Banned deucer.
How is the release of pokemon home going to be handled for tiering ? Is it going to be ignored and we'll have tiers based on things being available only for half of the month ?
Good question. Marty I know you may not know / have the answer, but can you confirm who would?
 

Marty

Always more to find
is a Site Content Manageris a Battle Simulator Administratoris a Programmeris a Member of Senior Staffis a Community Contributoris a Top Researcheris a Top Tiering Contributor
Research Leader
How is the release of pokemon home going to be handled for tiering ? Is it going to be ignored and we'll have tiers based on things being available only for half of the month ?
Good question. Marty I know you may not know / have the answer, but can you confirm who would?
Stats for March will go ahead as if HOME came out the first day of February. Either way, OU and Doubles OU see enough activity to support Pokemon being released at almost any point during any given month, so this latest batch being available for 19 out of 29 days isn't so bad.
 
Hi guys,

I was looking for in depth usage stats for every tiers. I found this https://www.smogon.com/stats/2020-02/chaos/ and it has been really useful for me.
However I would like to know how the values in "Items", "Abilities", "Teammates" or "Moves" objects are computed. Is there a way to convert these values into statistical frequencies for example ?

Thanks !
 
Is anyone else having trouble loading the movepool stats? I've tried both using WiFi and using data, as well as using other websites (everything else loads fine). I can see some stats at the top, but nothing ever loads below it. I have tried using the "find on page" tool, but nothing loads.
 

Marty

Always more to find
is a Site Content Manageris a Battle Simulator Administratoris a Programmeris a Member of Senior Staffis a Community Contributoris a Top Researcheris a Top Tiering Contributor
Research Leader
Stats for March are now up!
_______
Hi guys,

I was looking for in depth usage stats for every tiers. I found this https://www.smogon.com/stats/2020-02/chaos/ and it has been really useful for me.
However I would like to know how the values in "Items", "Abilities", "Teammates" or "Moves" objects are computed. Is there a way to convert these values into statistical frequencies for example ?

Thanks !
Everything included in the chaos files is computed here: https://github.com/Antar1011/Smogon-Usage-Stats/blob/master/batchMovesetCounter.py
Is anyone else having trouble loading the movepool stats? I've tried both using WiFi and using data, as well as using other websites (everything else loads fine). I can see some stats at the top, but nothing ever loads below it. I have tried using the "find on page" tool, but nothing loads.
It sounds like your browser has issues loading the whole file at once? Try right clicking and saving a file locally to view it that way.
 

Lily

wouldn't that be fine, dear
is a Tutoris a Site Content Manageris a Top Social Media Contributoris a Community Leaderis a Community Contributoris a Top Tiering Contributoris a Top Contributoris a Smogon Media Contributoris a member of the Battle Simulator Staffis a Dedicated Tournament Hostis a Senior Staff Member Alumnus
UU Leader
Heya! Prefacing this with the fact that I'm far from an expert on statistics, but I have an inquiry about the recent UU stats. I also don't want to come off as disrespectful, so sorry in advance if I do!

So, today we saw Steelix and Snorlax rise to UU from RU - this was a huge surprise to UUers and RUers alike, because while these Pokemon aren't bad in UU, per se, they're far from stellar and are not consistently used in tour play or on the ladder. Their usage at 1630 was as follows:
Code:
| 28   | Steelix            |  6.86598% | 1750   |  2.102% | 1537   |  2.287% |
| 40   | Snorlax            |  4.64354% | 2587   |  3.107% | 2095   |  3.117% |
This seemed highly abnormal to begin with - after all, just last month Steelix was at 1.86%, and Snorlax was at a mere 1.37%. Originally I had put this down to something like a potential metagame shift I hadn't noticed, or a laddering anomaly - but these numbers are just way too high. On further inspection, though, I noticed even more outliers. The biggest one was Qwilfish.

Qwilfish is a Pokemon that's not even ranked on our viability rankings - this isn't normally the biggest of deals, the ladder loves to use Pokemon considered unviable (see something like Arcanine in the last generation of UU, which was considered totally unviable but remained UU by usage for a long, long time). The reason it's weird in this case, though, is that Qwilfish is a) far from a fan favourite, b) not UU by usage (so it wouldn't show up in the teambuilder and therefore newer players are less inclined to automatically put it on their team), and c) had very few raw usages:
Code:
 | 44   | Qwilfish           |  4.05179% | 761    |  0.914% | 693    |  1.031% |
This is very high on its own, but what's even stranger is that when you change to 1760 stats (there are only 2 players who consistently maintain a rating above 1760 on the ladder, and only one of those is above 1760 on the ladder right now), Qwilfish's usage goes even higher:
Code:
| 33   | Qwilfish           |  6.03820% | 761    |  0.914% | 693    |  1.031% |
This is very abnormal. This trend can be observed in other Pokemon, too, primarily those that show up in stall (the two top players, pif and hjkhj, often ladder with stall) - Vileplume at 1630 has ~2.4% usage, but sits at ~5.1% at 1760. Quagsire is at ~1.7% at 1630, but ~4.5% at 1760.

I'm not sure if this is just down to how weighting works, and I would love to have this explained to me if so. But the point of this post is that I'm unconvinced that 1630 stats are currently an inaccurate way to determine usage for lower tiers. On the UU ladder, only 9 players are >1629 right now, while on the RU ladder, it's only 1. This has meant that stats are extremely skewed and has resulted in what I would consider "unfair" tier shifts - in this case, UU has taken two very important Pokemon from RU, two Pokemon that are not truly relevant in our metagame and do not really depict it. Perhaps the weighting could be re-looked at? I'm sure there were no errors in the actual pulling of the usage statistics, but this doesn't seem entirely accurate at all. Thank you in advance. :>

quick edit: i really don't want this to seem like an attack on hjkhj or pif! they're both excellent players who just happen to be high on the ladder, this weighting situation thing isn't their fault at all, sorry if it came off accusative!
 
Last edited:

Rabia

is a Site Content Manageris a Top Social Media Contributoris a Community Leaderis a Community Contributoris a Smogon Discord Contributoris a CAP Contributoris a Top Tiering Contributoris a Top Contributoris a Top Smogon Media Contributoris a Battle Simulator Moderator
GP & NU Leader
Usage-based tiering is inherently flawed because it allows a select group of ladder heroes to dictate tiering shifts to an extent (see: xbossarux being the sole cause of Mesprit rising to NU last generation despite it being god fucking awful). I have no idea what good shifting weights would actually do because it just changes the group of people that are capable of causing unrealistic changes to occur, but what Lilburr points out is at least concerning to me because metagames shouldn't be forced to randomly adapt as a result of the dedication of some few users.
 

quziel

I am the Scientist now
is a Site Content Manageris a Forum Moderatoris a Community Contributoris a Smogon Discord Contributoris a Top CAP Contributoris a Contributor to Smogonis a member of the Battle Simulator Staff
This is a tiny nitpick, and doesn't fully detract from the point, but iirc the weighting used for usage stats is Glicko-2 rating, rather than elo; for example my main account has 1524 elo in NU with a 1768 ± 26 glicko-2 rating. Because of the confidence intervals iirc they can weight your contribution to the usage stats by the probability that your true rating is 1695 or higher.There still are issues with these thresholds and well, smaller ladders allowing a small number of players to heavily influence usage stats, but that's why lower tiers typically use lower thresholds for their tiering than higher ones.
 
Last edited:

Adaam

إسمي جف
is a Community Contributoris a Forum Moderator Alumnusis a Top Tiering Contributor Alumnusis the 8th Grand Slam Winner
Heya! Prefacing this with the fact that I'm far from an expert on statistics, but I have an inquiry about the recent UU stats. I also don't want to come off as disrespectful, so sorry in advance if I do!

So, today we saw Steelix and Snorlax rise to UU from RU - this was a huge surprise to UUers and RUers alike, because while these Pokemon aren't bad in UU, per se, they're far from stellar and are not consistently used in tour play or on the ladder. Their usage at 1630 was as follows:
Code:
| 28   | Steelix            |  6.86598% | 1750   |  2.102% | 1537   |  2.287% |
| 40   | Snorlax            |  4.64354% | 2587   |  3.107% | 2095   |  3.117% |
This seemed highly abnormal to begin with - after all, just last month Steelix was at 1.86%, and Snorlax was at a mere 1.37%. Originally I had put this down to something like a potential metagame shift I hadn't noticed, or a laddering anomaly - but these numbers are just way too high. On further inspection, though, I noticed even more outliers. The biggest one was Qwilfish.

Qwilfish is a Pokemon that's not even ranked on our viability rankings - this isn't normally the biggest of deals, the ladder loves to use Pokemon considered unviable (see something like Arcanine in the last generation of UU, which was considered totally unviable but remained UU by usage for a long, long time). The reason it's weird in this case, though, is that Qwilfish is a) far from a fan favourite, b) not UU by usage (so it wouldn't show up in the teambuilder and therefore newer players are less inclined to automatically put it on their team), and c) had very few raw usages:
Code:
 | 44   | Qwilfish           |  4.05179% | 761    |  0.914% | 693    |  1.031% |
This is very high on its own, but what's even stranger is that when you change to 1760 stats (there are only 2 players who consistently maintain a rating above 1760 on the ladder, and only one of those is above 1760 on the ladder right now), Qwilfish's usage goes even higher:
Code:
| 33   | Qwilfish           |  6.03820% | 761    |  0.914% | 693    |  1.031% |
This is very abnormal. This trend can be observed in other Pokemon, too, primarily those that show up in stall (the two top players, pif and hjkhj, often ladder with stall) - Vileplume at 1630 has ~2.4% usage, but sits at ~5.1% at 1760. Quagsire is at ~1.7% at 1630, but ~4.5% at 1760.

I'm not sure if this is just down to how weighting works, and I would love to have this explained to me if so. But the point of this post is that I'm unconvinced that 1630 stats are currently an inaccurate way to determine usage for lower tiers. On the UU ladder, only 9 players are >1629 right now, while on the RU ladder, it's only 1. This has meant that stats are extremely skewed and has resulted in what I would consider "unfair" tier shifts - in this case, UU has taken two very important Pokemon from RU, two Pokemon that are not truly relevant in our metagame and do not really depict it. Perhaps the weighting could be re-looked at? I'm sure there were no errors in the actual pulling of the usage statistics, but this doesn't seem entirely accurate at all. Thank you in advance. :>

quick edit: i really don't want this to seem like an attack on hjkhj or pif! they're both excellent players who just happen to be high on the ladder, this weighting situation thing isn't their fault at all, sorry if it came off accusative!
Any response to this Marty? I know stats don’t lie but Steelix having 6.8% usage in UU is just highly suspect to me when the past two months it had 1.868% and 2%.
 
Last edited:
Status
Not open for further replies.

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top