Research Results: Opinion Research on Tiering, Looking Forward to XY

[Chou Toshio]

I have studied the results, and put together a full report of key findings.

These Key Findings Can be Downloaded Here: Report on Key Findings of Survey
PLEASE READ THE NOTES ON EACH SLIDE-- THEY ALSO INCLUDE IMPORTANT INFO NOT INCLUDED IN MAIN POWER POINT

Also, the results themselves (raw data report) can be found here: Raw Results


Major Topics Include:
-Community Perceptions of Smogon
-Importance of Smogon Site Areas to Mission Statement
-Ratings of Loyalty to our community
-Ratings for Excellence of SmogonMetagames and OU
-Ideal Metagame Characteristics
-Tiering Philosophy Sentiments
-Opinions on Weather in BW
-Opinions on Strict Cartridge Mechanics
-Opinions on Initial XY Tiering (Process)
-Votes on Pokemon to be Initially Unbanned in XY


Please use this thread to discuss research results, and or ask me any questions about the answers. I can also answer questions about information shown here, such as the answers to question based on demographics (How did just UU players answer this? How did just players with 4+ years of experience answer? etc.).

-Love how Uber players rate OU better than OU players do (15% top box v. 12%), while UU players give OU the worst raitings (1.4%).

Also if OU Mods or Tiering Leaders want to keep these results private, please let me know-- otherwise I will eventually make these results public in the Smogon Metagame forums. Other users, please keep the results confidential for now.

Thank you, that is all.

Chou

 

[Chou Toshio]

So brief summary of important findings:
Notes on loyalty:
-Smogon's contributions to competitive Pokemon were rated extremely well. 70% of players gave Smogon's contributions 9 and 10 ratings.
-TCPi and GF's ratings are surpringly low-- barely over mean 5 (smogon is close to mean 9), and only about 5% of players gave them 9 and 10 rankings. This is a bit concerning, as I worry about our community's ability to cooperate with official events and so forth.

Importance / Excellence of Site Sections:
-Strategydex, Smogon Metagame Tiering, Smogon Metagame Discussion, and Official Tournaments were the most important sections in that order.
-Strategydex, Official Tournaments, The Smog, and Smogon Metagame Discussion were the most excellent sections, in that order.

^While Smogon's Tiering is ranked number 6 in site areas, its ratings are not that far distinguished from the top sections (besides C&C which has incredible excellence ratings). Players rate the excellence of Smogon's Tiering as fairly high, and believe it to be of great value. This can also be seen in the high excellence rating of Smogon's Metagames overall.

-Over performing site areas (Excellence rating > Importance rating): The Smog, Non-Strategydex C&C articles.


Obviously, Tiering is rated as extremely important. The vast majority of our players would continue playing our tiers, even if Official Singles was endorsed.

OU's Tiering was rated as VERY important to players. Over 50% of OU players gave OU's tiering a 9-10 (out of 10) rating of importance, and even in other tiers, 1/3 to almost 1/2 of players in every other tier rated OU's tiering importance 9-10 out of 10.


However, while OU's tiering was rated as very important, satisfaction with OU is fairly low.
Smogon metagame's rating of excellence overall is 7.7. OU's rating of excellence is 5.6


Metagame characteristics
In the metagame, diversity is fairly important (but not too vital), and concern with over diversity is fairly low. Players prefer a slightly offensive game.


Weather
Generally speaking, players WANT themes, and think Weather is a distinct characteristic of BW that should be incorporated in OU--

-However, a majority of players are dissatisfied with how Weather has been controlled.
-However (again), the margin is surprisingly slim-- 56% Dissatisfied v. 42% Satisfied.



Faithful Simulator Mechanics
Generally speaking, players care about faithful mechanics, but do NOT want faithful mechanics in regards to Glitches, Clauses, or Information Reporting.

-Players who demand faithfulness to cartridge mechanics in these areas are a very small minority (just over 10% for Glitches and Clauses, and 20% in regards to information reporting)

-While I expected "Pragmatic Reasons" or "Competitive reasons" to be highly sighted, in regards to these 3 segments: "Not important to follow cartridge mechanics at all" was the most highly selected option overall (2nd highest by a very small margin in regard to clauses).


Initial Tiering Course of Action for XY Tiering
-The majority of players want a similar start to BW OU, with 670+ BST Pokemon banned.
-A fourth of players want some 600- Pokes such as Deo-A and Darkrai banned additionally to the 670+ crew.
-Though a minority, a substantial 22% of players want a complete unban.


670+ BST is a Stat-Significant Characteristics in regards to initial ban
-Using the data in a comparison test, one can be 99% confident that 670+ BST is a factor in how people vote on whether a Pokemon should be initially banned or not.
-Lugia (58) and Kyurem-@ (52) were the only 670+ BST Pokes to get more than 50 votes to unban.
-Deoxys-A (71) was the only 600- BST Pokemon to get less than 100 votes to unban.

Based on this, I think we have an objective basis to use 670 BST as a justification for initial bans.



That concludes the summary.

 

[SHUCKLE MAN]

Thanks for presenting these results. :)

With this situation about the initial banlist (or not) in Gen VI, I thought of a couple of other compromises which weren't included in the poll (since none of the categories in the poll gota vote of over 50%, so maybe other options could be considered):

1. Perhaps have a very brief period of testing (e.g. 1-2 weeks) with everything unbanned, and then come up with the initial banlist based on that brief testing period, rather than BSTs or how those Pokemon fared in previous generations. It won't be a full-on test of the Pokemon, but it would be very short, so that the people can play Gen VI "OU" very quickly (as everybody is always excited to start playing when a new generation gets released).

2. Have two ladders to start with. One with everything unbanned (where the tests can be conducted), and another with an initial estimated banlist (so that players who are eager to play OU rather than "Ubers" can get into it right away). When a proper banlist starts getting worked out, then the two ladders can be merged, so to speak. (Sorry if this suggestion doesn't make sense, it's difficult to explain, but it makes sense in my head at least).

 

[Homeslice]

I went through the Powerpoint, great effort put into all of this, thank you for it. I think this is generally the most unbiased representation of players' opinions we can get, so thanks again.

I like how dedicated Smogon is to Smogon, especially apparent by the theoretical 6v6 official singles metagame.
What surprises me the most is the Pokemon voted on for insta-ban at the beginning of XY. This list is ridiculous. I guess it just shows how Speed based OU is. Correct me if I'm wrong, but was Arceus the least voted for? Because that's what I'm seeing. I'm assuming the graph is based off of people thinking "Oh well these are all getting banned anyways so I want these banned ALSO"
But seriously, Groudon, Kyogre, and Mewtwo all being that low is stupid. Kyogre should always be insta-banned from OU imo, most powerful poke in the game.

 

[MikeDawg]

The fact that 22 and 24 have opposite answers but were essentially the same question makes me question somewhat the validity of the voter pool...


Shoutout to the 5 girls out there

 

[Chou Toshio]

Home slice, the Pokemon voted for are the ones to be unbanned-- so arceus, Kyogre and co would be banned

 

[Princess Bubblegum]

Would you mind posting the initial ban list voting decisions per Pokemon Chou? my current computer doesn't understand power point.

 

[Chou Toshio]

Scarf, if you click on the raw results, there should be a nice pretty graph showing it-- it's question 19.

 

[Homeslice]

Whoops, this is what I get for skimming. Data makes a lot more sense now bar Genesect imo