Continuing Matchmaking Updates
Today’s update is once again centered on improving our Matchmaking system, focusing primarily on fixing issues that arose from our previous update.
We understand that some players, especially at the high end, have experienced a significant drop in matchmaking quality as a result of our recent iterations. We appreciate all of the feedback and match IDs that we’ve been receiving—your input has been very helpful and we hope you’ll continue to give us feedback after this update as well. We are committed to making matchmaking as good as it can be.
Please check out some more detailed information on the different aspects of today’s update below.
High MMR Partying
====
Due to the unusually high frequency of matchmaking abuses found in party games in high level matches–we’re adding a new rule to party matchmaking for Divine and Immortal ranks to help reduce some of the negative behaviors that have become common at these ranks and to help make solo queuing more viable. For any party that includes a Divine or Immortal player, every player in that party will be considered the same rank as the highest player in that party. While this will significantly reduce some of the more common abuse scenarios, the trade-off is that it will also affect “normal” parties with large MMR disparities. We think this trade-off is worthwhile at this high level, however, because matches are unlikely to be of high quality anyways with extremely disparate skill levels within a party given the shallow pool available. Such disparity has often made the games less fun for most players involved. We expect that this will have the net effect of reducing the number of party games at the highest skill levels by some amount, but compared to how infrequent high-MMR party games were in the past, the number of party games should still be higher than the historical average.
Matchup Combinations
====
We are making changes regarding what solo and party matchups are allowed. For any five-player party, they will now only be matched against other five-player parties, regardless of any other matchmaking consideration. We are unsure if an absolute requirement like this will result in net higher quality matchmaking due to the more limited opponent spectrum available in that case, but we know this is an aspect the community talks about often and we are going to do our best to work within those constraints. Similarly, any solo player will now only ever be considered for matches against at most one party of two. This means that from a solo player’s perspective, matches will always be either against a team of all five solo players or against three solo players and one party of two. This will be a hard requirement for the matchmaker instead of a situational consideration.
Match Quality and Role Symmetry
====
As a result of some of the new features we added recently, we found numerous bugs in the matchmaker’s evaluation that caused it to incorrectly rank various considerations. We’ve fixed those bugs and are expecting them to have a notable improvement in matchmaking. We will be monitoring the system after release to make sure the fixes have the intended effect. It is possible that there will be more issues to work out, so if you notice anything too far off please let us know.
We’ve also done various tuning to make Role Symmetry perform better, as there have been a few cases at certain brackets where it did not do a good job at all.
Finally, we are putting a bigger emphasis on having supports not be the highest MMR players in a game, to the extent that it’s possible within parties. We’ve heard feedback from both core and support players that they would prefer the intra-match rank allocation to be that way.
Medal Rework
====
We are back to 5 stars per medal now, 7 stars didn’t add enough upside compared to the visual noise and confusion.
In previous ranked seasons we’ve had a mechanism that caused the rank ranges that a given medal represents to shift over the season. While this gave some loose sense of progression, it made comprehension of what medals represent more difficult. It especially caused confusion after calibration where most players dropped medals despite winning most of their games.
As a result of removing this mechanism, and the star count re-scale, we are rebalancing what MMR range each medal represents. The distribution model we are using for this is the curve we had in the final months of the previous season (and what we’ve had historically). This will help ensure that medals long term represent the same skill level at the beginning of the season as they do at the end.
We are expecting there to be some confusion as a result of this rebalancing, since most players’ medals will be adjusted upwards somewhat to be more in tune with what the medals meant at the end of last season, but there is no actual change to anyone’s MMR number.
High-Water Mark
====
In the past, once a player reached a certain rank medal, they would retain that medal for the remainder of the ranked season, even if their actual MMR fell dramatically below that medal’s threshold. With today’s update, your medal will no longer represent a strict high-water mark. Instead, if you are ever more than one star away from your actual current rank, your medal will be adjusted to reflect that.
We originally added the high water mark because we thought it helped with player anxiety in ranked games. We think that is still true to some degree, however our data suggests that a one-star buffer captures most of that benefit for the majority of players. There are multiple downsides with having too broad of a spectrum on the high-water mark. First, it makes match fairness really hard to evaluate, because we always match based on MMR and not medals, and thus sometimes a game can appear to be good or bad as a result of the medals not representing reality. As it relates to the above goal in making it easier to evaluate matches, we think a one-star buffer helps balance both goals.
The second benefit is that it makes account buying indirectly less valuable. A player can no longer be on another account and keep a high medal for an entire season. Moreover, this change will also work in conjunction with the smurf detection system we’ve been working on. We are currently only using smurf detection to increase ranks upwards to the correct levels, but soon we plan to have it work in the reverse direction as well. With high-water marks only holding onto one star’s worth of a buffer, boosted or purchased accounts will now fall faster and lose the medal along with them.
Post-Game Match Evaluation
====
We’re very thankful for the feedback we’ve received in the last few weeks. We think the feedback people had was well rooted in real issues. In our eyes, creating high quality matches is the most important thing we can do to help players enjoy the game, and we’re committed to improving our matchmaking system to achieve that goal.
In an effort to help facilitate the discussions on matchmaking, we’ve made a couple small changes to the post-game screen. The post-game screen will now show the average and max queue times for players in the match to help you determine if we formed a bad match too quickly or not. A summarized behavior category of the match will also be displayed, to help players understand if the reason for the lack of a better match was a shallower pool as a result of poor behavior or if it was because of the matchmaker doing a bad job. Finally, we are sorting the list of players in the post game screen from left to right with position 1 through 5 (parties will have a new type of indicator on them still so you can still see what the allocations were). We are hoping this makes it easier for you to evaluate if Role Symmetry is working as expected or not.
We recognize that adding more information here may cause a short-term increase in complaints rather than a decrease, but our long-term goal is to create the highest quality matches, and having more information exposed to players will ultimately lead to that end.
Post-Game Survey
====
We’ve added a new post-game survey that is similar to ones we had a long time ago. Once a match has concluded, players will sometimes be surveyed to rate the experience in a match. This will allow us to gather more player sentiments on match quality to help find common patterns to what makes matches enjoyable vs what makes them unenjoyable. It’ll also help us find bugs or mistakes in our decisions sooner than later. Of course it’s natural that players will be more likely to equate winning a match to having a good match and losing a match to having a bad match, so we’ll normalize the data for those obvious scenarios to help filter through the background noise and bubble up useful signals. We’ll hold off on enabling this survey for a few days, to work through some bugs and update-related issues first.
Communication with Low Behavior Score
====
Our last update included a wave of bans on users that were at the very extreme edge of the player population in terms of toxicity and negative behavior. However, those bans affected only a very small percentage of all players, and we also wanted a more gradual system to address players that are not at the furthest extreme but are still big outliers from the general player population. To that end, players that have a behavior score below 3000 are no longer able to use chat or voice until their score rises above the acceptable threshold. We believe this more gradual user-facing reinforcement mechanism will be valuable for both protecting the larger population from outliers and as a warning system for players who are moving in the wrong direction that might encourage them to improve. We set the threshold score at 3000 due to the low probability that anyone would be that low without having had a consistently negative impact on the experiences of many different teammates.
Party Spread
====
As we’ve mentioned, extremely large MMR spreads within a party makes it hard for the matchmaker to find compatible opponents, and even in cases where it can, the matches are not necessarily enjoyable for the players involved due to some players having low impact/agency in the game while others have an outsized one. We recognize, however, that players still want to play with friends, so this aspect is not without trade-offs. To help address that, we’re reducing the acceptable MMR spread within a party from 2800 to 2000. We’ll consider further reductions to this spread over time, as the current value is still quite large, but we’d like to find the correct balance incrementally. This limitation only applies to ranked matchmaking.
Core Matchmaking Code
====
We’ve made a significant revision to how the heart of the matchmaker iterates through the many players that are queuing at any given time. This revision restructures the matchmaker’s search process to have more flexibility with how considers the increasing number of requirements. Because the matchmaker must segment the large number of players into smaller combinations to consider, it was possible that the list of evaluable matches weren’t the highest quality in cases where the matchmaker excluded players due to soft considerations early in the process. Previously it was not very efficient if there were too many considerations within a team and against a team. Because of the nature of this rewrite, we are expecting there to be bugs for the next few days that may cause matchmaking to be sub-optimal, but are hopeful that this investment will allow us to more easily increase matchmaking quality in the long run.
Reports and Player Bans
====
This update will remove some methods of reporting that were not rate limited in order to improve the signal-to-noise ratio, especially at the conclusion of a game.
We have also been refining our booster and exploiter detection algorithms and have been doing regular ban waves in the background. We will continue refining these systems over the coming weeks. Banned players will also no longer show up or consume rank spots on the leaderboard. This fix was implemented a few days ago, as some of you may have noticed due to leaderboard ranking shifts.
Going Forward
====
We are expecting the next day or two to be a little unstable as we iron out issues with the update, but after that we’d appreciate continuing with the feedback on your match experiences. Please make sure to always include a matchid with your feedback, as we often find them missing that crucial data needed to follow up.
Our focus going into the next few weeks will be on monitoring and measuring the results of this update and following up as needed based on what we learn. The other big category we are going to be focusing on soon is revamping the new player experience for an update in the future. In addition, a lot of the ongoing work we are doing on smurf detection will indirectly help us in the future with placing new players into more ideal matches once they’ve completed their initial new player experience phase. We are hoping that after we get matchmaking into a better spot, it’ll be an ideal time for brand new players to join in as well as for existing players to invite their friends into the game with the help of a better new player ramp.
by Dota Team via Dota 2
Post a Comment