Tom Clancy’s Rainbow Six Siege

A short history of bug reports on r/Rainbow6 & what’s next

Rainbow7 - A short history of bug reports on r/Rainbow6 & what's next
Loading...

Hello there and happy Neon Dawn launch day!

We hope everyone is already enjoying the
neondawn - A short history of bug reports on r/Rainbow6 & what's next


new season and got a chance to try out Aruni and the Skyscraper rework.

With today's Neon Dawn launch, we want to provide an update on one of our rules. As many of you will know, we currently do not allow bug reports to be posted to this subreddit. In this post, we will go over how we arrived here and what we're doing now to address some of the concerns with this approach.

TL;DR at the bottom of the post.


Attack of the Bug Reports

A long time ago in a subreddit far, far away…. it was Year 2 Season 1: Operation Velvet Shell launch day. We'll start here because this is about as far back as I can remember.

Back then, the game had a lot of problems. The newly released Mira launched completely broken as you could open her window by meleeing it from the outside. The X-Kairos pallets from Hibana, who was just released three months earlier, were not detonating reliably, rendering her borderline useless. Bullets coming out of certain weapons, including Jackal's new C7E, weren't landing exactly where you were aiming. Everyone that can remember back that far can attest that the bug situation was much, much worse back then than what it was in the recent past.

r/Rainbow6 reflected the state of the game quite well at that tome. As an avid user of /new back then, I can tell you that the subreddit was flooded by posts complaining about all the bugs, connectivity issues, and matchmaking problems plaguing the game. I don't have any hard numbers to back up this claim but about one in every three posts was about the broken state of the game. It was certainly fatiguing to see the same posts over and over again.

I personally wasn't part of the moderation team then so I can't provide any context from that point of view but even a regular user could tell that the subreddit was drastically different compared to what it is now. In early 2017, right around the Velvet Shell launch, r/Rainbow6 has surpassed 100,000 members. The overall volume of posts was much lower and it was possible to see and check every single post here which also meant that Ubisoft could pick up every bug report. With the alternatives of Twitter and the Ubisoft forums in mind, r/Rainbow6 has arguably the best place to post bugs to.

But it was clear that the situation wasn't sustainable, neither from the perspective of this community nor from Ubisoft's perspective. Because of this, changes needed to happen.


A New Hope

With the Velvet Shell launch, game health was at a low point. It was clear to every player that things needed to change so Ubisoft introduced three separate measures:

  1. Technical Test Servers
  2. Operation Health
  3. R6Fix

The Technical Test Server was launched before Velvet Shell was even over. They were quite similar to the Test Servers now, the testing periods were just a lot shorter.

Ubisoft also took the controversial step of canceling the originally planned Year 2 Season 2 to introduce Operation Health. During Health, they fixed a lot of the then most relevant bugs, introduced one-step matchmaking, and laid the groundwork to build a more sustainable game.

Later in Year 2, specifically with the launch of the Season 4: Operation White Noise Technical Test Server, R6Fix was introduced. For the first six months, it was only used to collect bug reports for the TTS.


The Rise of R6Fix

R6Fix was first introduced to the live servers with Year 3 Season 2: Operation Para Bellum. At that point, r/Rainbow6 had just shy of 400,000 members. The increased popularity of Siege and therefore of this subreddit meant that the idea of what content should and shouldn't be allowed had to evolve. The approach to moderation has to change drastically from being a tight-nit community, to passing 100,000 members, to passing 400,000 members, and even to now passing 1,250,000 members.

Up to the 400,000 mark, people were allowed to post bug reports as they pleased but that was becoming a problem. Even though the situation wasn't as dire as it was during Velvet Shell, there were still a number of bugs in the game and r/Rainbow6 had now inadvertedly established itself as the place to post bug reports to.

This meant that a relatively large portion of all posts were still bug reports. Unlike before, the increased volume meant that (a) a lot of duplicates were being submitted which did nothing to help solve the issues and (b) it wasn't realistic to keep track of every single report that was being posted here. Due to Reddit's structure and our size, r/Rainbow6 was simply not a good place to report bugs to anymore.

All these issues could be solved overnight with the introduction to R6Fix. After careful deliberation leading up to its launch, we jumped right on it with the Para Bellum release. A rule was introduced that disallowed all bug reports. Period. No exceptions. Instead, poster were forwarded to R6Fix to use it in place of the subreddit.

This rule had changed the face of the subreddit like no other, something that is still true to this day. To truly achieve this shift, we had to eliminate the image of r/Rainbow6 being the go-to place for bug reports which was ultimately the reason for why we were so firm on our stance and made no exceptions. In the end, the rule read "Do not post any bug-related content".

At the time, R6Fix was a perfect solution to the problems we were facing. Yet, as it turned out, it wasn't so perfect after all and problems with our approach emerged over time.


R6Fix & You

Since its introduction, we received quite a lot of feedback for our "no bug reports" rule. While there we certainly a lot of valid points being brought up, we also picked up that there are a lot of misconceptions floating around relating to R6Fix.

In the following paragraphs, I'll go over the main concerns we've heard about our approach. This includes hopefully clearing up some of the misconceptions and for that purpose, I had a chat with community developer Karen Lee (aka u/UbiKalyrical) to get some insight on how R6Fix reports are utilized at Ubisoft to improve the game.

Barely any reports are reaching the reproduction threshold

The perhaps most notable criticism of R6Fix is that it's relatively inactive, that reports only get a few (if any) reproductions, and that the reports aren't reviewed by Ubisoft because they don't reach the necessary level of engagement.

In reality, none of this is a problem. Like any other software project that has more than a handful of engineers working on it, Siege has a quality control team, or QC for short. As the name implies, its purpose is to measure and help improve the quality of the product which includes trying to find bugs, finding ways to consistently reproduce these bugs, and forwarding that information to the engineering team so that they can easily identify the cause of a problem and eliminate it.

Siege's QC team uses R6Fix as a supplement to their regular work. Especially during the launch week of a patch, whether on the live servers or the TS, the team is scouring R6Fix for new reports and tries to reproduce the issues. The resulting information is then passed on to the respective engineering team leads internally so that they can start work on a fix. Even outside of launch weeks, QC is constantly checking up on R6Fix reports.

All of that happens independently of any further user interaction on the platform. Instead, user reproduction and votes are used to complement the work that is already being done. When a report reaches a certain number of user reproductions, ten for live servers and five for test servers, an automatic report is generated in Jira, a tool that is used to internally track player-reported bugs. In reality, most of these problems are already caught in advance by the QC team before they reach the threshold.

All in all, if you submit a valid bug report to R6Fix with steps to reproduce included, it is extremely helpful for getting the problem solved, completely independent of how many other users even see the report.

I reported a valid bug but it still isn't fixed

This is an understandable point of frustration. You take time out of your day to submit a report of a bug that definitely exists, yet, months later, it still isn't fixed. Not very rewarding to say the least.

I don't want to suggest that everyone who has every experienced this has filed a bad report but there is definitely a right and a wrong way to submit a bug to R6Fix. So I asked Ubisoft, "how does a report help identifying and fixing problems"?

Filling out a report properly and comprehensively is the most important aspect.

For one, this includes a concise description of the behavior you're reporting and a description of what you would instead expect to happen.

Загрузка...

Maybe the most important part of a great report are very clear steps to reproduce the issue. Unfortunately, this tends to be quite hard for a non-technical person to nail down. Every little action you take when encountering a problem should be documented in your report, maybe even down to the timing of your button presses depending on the issue you're facing. It is very understandable that this may seem overwhelming but don't be discouraged to submit a report anyway. Every little bit of information you provide is helpful at the end of the day, even a report that's not perfect might help trigger a breakthrough that finally fixes the bug. Including videos also helps to speed up the reproduction process massively. While you might not be able to pick up on a pattern that causes a bug, a trained professional on the QC team might be able to but for that they need a video that they can look at multiple times. If you aren't confident in your ability to submit a technical description of the reproduction steps, make sure to also attach a video.

A report also includes metadata on the system you're using. Before submitting your first report, you're asked to fill out information about the system you're using. This data can be used to check if a particular problem is PC-specific, specific to a console, or even specific to a certain graphics card.

We will compile a concise guide on how to submit a good R6Fix report based on what we heard from Ubisoft and surface it in relevant places.

But the bug is so easy to fix yet it's still there

Saying a bug is easy to fix is simply a fallacy for a project as large as Siege. While the code changes may in fact be simple, there are a lot more steps to the process. A short overview of the typical work that is needed to fix a bug in the game industry:

  1. The issue needs to be found, either by a QA tester or through a community report that is then validated by a QA tester.
  2. The issue needs to be scoped out to better understand what systems are related to it. This will include finding a procedure to reliably reproduce the problem.
  3. It needs to be evaluated if it is viable to fix the issue with a reasonable amount of work in the near future or if it should go somewhere in the backlog.
  4. The issue needs to be investigated more deeply to understand what the actual workload would be to fix it.
  5. It needs to be re-evaluated if it is viable to fix the issue now or if it should go somewhere in the backlog.
  6. The issue is fixed by expending a certain amount of man-hours.
  7. It needs to be validated that the fix is working by (a) confirming that reproduction steps aren't working anymore and by (b) trying to find other (new) ways to reproduce the original problem. If it turns out that it's not working, go back to step #2.
  8. It needs to be validated that the fix isn't negatively impacting any other systems. This may include re-testing a majority of the whole game. If it turns out there are now some regressions in other areas, go back to step #2.
  9. It needs to be validated that the fix works in conjunction with all the other changes made to the game. If it turns out there are conflicts, go back to step #2.
  10. The fix can be pushed to the end user.

Needless to say, this will always require a minimum amount of man-hours to be spent, no matter how trivial a problem may seem. This means that more important problems will be prioritized over effectively inconsequential problems like cosmetic bugs.

A process like this one isn't exclusive to Ubisoft either and chances are that you experienced its effects with other AAA titles, too.

There are too many duplicates

This should really go without saying but: do not submit duplicate reports. Having more than one report for a single issue doesn't help, on the contrary. Duplicates only make it harder to track a problem while simultaneous drowning out other newly submitted reports. Instead, search R6Fix before submitting to check for already existing reports, for example by looking at keywords that relate to your problem. When a report already exists, contribute to it by saying that you could reproduce the issue. You can also add additional comments to your contribution.

But the duplicate situation also touches on another R6Fix criticism: the moderation, or lack thereof. When asked about how all the duplicates are affecting the platform, Ubisoft said that the QC team members, who incidentally also moderate the platform, get overwhelmed with the number of duplicate reports, especially at the start of a new season. They had to make the decision to just let the duplicates be instead of merging all reports together because the time could simply be spent better elsewhere.

Removing bug reports on Reddit means that Ubisoft loses a valuable source of information

This is an argument that comes up due to an inherent bias with viewing this subreddit after it was filtered by moderation. While it is true that you don't see the removed posts, we still do. We're in contact with Ubisoft's community team and pass along hot topics of conversation as necessary, for example when a new exploit pops up. We can ensure you that no important is lost when we remove a post or comment.

Removing bug reports on Reddit suppresses community discussion

And now onto the biggest problem with our approach to removing all bug reports, the issue we'll try to fix with today's changes.

There is simply a need to discuss issues with other community beyond submitting a dry bug report and Reddit should be a great place to have these discussions. Yet, in reality, we remove any option for that which is detrimental to both the game and its community.

So, how do we solve this problem without reintroducing the problem that originally led us to remove bug reports, namely the subreddit being flooded by them? The answer:


Bug Megathreads

Starting with this season, we'll be hosting bug megathreads. We want to go over what we hope these will achieve and what they will not try to achieve.

What a bug megathread is & what opportunities it brings

The main structure of the post itself will be a list of the most relevant issues with the game at the time. Those may include newly discovered bugs but also major long-standing problems. All items on this list will come with a short description and a link to an R6Fix report that you'll be able to contribute to. We hope this will encourage everyone to contribute on R6Fix more and also in a much more focused manner. We also hope that this will be a great source for information if you're looking for details on a particular problem that you're encountering.

On that note, the comments section will be a place for open discussion. If you're in search of information about an issue that didn't make the list, you can still discuss it in the comments, for example to see if other players are running into the same problem or if it's just you.

That being said, this is just the format that we think will work well. We're expecting that these posts will develop over time to better suit the needs of the community. If you have some early feedback or ideas in general, let us know!

What a bug megathread is not & the limitations of this approach

Most notably, the arrival of bug megathreads will not change our stance on bug reports that will be posted to r/Rainbow6. Those will continue to be removed, although we will now include a link to the most recent megathread within the removal notice.

A certain type of problem will also never be included on the list, namely game-breaking exploits. The main concern is that an inclusion would further spread the issue across the community which is something we want to avoid.

I also want to mention a few of our policies that relate to game-breaking exploits that won't change at this time. We will continue to remove any content that explains how to use these exploits and warn the user in line with our "Do Not Harm the Rainbow Six Community" rule. Additionally, we will also continue to ban users that admit to exploiting game-breaking bugs, again in line with our "Do Not Harm the Rainbow Six Community" rule. As far as we are concerned, exploiting is cheating which means that these will be permanent bans.

As for the timing of these posts, we will start off by posting one for each seasonal update. This may or may not change in the future depending on how big the demand will be for mid-season updates.

We also cannot guarantee that the megathreads will be constantly stickied for the start of the season. Reddit sadly limits us to only two pinned posts at a time and more important information may take priority. That being said, we'll try to at least link the megathread in a stickied comment under the then-pinned posts.

Current Megathread

Coming with this announcement is also first megathread itself. You can find it here: https://redd.it/k4t6cw


Conclusion

The size of the subreddit is more or less forced us into removing bug reports. We hope the introduction of megathreads will solve the biggest problem with this approach, namely removing the option to discuss new bugs and other problems with the community. Sadly, megathreads still have their limitations but we're confident that they will evolve into something that's helpful for everyone involved.

We look forward to hearing your feedback!

TL;DR: There will now be seasonal bug megathreads, you can find the first one here.

Source: Original link


Loading...
© Post "A short history of bug reports on r/Rainbow6 & what’s next" for game Tom Clancy’s Rainbow Six Siege.


Top 10 Most Anticipated Video Games of 2020

2020 will have something to satisfy classic and modern gamers alike. To be eligible for the list, the game must be confirmed for 2020, or there should be good reason to expect its release in that year. Therefore, upcoming games with a mere announcement and no discernible release date will not be included.

Top 15 NEW Games of 2020 [FIRST HALF]

2020 has a ton to look forward to...in the video gaming world. Here are fifteen games we're looking forward to in the first half of 2020.

You Might Also Like

Leave a Reply

Your email address will not be published. Required fields are marked *