Some thoughts on improving Bugzilla

One of the keys to making an open source project work is getting feedback from users and developers about problems (bugs) in the code or system. Mozilla (the organization behind Firefox and Thunderbird) uses Bugzilla, but organizations have developed a variety of systems for dealing with this issue. For example, many cities use 311. I’m going to talk about Bugzilla and Mozilla in this case, but I think the lessons can be applied more broadly for some of my policy geek friends.

So first, some first principles. Why does getting the system right matter? A few reasons come to mind:

  1. Engagement: For many people Bugzilla is their first contact with “the community.” We should want users to have a good experience so they feel some affinity towards us and we should want developers to have a great experience so that they want to deepen their level of participation and engagement.
  2. Efficiency: If you have the wrong or incomplete information it is hard (or impossible) to solve a problem, wasting the precious time of volunteer contributors.

I also concede that these two objectives may not always be congruent. Indeed, at times there may be trade offs between them… but I think there is a lot that can be done to improve both.

I’ve probably got more ideas than can fit (or should fit) into one post so I’m going to unload a few. I’ve got more that relate to the negotiation and empathetic approaches I talked about at the Mozilla Summit.

One additional thought. Please feel free to dump all over these. Some changes many not be as simple as I’ve assumed. Others may break or contravene important features I’m not aware of. Happy to engage people on these, please do not see them as an end point, but rather a beginning. My main goal with this first batch of suggestions was to find things that felt easier to do and so could be implemented quickly if there was interest and would help reduce transactions costs right away.

1. Simplifying Menus

First. I thought there were some simple changes that could render the interface cleaner and friendlier. It’s pretty text heavy – which is great for advanced users, but less inviting for newer users. More importantly however, we could streamline things to make it easier for people to onboard.

Take for example, the landing page of Bugzilla. It is unclear to me why “Open a new Account” should be on this page. Advanced users will know they want to file a bug, novices (who may be on the wrong site and who should be looking for support) might believe they have to open and account to get support. So why not eliminate the option altogether. You are going to get it anyways if you click on “File a bug.”

Bugzilla-landing-page

Current

Bugzilla-landing-page-v2

Proposed

In addition, I got rid of the bottom menu bar (which I don’t think is necessary on this screenƒclu given all the features were along the top as well). I also ditched the Release Notes and User Guide for Bugzilla as I had doubts about whether users were, at this point and on this screen, looking for those things)

2. Gather more information about our users (and, while I’m at it, some more simplifying)

Once you choose to file a bug you get prompted to either log in or create an account. At this point, if you want to create an account. I thought this page was hard to read with the text spanning the whole width, plus, there is some good info we could gather about users at this point (the point it feels they are mostly likely going to add to their profile).

Current

Bugzilla-registration-v2

Proposed

Couple things a like about this proposed screen.

One, if you are a lost user just looking for support we likely snag you before you fill out a bugzilla account. My feeling is the bugzilla is a scary place that most users shouldn’t end up in… we need to give people lots of opportunities to opt for support before diving in, in case that is what they really need.

Second, in this proposed version we tell people to read the bugzilla guidelines and suggest using an alternate email before they punch their email into the email field box.

In addition, we ask the user for their real name now (as opposed to relying on them to fill it out later). This nudge feels important as the more people with real names on the site, the more I think people will develop relationships with one another. Finally we ask people if English is their second language and if this is their first open source project.

Finally, with the extra data fields we can help flag users as ESL or new and thus in need of more care, patience and help as they on-ramp (see screen shots below). We could even modify the Bugzilla guidelines to inform people to provide newbies and ESL’s with appropriate respect and support.

Bugzilla-Raw1

Current

Bugzilla-New

Proposed

Proposed

I imagine that your “newbie” status would disappear either when you want (some sort of preference in your profile) or after you’ve engaged in a certain amount of activity.

3. Make life easier for users and the triage guys

Here is an idea I had talking with some of the triage guys at the Mozilla Summit.

Let’s suppose that someone submits a bug that isn’t really a bug but a support issue. I’m informed that this happens with a high degree of frequency. Would it be nice if, with a click of a mouse, the triage guys could move that bug out of Bugzilla and into a separate database (ideally this would be straight into SUMO, but I respect that this might not be easy – so just moving it to a separate database and de-cluttering bugzilla would be a great first start – the SUMO guys could then create a way to import it). My sense is that this simply requires creating a new resolution field – I’ve opted to call it “Support” but am happy to name it something else.

Current

Status-v2

Proposed

This feels like a simple fix and it would quickly move a lot of bugs that are cluttering up bugzilla… out. This is important as searches for bugs often return many results that are support oriented, making it harder to find the bugs you are actually searching for. Better still, it would get them somewhere where they could more likely help users (who are probably waiting for us to respond).

Of course, presently bugzilla will auto generate an email that looks like the first one and this isn’t going to help. So what if we did something else?

unresolved

Current

SUMO-transfer-v2

Proposed

Here is the auto-generated email I think we should be sending users whos bugs get sent to SUMO. I’ve proposed a few things.

First, if these are users who’ve submitted inappropriate bugs and who really need support, giving them a bugzilla email isn’t going to help them, they aren’t even going to know how to read it.

Second, there is an opportunity to explain to them where they should go for help – I haven’t done that explicitly enough in this email – but you get the idea

Third, when the bug gets moved to SUMO it might be possible to do a simple key word analysis of the bug and, from that, determine what are the most likely support articles they are looking for. Why don’t we send them the top 3 or 5 as hyperlinks in the email?

Fourth, if this really is a bug from a more sophisticated user, we give them a hyperlink back to bugzilla so they can make a note or comment.

What I like about this is it is customized engagement at a low cost. More importantly, it helps unclutter things while also making us more responsive and creating a better experience for users.

4. Make Bugzilla Celebrate, enhance our brand and build community

Okay, so here’s the thing that really bugs me about bugzilla. If we want to be onramping people and building community, shouldn’t we celebrate people’s successes? At the moment this is the email you get from Bugzilla when a bug you’ve submitted gets patched:

BORING! Here, at the moment of maximum joy, especially for casual or new bugzilla participants we do nothing to engage or celebrate.

This, is what I think the auto-generated bugzilla email should look like.

Congrats-v2

Yes, I agree that hard core community members probably won’t care about these types of bugs, but for more casual participants this is an opportunity to explain how open source and mozilla works (the graphic) as well as a chance to educate them. I’ve even been more explicit about this by offering links to a) explain the open web, b) learn about mozilla and open source; and c) donate to the foundation (given this is a moment of pride for many non-developer end users)

Again, I’m not overly attached to this design per se, it would just be nice to have something fun, celebratory and mozillaesque.

Okay, it is super late and I’m on an early flight tomorrow. Would love feedback on all or any of this for those who’ve made it this far. I’ll be sharing more thoughts, especially on empathetic nudges and community management in bugzilla ASAP.

60 thoughts on “Some thoughts on improving Bugzilla

  1. sabret00the

    I really like those proposals. Obviously some people will love the html emails while others will hate them, but I think it could make a huge difference. Bugzilla can be incredibly daunting, especially for new users. It would be nice to give it that little friendlier touch.

    Reply
  2. cuz84d

    Oh, heck yeah.. do it to it! Filing a new bug is such long and drawn out and cluttered process for new user accounts that cannot get the advanced bug filing.. I'd like to see most of the proposals put in to place. Maybe on the feed back point I like the idea of using HTML markup bugmailfor newbies. . but allow some option in their user account preferences to change it to advanced or simple bugmail.

    Reply
  3. Paul Schreiber

    Two suggestions for the I Need Help/I Want Help screen:- checkboxes should be to the left of the text- the phrase “what is a bug” should be a hyperlink, not just the word “bug”

    Reply
  4. Josh

    I'm loving these ideas. Something that I've been wanting to see is for Bugzilla to use AJAX in some places. For example, when you vote for something.

    Reply
  5. Alexander Limi

    Yes, please! Thank you for starting this conversation and putting up these ideas, getting the right people where they should be is important. We funnel way too many people through Bugzilla that should really visit the support website.

    Reply
  6. cuz84d

    Lots of great ideas here. The simplier and streamlined looks will help.. and remove bugs out of bugzilla would help push them over to the correct support channel or a SUMO bugzilla to get that front line support from SUMO people. We could help this process by starting with keywording bugs with SUMO-needed, or SUMO for later, instead of using QA wanted for them. Another idea that comes to mind is some kind of way to display a common set of Sumo links in bugzilla, .. ie we tend to copy and paste lots of the same sumo article links in to bugs over and over again for users to try..and it would nice if we could streamline that workflow from bugzilla somehow.

    Reply
  7. Pingback: Tweets that mention Some thoughts on improving Bugzilla | eaves.ca -- Topsy.com

  8. Chelsea

    Love all of it, particularly the bit about support. It would help push people into the right engagement channel a lot sooner. Also, what about an e-mail after you've filed your first bug, something akin to the “your bug got fixed” message. Since not all bugs get fixed or some end up being duplicates, it would be great to give people kudos for taking the time to report it. That way they get presented the same engagement opportunities even if they aren't the first to report the bug.

    Reply
  9. Tyler Downer

    I agree. This really is something that we have needed for a long time. Simply making it obvious to users that BMO does not provide support will be a big benefit. Perhaps we should also do something to assist extensions developers. If we could implement a bug tracking database on AMO, then make it easy to kick extension bugs to that database, it will make it easier to track that a bug actually got reported to a developer, as well as provide extension developers their own bug tracker. Now, it doesn't need to be a full bug tracker, something simple would work, but something is better than nothing.

    Reply
  10. Max Kanat-Alexander

    Not showing the Open a New Account button when logged out is a good customization for bugzilla.mozilla.org to make. We wouldn't do it upstream because at corporate installations, there's a lot of “Hey, go open a new account now,” and I want to make that easy for that situation. People tend to miss the “New Account” link.The Users Guide isn't all that helpful, so removing the link to it is probably okay. The release notes are mostly only interesting to admins, and at the worst, logged-in users. So both of those links can be hidden from logged-out users, and we can do that upstream.Removing the footer only from the front page and only for logged-out users might be helpful, but would that actually be worth it, just to remove it from the front page for logged-out users?If you're going to get extra information from the user, you should do it after they click the link in their email, not on the initial “create account” page. Also, I'm generally wary of adding more things to that page, since simply the fact of having to create an account already generates a fairly significant amount of angst.Instead of making (New) a checkbox, it would probably be best to do it heuristically. It could be done in a Bugzilla Extension.If you want to move support issues, there should be either a UI or a button specifically for that, instead of trying to hack it into the resolution, I think. Also remember that there are many products on Bugzilla that don't use SUMO, but resolutions are global. So this would have to be a customization for a limited set of products.Sending the congratulations email is certainly a good idea, but it would require a bit of thought on exactly how you determine when to send it.Anyhow, I think that these would all be reasonable customizations to make to your Bugzilla. Of course, Mozilla would, most likely, have to first employ somebody who actually has time to do that sort of work on Bugzilla.-Max

    Reply
  11. David Eaves

    Thanks Sabret00the – agreed, I think we can make Bugzilla a little more human all while reducing the transactions costs of using it… lots of potential.

    Reply
  12. David Eaves

    Agreed – having people looking for support ending up in Bugzilla is bad for everyone. Users looking for help are probably left confused and unhappy, developers, triage and others have to devote precious volunteer hours to filtering or dealing with these bugs. Plus, it clutters up search results… Mostly I'm trying to think of low cost ways to lower these transactions. Glad these resonated Alex.

    Reply
  13. Tony_Mechelynck

    1) English is not my mother language, but together with Esperanto and my native Fench, it is one of the few languages I speak fluently and feel at ease in. Should I tick “English is not my first language”?2) Please, don't make those fancy HTML emails the only possible way to get info from Bugzilla. BMO is for the whole Mozilla world, not just for newbies who were still using IE the day before, and maybe haven't yet moved their email out of whatever webmail service M$W “provides”. If I can't get no-nonsense plaintext emails the way I've come to expect them, I think I'll close my account, or change the email addy to “nobody@nowhere.net”, and stop not only reporting bugs but also triaging them. And if that makes one fewer volunteer for the SeaMonkey project, well, “it's the fault of those Firefox kiddie toy lovers again”.

    Reply
  14. Ludovic Hirlimann

    I like the email idea too, can we just tweak it so all my bugmails don't start tripling in size and don't start taking ages to load ? Could those cutomized emails be say only sent to the reporter and not everybody on the cc list ?

    Reply
  15. Tony_Mechelynck

    Hm, that's an idea: maybe add a set of radio buttons to the BMO email preferences page?Send prettified HTML mail:(*) Always( ) Only for bugs I reported( ) Never

    Reply
  16. Tony_Mechelynck

    On the front page and for logged-*in* users, I suppose the top line of the footer could be omitted, since it duplicates the header which (on this short page) would still be visible. But is it worth the trouble?Other pages are longer, and there the duplication is useful, as with the status/resolution on bug pages.

    Reply
  17. Tony_Mechelynck

    Maybe a new Component (or several) under the “addons.mozilla.org” BMO Product, to make it easy for triagers to move a bug filed under Firefox/General, but belonging under (let's say) AMO/Extensions, to find its rightful home by just moving the bug, rather than having to explain to the reporter, “Sorry, you got the wrong place, you have to file that bug again with the AMO bug tracker”? Of course, if we want a different Component (or the equivalent) for every single addon, BMO wouldn't be the place; but I think it can be managed if we keep it at BMO and make the triagers (and, if possible, the users) understand that, *when reporting in this Component* the extension author should be liseted as “assignee” of the new bug.

    Reply
  18. Pingback: Some thoughts on improving Bugzilla | eaves.ca | Trends Now

  19. James John Malcolm

    Thing is, 'Additional “I see this too” or “It works for me” comments are unnecessary' should be implemented like Facebook's 'like' button (hear me out).It would mean newbies still can provide their input (which can be important for them) without cluttering up bug resolution. Plus, if, say, everybody clicks the “I see this too” button, it can help make devs realise this is a 'hot' issue.

    Reply
  20. Tony_Mechelynck

    Rather than type a “me too” comment which would spam however many people are CC to the bug, watching its QA, etc., people can already vote, which is reflected in the bug header, makes devs realize the issue is hot, and spams nobody. But then again, “these people don't read the netiquette”. Maybe the Vote link should simply be made more prominent.

    Reply
  21. Guy Pyrzak

    Great Post David, I think a bunch could be great extensions ( I really like the new/ESA one). The ESA/New capability could just be groups you add people to as part of the account creation, we'd have to figure out how to remove them. But then that would appear next to their names.HTML Emails in general are one feature many of us are looking forward to. Once we have that doing something fancy is just a few lines of css away. Getting that done might be a big task though.I think making life easier for triagers is another important one, not sure if just adding a “support” resolution is good enough though. I think we can do better. The auto-dups capability in 4.2 will be great. It coudl be just as much as adding a checkbox that does something fancy. I think something that you point out is the “welcome to bugzilla” for open source instances isn't as great as it could be. We should work to figure out if there is a good place between adding LOTS of text which people don't read and what we currently have.

    Reply
  22. Reece Dunn

    There could also be a “You are getting this email because the 'send email on CC changes' option is set. Go _here_ to change this setting.” bit at the bottom to point people in the right direction — this is not the exact wording or workflow.Perhaps an “unsubscribe to CC changes” link should be provided?

    Reply
  23. Reece Dunn

    I really like these suggestions. These should not just be limited to Mozilla, but be applied to other projects that are using bugzilla to track bugs.

    Reply
  24. Gervase Markham

    1. Simplifications to the front page.In general, I agree. Both simplicity and choice paralysis suggest that we should remove either the header links or the footer links – having both is slowing people down. Many “Bugzilla mod” scripts (e.g. TidyBug) start by doing this. My only quibble is that I take Max's point about the big Create Account (shorter than “Open a New Account”) button needing to be there by default. But we can and should remove it on b.m.o.Here are some more suggestions:The “Forgot Password” link is unnecessary. No-one ever goes to a site and immediately says “I forgot my password”. They try a few common ones first. So the “Forgot Password” link only needs to be on the password error page and the dedicated login form.The titles of the three remaining links should be the shorter:New Bug | Search | Helpand they should be in that order, as that is probably the order of greatest to least use.The words “Main Page” in the header are unnecessary.There are three access points for search on the page; choice paralysis suggests we should have fewer. People tend to look in a header for a search box these days; I would remove the one in the centre of the page. This leaves the “install Quick Search plugin” link orphaned; perhaps there's a home for it on the main Search page or the results page.The difference between “Browse” and “Search” is not obvious, and clicking the Browse link asks you to choose a product with no additional explanatory text, which is not particularly illuminating. The “Log In” fields should be available by default (and so the browser will pre-fill them); that's a _very_ common operation.2. New “create account” pageI love this! We've had various goes at this sort of funnelling – http://feedback.mozilla.org/ is another. I would shorten the text even more:Header: I need help:icon and link: “using Mozilla Firefox”icon and link: “using Mozilla Thunderbird”icon and link: “using other Mozilla software”I like the ESL idea, although I think that new bug filers should be treated well whatever their level of English. And if that's not happening, we need to fix it.As Max says, “New” we can work out programmatically. A month seems like a reasonable length of time.3. Moving bugs to SUMO.The best way to implement this would be a field on SUMO where you paste a bug number, and it reaches out, downloads the Bugzilla information using the Bugzilla API, and creates a new SUMO entry using it. It then goes back and uses the API to automatically resolve the Bugzilla bug – either as SUPPORT, if we have that new resolution, or INVALID, or MOVED (which is a resolution Bugzilla has had in the past for bugs moved elsewhere), or something else.The SUMO end could then send them a custom email, and it could include hyperlinks to appropriate articles if the SUMO engine thought there were any.4. Celebrate and build communityDoing an email _instead_ of the Bugzilla one is a little tricky. Doing one _as_well_ is much easier. If you can provide the HTML for your mockup, I'll write an extension to do it.Both this and the second idea benefit from Bugzilla having some idea of who is new and who is not. One possible proxy for that would be whether the person has the “editbugs” privilege, which people tend to acquire when they start getting significantly involved in Bugzilla. If they don't have it, we can send them the engaging stuff. If they do have it, we can go back to boring ;-)One comment: you use Disqus. One disadvantage of that is that it interacts badly with offline. I loaded the page and went offline. I then accidentally clicked an image, and ended up on a bigger version. I went Back, and got the article back… but no longer had the comments.Gerv

    Reply
  25. Tyler Downer

    Tony, yes, we talked about that, and several triagers, myself included, have been moving extension bugs to the Extension Compatibility component of Firefox. But creating something under AMO would be a step in the right direction. I still think that having a full bug tracking database on AMO for each extension that provides simple bug reporting and tracking would be a great benefit to all extension developers, users, and traigers.

    Reply
  26. Tyler Downer

    I think voting should be made more prominent for those end users. I personally only use voting to keep tracking of interesting or hot bugs (bad me), but I think that it can be so much more.

    Reply
  27. Tyler Downer

    Gerv, I love you point 3. Exactly what I had in mind, have SUMO pull the relevant data from the bug report (we just need BMO to autodetect firefox version numbers, bug 577561 ;) and then it should have most of the required data. That would save the user so much time and remove a major time barrier. They think “I just filed a big, now they want me to start a forum thread?” If it does it automatically, the user would be so much better served.For the ESL, I think that it would be nice to have some community members who are willing and able to help with say french, german, spanish, etc. I have resolved an entire bug in french, using google translate to translate my english into french and the reporter's french into english. But the bug got resolved. I'm not sure what a good way to resolve this would be…

    Reply
  28. Michael Richardson

    I'm not a fan of html-email.I like that most things from bugzilla are not html-email.multipart/alternative doesn't have to have the same information though…. text/plain can continue to contain output for geeks, and the text/html can have the pretty version.(At least, until I turn it off)As for concerns about Bugzilla “uptream” — make the changes to make Mozilla work. It's open source: if “corporate” installs aren't paying for support, then gosh, they could, I dunno, CONTRIBUTE the patches they need back. Bugzilla was created my Mozilla Foundation because it has an itch to scratch.

    Reply
  29. Dan Mosedale

    Perhaps a short, one-sentence tooltip that tries to set the expectation of what a vote is with a link to a more detailed explanation would be helpful here.

    Reply
  30. Dan Mosedale

    It seems like there's a bunch that could be done here, in addition to the things already mentioned (redirecting people looking for support to the support site, using votes for advocacy). A couple of possibilities that come to mind include: * A “report misuse” button next to each comment so that it would be easy for folks who are too busy/uncomfortable/uninterested to gently remind folks who have forgotten to ask for community members' help in doing that reminding.* A wiki page or separate page automatically created for each bug that is specifically designed for general discussion and advocacy that are not adding new technical data or discussion to the bug.

    Reply
  31. Tony_Mechelynck

    Something like that. In mailing lists I've seen people still unsure: writing perfectly understandable English, but with “Please excuse my bad English” on top of their posts. What worries me more is the guy whose English makes so little sense that I start to search (often in vain) for a _different_ common language. Which gives me an idea: maybe an option which would say something like: “My English is not very good; my best languages are: kiKongo, French, Portuguese (Angola).”

    Reply
  32. Tony_Mechelynck

    Hm, yes, you've got to be realistic: a bug with many votes will not necessarily be fixed soon because some bugs are just plain hard to fix, or even (for ill-documented bugs but also for sporadic ones) to reproduce. OTOH (but maybe this need not be explained in detail) a bug with many votes won't remain UNCONFIRMED, so voting still helps somewhat.

    Reply
  33. Tony_Mechelynck

    Report misuse, why not? but someone (who?) will have to decide. A parallel “general discussion” page, hm, the boundary doesn't look clear-cut to me between “comments that help move the bug forward” and “general discussion”. What worries me is of course not the “I'm setting r- because of this, this, this and that”, it's (a) “Shall I write this in the bug or in the general discussion” hesitation paralysis, and (b) the “Hey guys, this bug was filed in 1994 and still no progress?!?” whose author will “of course” ;-) never think of writing it as “general discussion” or (even better) leaving it off altogether.

    Reply
  34. Reece Dunn

    The “Add me to the CC list for a bug” workflow is counter-intuitive: you go to the bug and then press the “Save Changes” button!As someone not used to bugzilla, I would expect something like an “Add me to the CC list” button below the CC list control.I'm not sure how this will work with the “modify some properties and CC me” workflow, or with users who are used to the “save changes” model.

    Reply
  35. Tony_Mechelynck

    It doesn't sound counter-intuitive once you realize that making *any* change to a bug requires clicking the “Save Change” button (either the one near the top or the one at the bottom, they have the same function). If the change you wish to make consists of adding yourself to the CC list, then *first* you make sure that the checkbox, “Add me to CC list”, located next to the CC box, is ticked, and only *thereafter* do you click the “Save Changes” button. No sweat.

    Reply
  36. Michael Lefevre

    The celebration email is a nice idea, but I think it might be difficult to make it work in practice and still have some meaning. Bugs sometimes get morphed to fix things that weren't the original report, they get marked as duplicates of bugs filed later, they get checked in to different branches at different times, they can get reopened after being fixed…Something I've seen quite a few times is that bug is marked fixed, and then the reporter says “cool, how do I get the fixed version?” and the answer is that they either have to switch to nightly builds or wait another few months for a stable version.There are also a bunch of other simple things that would make things easier, but there doesn't seem to be anyone with the time and ability to be customising Mozilla's bugzilla (and there isn't a huge amount of resources for Bugzilla in general, and stuff happening there has to be appropriate for everyone using Bugzilla, not just Mozilla).

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s