Tag Archives: bugzilla

Making Bug Fixing more Efficient (and pleasant) – This Made Me Smile

1 Reply

The other week I was invited down to the Bay Area Drupal Camp (#BadCamp) to give a talk on community management to a side meeting of the 100 or so core Drupal developers.

I gave a hour long version of my OSCON keynote on the Science of Community Management and had a great time engaging what was clearly a room of smart, caring people who want to do good things, ship great code, and work well with one anther. As part of my talk I ran them through some basic negotiation skills – particularly around separating positions (a demand) from interests (the reasons/concerns that created that demand). Positions are challenging to work with as they tend to lock people into what they are asking and makes outcomes either binary or fosters compromises that may make little sense, where as interests (which you get by being curious and asking lots of whys) can create the conditions for make creative, value generative outcomes that also strengthen the relationship.

Obviously, understanding the difference is key, but so is acting on it, e.g. asking questions are critical moments to try to open up the dialogue and uncover interests.

Seems like someone was listening during the workshop since I just sent this link to a conversation about a tricky drupal bug (Screen shot below)

Drupal-bug-fixing2

I love the questions. This is exactly the type of skill and community norms I think we need to build tino more of bug tracking environments/communities, which can sometimes be pretty hostile and aggressive, something that I think turns off many potentially good contributors.

Design Matters: Looking at a Re-themed Bugzilla

9 Replies

I’ll be honest. There was a time when I thought design didn’t matter. To my credit, it was a long time ago… but I used to think, if the tool was good enough, the design won’t matter, people will use it cause it is helpful. (This may or may not have influenced some fashion choices earlier in life as well – I’d like to think things have improved – but not everyone may agree it has improved sufficiently).

Being useful may be sufficient (although take a look at the government website at the bottom of this post – it’s a very useful website). But it’s no excuse for not making things easier to use. Especially when you are running an open source community and want to encourage participation and ease people up the learning curve faster.

Hence why I enjoyed recently discovering ActiveState’s implementation of Bugzilla (re-themed by Tara Gibbs). Bugzilla is the software many open source projects use to identify, track and resolve bugs.

Here we have two identical pieces of software (so the “usefulness” is the same) but what makes ActiveState’s version of Bugzilla so nice are a few simple things they’ve done to make it more user friendly (doubly pleased to see them implement some ideas I’d blogged earlier as well – great minds think alike!).

So let’s start with the Mozilla instance of Bugzilla – as this was the one I was used to.

I’ve circled a couple of the key features to zero in on. Let me go through them as I want you to be thinking about them when you look at the ActiveState version:

  • Red circle: Notice that this has a lot of key items in it, but it is lost next to the “search” button, which pulls your attention away
  • Dark orange arrow: the search button! most often you won’t find a search box located here in an application.
  • Green circle: Tons of useful stuff down here, but arranged in one long horizontal list, that makes it hard to find what you’re look for (and another search box!)
  • Light orange arrow: another log out option… didn’t I see that somewhere else as well?

I want to be clear, the Bugzilla team at Mozilla is awesome. Recently hired they are trying to do a ton of stuff and this is not where I’d expect them to start (and they’ve been super responsive to everything I’ve blogged about so I’m a huge fan), I want to flag this because everyone, from software engineers to government officials need to recognize that we rely on good design to make our lives easier, to help with decision fatigue and streamline our work, every day.

Now check out the ActiveState version of the exact same software, but re-designed.

So, my cartoonish circles and arrows are mucking up the design and ascetics of both theses sites, so please forgive that. (I suggest opening them in adjacent tabs – Mozilla here, ActiveState here – so you can see them uninterrupted).

So, a few things:

  • Red circle: Now everything to do with the administration of your account is in the top, top right hand corner. This is where Google, Facebook and most websites put this info now, that’s why you’re expecting to find it there!
  • Dark orange arrow: now the search button is in the top right hand corner. Pretty much the same location it appears in Firefox (and safari, IE, chrome, OS X, etc…) and so where users have come to expect it.
  • Green circle: This part really is genius. Did you know there were saved searches in the above version? There are, but the feature didn’t stand out. This theme sorts the users options and displays them vertically within a menu: much, much easier to digest quickly.
  • Light orange arrow: Features appear only once! For example, the sign out and search feature do not appear at the top and bottom. This helps reduce clutter and allows the user to find things more quickly

My point is that a few minor changes can dramatically improve the usability of a website or tool. Is Mozilla’s bugzilla radically worse than ActiveState’s? No, but I definitely prefer ActiveState’s design. Moreover, when you are relying on volunteer contributors and attracting new contributors is something that matters to you, this is an important gateway and so you want it to be as seamless as possible.

What’s interesting is that often it is in the non-profit and government sector that design gets neglected because it is deemed a luxury, or the “substantive” people don’t think design matters and so ignore it.

The results can be disastrous.

I mean, especially if you are in government, then you’ve really got to be advocating for better design. Consider the website below. Remember, this may be the most important citizen facing website in the Canadian government – the one stop shop to find every service you need. It is better than most government website, and yet, you’ve got a site that is still maddeningly difficult to navigate. Where am I supposed to look??? Eyes… being… pulled… in… so… many… directions…

Personally, I think you could solve 80% of the problem with this page just by getting rid of the left hand column and put a search button in the top right hand corner. But I’m supremely confident that would violate some arcane website design rule the government has and so will remain a post for another day…

Lessons for Open Source Communities: Making Bug Tracking More Efficient

12 Replies

This post is a discussion about making bug tracking in Bugzilla for the Mozilla project more efficient. However, I believe it is applicable to any open source project or even companies or governments running service desks (think 311).

Almost exactly a year ago I wrote a blog post titled: Some thoughts on improving Bugzilla in which I made several suggestions for improving the work flow in bugzilla. Happily a number of those ideas have been implemented.

One however, remains outstanding and, I believe, creates an unnecessary amount of triage work as well as a terrible experience for end users. My understanding is that while the bug could not be resolved last year for a few reasons, there is growing interest (exemplified originally in the comment field of my original post) to tackle it once again. This is my attempt at a rallying cry to get that process moving.

Those who are already keen on this idea and don’t want to read anything more below, this refers to bug 444302.

The Challenge: Dealing with Support Requests that Arrive in Bugzilla

I first had this idea last summer while talking to the triage team at the Mozilla Summit. These are the guys who look at the firehose of bugs being submitted to Mozilla every day. They have a finite amount of time, so anything we can do to automate their work is going to help them, and the project, out significantly.

Presently, I’m told that Mozilla gets a huge number of bugs submitted that are not actually bugs, but support issues. This creates several challenges.

First, it means that support related issues, as opposed to real problems with the software, are clogging up the bug tracking system. This increases the amount of noise in the system – making it harder for everyone to find the information they need.

Second, it means the triage teams has to spend time filtering bugs that are actually support issues. Not a good use of their time.

Third, it means that users who have real support issues but submit them accidentally though Bugzilla, get a terrible experience.

This last one is a real problem. If you are a user, feeling frustrated (and possibly not behaving as your usual rational self – we’ve all been there) because your software is not working the way you expect, and then you submit what a triage person considers a support issue (Resolve-Invalid)  you get an email that looks like this:


If I’m already cheesed that my software isn’t doing what I want, getting an email that says “Invalid” and “Verified” is really going to cheese me off. That of course presumes I even know what this email means. More likely, I’ll be thinking that some ancient machine in the bowels of mozilla using software created in the late 1990s received my plea and has, in its 640K confusion, has spammed me. (I mean look at it… from a user’s perspective!)

The Proposal: Re-Automating the Process for a better result

Step 1: My sense is that this issue – especially problem #3 – could be resolved by simply creating a new resolution field. I’ve opted to call it “Support” but am happy to name it something else.

This feels like a simple fix and it would quickly move a lot of bugs that are cluttering up bugzilla… out.

Step 2: Query the text of bugs marked “support” against Mozilla’s database. Then insert the results in an email that goes back to the user. I’m imagining something that might look like this:

SUMO-transfer-v2

Such an email has several advantages:

First, if these are users who’ve submitted inappropriate bugs and who really need support, giving them a bugzilla email isn’t going to help them, they aren’t even going to know how to read it.

Second, there is an opportunity to explain to them where they should go for help – I haven’t done that explicitly enough in this email – but you get the idea.

Because, because we’ve done a query of the Mozilla support database (SUMO) we are able to include some support articles that might resolve their issue.

Fourth, if this really is a bug from a more sophisticated user, we give them a hyperlink back to bugzilla so they can make a note or comment.

What I like about this is it is customized engagement at a low cost. More importantly, it helps unclutter things while also making us more responsive and creating a better experience for users.

Next Steps:

It’s my understanding that this is all pretty doable. After last year’s post there were several helpful comments. Including this one from Bugzilla expert Gervase Markham:

The best way to implement this would be a field on SUMO where you paste a bug number, and it reaches out, downloads the Bugzilla information using the Bugzilla API, and creates a new SUMO entry using it. It then goes back and uses the API to automatically resolve the Bugzilla bug – either as SUPPORT, if we have that new resolution, or INVALID, or MOVED (which is a resolution Bugzilla has had in the past for bugs moved elsewhere), or something else.

The SUMO end could then send them a custom email, and it could include hyperlinks to appropriate articles if the SUMO engine thought there were any.

And Tyler Downer noted in this comment that there maybe be a dependency bug (#577561) that would also need resolving:

Gerv, I love you point 3. Exactly what I had in mind, have SUMO pull the relevant data from the bug report (we just need BMO to autodetect firefox version numbers, bug 577561 ;) and then it should have most of the required data. That would save the user so much time and remove a major time barrier. They think “I just filed a bug, now they want me to start a forum thread?” If it does it automatically, the user would be so much better served.

So, if there is interest in doing this, let me know. I’m happy to support any discussion, should it take place on the comment stream of the bug, the comments below, or somewhere else that might be helpful (maybe I should dial in on this call?). Regardless, this feels like a quick win, one that would better serve Mozilla users, teach them to go to the right place for support (over time) and improve the Bugzilla workflow. It might be worth implementing even for a bit, and we can assess any positive or negative feedback after 6 months.

Let me know how I can help.

Additional Resources

Bug 444302: Provide a means to migrate support issues that are misfiled as bugs over to the support.mozilla.com forums.

My previous post: Some thoughts on improving Bugzilla. The comments are worth checking out

Mozilla’s Bugzilla Wiki Page

Rethinking Freedom of Information Requests: from Bugzilla to AccessZilla

12 Replies

Last week I gave a talk at the Conference for Parliamentarians hosted by the Information Commission as part of Right to Know Week.

During the panel I noted that, if we are interested in improving response times for Freedom of Information (FOI) requests (or, in Canada, Access to Information (ATIP) requests) why doesn’t the Office of the Information Commissioner use a bugzilla type software to track requests?

Such a system would have a number of serious advantages, including:

  1. Requests would be public (although the identity of the requester could remain anonymous), this means if numerous people request the same document they could bandwagon onto a single request
  2. Requests would be searchable – this would make it easier to find documents already released and requests already completed
  3. You could track performance in real time – you could see how quickly different ministries, individuals, groups, etc… respond to FOI/ATIP requests, you could even sort performance by keywords, requester or time of the year
  4. You could see who specifically is holding up a request

In short such a system would bring a lot of transparency to the process itself and, I suspect, would provide a powerful incentive for ministries and individuals to improve their performance in responding to requests.

For those unfamiliar with Bugzilla it is an open source software application used by a number of projects to track “bugs” and feature requests in the software. So, for example, if you notice the software has a bug, you register it in Bugzilla, and then, if you are lucky and/or if the bug is really important, so intrepid developer will come along and develop a patch for it. Posted below, for example, is a bug I submitted for Thunderbird, an email client developed by Mozilla. It’s not as intuitive as it could be but you can get the general sense of things: when I submitted the bug (2010-01-09), who developed the patch (David Bienvenu), it’s current status (Fixed), etc…

ATIPPER

Interestingly, an FOI or ATIP request really isn’t that different than a “bug” in a software program. In many ways, bugzilla is just a complex and collaborative “to do” list manager. I could imagine it wouldn’t be that hard to reskin it so that it could be used to manage and monitor access to information requests. Indeed, I suspect there might even be a community of volunteers who would be willing to work with the Office of the Information Commissioner to help make it happen.

Below I’ve done a mock up of what I think revamped Bugzilla, (renamed AccessZilla) might look like. I’m put numbers next to some of the features so that I can explain in detail about them below.

ATIPPER-OIC1

So what are some of the features I’ve included?

1. Status: Now an ATIP request can be marked with a status, these might be as simple as submitted, in process, under review, fixed and verified fixed (meaning the submitter has confirmed they’ve received it). This alone would allow the Information Commissioner, the submitter, and the public to track how long an individual request (or an aggregate of requests) stay in each part of the process.

2.Keywords: Wouldn’t it be nice to search of other FOR/ATIP requests with similar keywords? Perhaps someone has submitted a request for a document that is similar to your own, but not something you knew existed or had thought of… Keywords could be a powerful way to find government documents.

3. Individual accountability: Now you can see who is monitoring the request on behalf of the Office of the information commissioner and who is the ATIP officer within the Ministry. If the rules permitted then potential the public servants involved in the document might have their names attached here as well (or maybe this option will only be available to those who log on as ATIP officers.

4. Logs: You would be able to see the last time the request was modified. This might include getting the documents ready, expressing concern about privacy or confidentiality, or simply asking for clarification about the request.

5. Related requests: Like keywords, but more sophisticated. Why not have the software look at the words and people involved in the request and suggest other, completed requests, that it thinks might similar in type and therefor of interest to the user. Seems obvious.

6. Simple and reusable resolution: Once the ATIP officer has the documentation, they can simply upload it as an attachment to the request. This way not only can the original user quickly download the document, but anyone subsequent user who stumbles upon the request during a search could download the documents. Better still any public servant who has unclassified documents that might relate to the request can simply upload them directly as well.

7. Search: This feels pretty obvious… it would certainly make citizens life much easier and be the basic ante for any government that claims to be interested in transparency and accountability.

8. Visualizing it (not shown): The nice thing about all of these features is that the data coming out of them could be visualized. We could generate realt time charts showing average response time by ministry, list of respondees by speed from slowest to fastest, even something as mundane as most searched keywords. The point being that with visualizations is that a governments performance around transparency and accountability becomes more accessible to the general public.

It may be that there is much better software out there for doing this (like JIRA), I’m definitely open do suggestions. What I like about bugzilla is that it can be hosted, it’s free and its open source. Mostly however, software like this creates an opportunity for the Office of the Information Commissioner in Canada, and access to information managers around the world, to alter the incentives for governments to complete FOI/ATIP requests as well as make it easier for citizens to find out information about their government. It could be a fascinating project to reskin bugzilla (or some other software platform) to do this. Maybe even a Information Commissioners from around the world could pool their funds to sponsor such a reskinning of bugzilla…

Bugzilla – progress made and new thoughts

A few weeks ago I published a post entitled Some Thoughts on Improving Bugzilla. The post got a fair bit a traction and received a large number of supportive comments. But what was best, about the post, about open source, about Mozilla, is that it drew me into a serious of conversations with people who wanted to make some of it reality.

Specifically, I’d like to thank Guy Pyrzak over at Bugzilla and Clint Talbert at Mozilla both of whom spent hours entertaining and conversing about these ideas with me, problem solving them and, if we are really honest, basically doing all the heavy lifting to transform them from ideas on this blog into real changes.

So in this post I have two things to share. First is an update on progress from the ideas in the last post (which will be this post) as well as some new thoughts about how Mozilla instance of Bugzilla could be further improved (which will be my next post).

So here we go…

Update!

1. Simplifying Menus

First up I made some suggestions around simplifying the bugzilla landing page. These were pretty cosmetic, but they make the landing page a little less intimidating to a new user and, frankly, nicer for everyone. We are presently drafting up the small changes to the code that would require this change and getting ready to submit it as a proposal. Status – Yellow.

2. Gather more information about our users (and, while I’m at it, some more simplifying)

Second, I outlined some ideas for streamlining the process of joining bugzilla and on the data we collect about users.

On the first part, which is about the steamlined pages (designed to help ensure that true bug submitters end up in bugzilla and not those seeking support) here too we will be submitting some new proposed pages shortly. Status – Yellow

On the second part I suggested that we ask users if they English is their second language and that we mark new bugzilla accounts with a “new” symbol. Guy is coding up an extension to Bugzilla that will both of these. Once done, I’ll suggest to Mozilla that they include this extension in their instance. Status – Green.

3. Make Life Easier for Users and the Triage Guys

I thought we could make life more efficient for triage and users if we added a status where bugs could be declared “RESOLVED-SUPPORT.” There’s been some reception to this idea. However, the second part of this idea is that once a bug is tagged as such a script automatically should scan the support database, find articles with a strong word correlation to the bug description and email the bug submitter links to those pages. Once again, Guy has stepped forward to develop such an extension which hopefully will be working in the not to distant future. Status – Green.

4. Make Bugzilla Celebrate, enhance our brand and build community

But probably the most exciting part is the final suggestion. That we send (at least non-developers) much nicer emails celebrating that the bug they submitted has been patched. It turns out (hardly surprising) that I wasn’t the first person to think that Bugzilla should be able to send HTML emails. Indeed, that feature request was first made back in 2001 and, when I blogged about this the other week, had not be updated since 2006. Once again, Guy has proven to be unbelievably helpful. It turns out that due to some changes to bugzilla many of the blocks to patching this had disappeared and so he has been working on the code. Status – Green.

Lots here for many people to be proud of. Hopefully some of these ideas will go live in the not too distant future. That said, still many hurdles to clear and if you are a decision maker on any of these and would like to talk about these ideas, please do not hesitate to contact me.

Some thoughts on improving Bugzilla

60 Replies

One of the keys to making an open source project work is getting feedback from users and developers about problems (bugs) in the code or system. Mozilla (the organization behind Firefox and Thunderbird) uses Bugzilla, but organizations have developed a variety of systems for dealing with this issue. For example, many cities use 311. I’m going to talk about Bugzilla and Mozilla in this case, but I think the lessons can be applied more broadly for some of my policy geek friends.

So first, some first principles. Why does getting the system right matter? A few reasons come to mind:

  1. Engagement: For many people Bugzilla is their first contact with “the community.” We should want users to have a good experience so they feel some affinity towards us and we should want developers to have a great experience so that they want to deepen their level of participation and engagement.
  2. Efficiency: If you have the wrong or incomplete information it is hard (or impossible) to solve a problem, wasting the precious time of volunteer contributors.

I also concede that these two objectives may not always be congruent. Indeed, at times there may be trade offs between them… but I think there is a lot that can be done to improve both.

I’ve probably got more ideas than can fit (or should fit) into one post so I’m going to unload a few. I’ve got more that relate to the negotiation and empathetic approaches I talked about at the Mozilla Summit.

One additional thought. Please feel free to dump all over these. Some changes many not be as simple as I’ve assumed. Others may break or contravene important features I’m not aware of. Happy to engage people on these, please do not see them as an end point, but rather a beginning. My main goal with this first batch of suggestions was to find things that felt easier to do and so could be implemented quickly if there was interest and would help reduce transactions costs right away.

1. Simplifying Menus

First. I thought there were some simple changes that could render the interface cleaner and friendlier. It’s pretty text heavy – which is great for advanced users, but less inviting for newer users. More importantly however, we could streamline things to make it easier for people to onboard.

Take for example, the landing page of Bugzilla. It is unclear to me why “Open a new Account” should be on this page. Advanced users will know they want to file a bug, novices (who may be on the wrong site and who should be looking for support) might believe they have to open and account to get support. So why not eliminate the option altogether. You are going to get it anyways if you click on “File a bug.”

Bugzilla-landing-page

Current

Bugzilla-landing-page-v2

Proposed

In addition, I got rid of the bottom menu bar (which I don’t think is necessary on this screenƒclu given all the features were along the top as well). I also ditched the Release Notes and User Guide for Bugzilla as I had doubts about whether users were, at this point and on this screen, looking for those things)

2. Gather more information about our users (and, while I’m at it, some more simplifying)

Once you choose to file a bug you get prompted to either log in or create an account. At this point, if you want to create an account. I thought this page was hard to read with the text spanning the whole width, plus, there is some good info we could gather about users at this point (the point it feels they are mostly likely going to add to their profile).

Current

Bugzilla-registration-v2

Proposed

Couple things a like about this proposed screen.

One, if you are a lost user just looking for support we likely snag you before you fill out a bugzilla account. My feeling is the bugzilla is a scary place that most users shouldn’t end up in… we need to give people lots of opportunities to opt for support before diving in, in case that is what they really need.

Second, in this proposed version we tell people to read the bugzilla guidelines and suggest using an alternate email before they punch their email into the email field box.

In addition, we ask the user for their real name now (as opposed to relying on them to fill it out later). This nudge feels important as the more people with real names on the site, the more I think people will develop relationships with one another. Finally we ask people if English is their second language and if this is their first open source project.

Finally, with the extra data fields we can help flag users as ESL or new and thus in need of more care, patience and help as they on-ramp (see screen shots below). We could even modify the Bugzilla guidelines to inform people to provide newbies and ESL’s with appropriate respect and support.

Bugzilla-Raw1

Current

Bugzilla-New

Proposed

Proposed

I imagine that your “newbie” status would disappear either when you want (some sort of preference in your profile) or after you’ve engaged in a certain amount of activity.

3. Make life easier for users and the triage guys

Here is an idea I had talking with some of the triage guys at the Mozilla Summit.

Let’s suppose that someone submits a bug that isn’t really a bug but a support issue. I’m informed that this happens with a high degree of frequency. Would it be nice if, with a click of a mouse, the triage guys could move that bug out of Bugzilla and into a separate database (ideally this would be straight into SUMO, but I respect that this might not be easy – so just moving it to a separate database and de-cluttering bugzilla would be a great first start – the SUMO guys could then create a way to import it). My sense is that this simply requires creating a new resolution field – I’ve opted to call it “Support” but am happy to name it something else.

Current

Status-v2

Proposed

This feels like a simple fix and it would quickly move a lot of bugs that are cluttering up bugzilla… out. This is important as searches for bugs often return many results that are support oriented, making it harder to find the bugs you are actually searching for. Better still, it would get them somewhere where they could more likely help users (who are probably waiting for us to respond).

Of course, presently bugzilla will auto generate an email that looks like the first one and this isn’t going to help. So what if we did something else?

unresolved

Current

SUMO-transfer-v2

Proposed

Here is the auto-generated email I think we should be sending users whos bugs get sent to SUMO. I’ve proposed a few things.

First, if these are users who’ve submitted inappropriate bugs and who really need support, giving them a bugzilla email isn’t going to help them, they aren’t even going to know how to read it.

Second, there is an opportunity to explain to them where they should go for help – I haven’t done that explicitly enough in this email – but you get the idea

Third, when the bug gets moved to SUMO it might be possible to do a simple key word analysis of the bug and, from that, determine what are the most likely support articles they are looking for. Why don’t we send them the top 3 or 5 as hyperlinks in the email?

Fourth, if this really is a bug from a more sophisticated user, we give them a hyperlink back to bugzilla so they can make a note or comment.

What I like about this is it is customized engagement at a low cost. More importantly, it helps unclutter things while also making us more responsive and creating a better experience for users.

4. Make Bugzilla Celebrate, enhance our brand and build community

Okay, so here’s the thing that really bugs me about bugzilla. If we want to be onramping people and building community, shouldn’t we celebrate people’s successes? At the moment this is the email you get from Bugzilla when a bug you’ve submitted gets patched:

BORING! Here, at the moment of maximum joy, especially for casual or new bugzilla participants we do nothing to engage or celebrate.

This, is what I think the auto-generated bugzilla email should look like.

Congrats-v2

Yes, I agree that hard core community members probably won’t care about these types of bugs, but for more casual participants this is an opportunity to explain how open source and mozilla works (the graphic) as well as a chance to educate them. I’ve even been more explicit about this by offering links to a) explain the open web, b) learn about mozilla and open source; and c) donate to the foundation (given this is a moment of pride for many non-developer end users)

Again, I’m not overly attached to this design per se, it would just be nice to have something fun, celebratory and mozillaesque.

Okay, it is super late and I’m on an early flight tomorrow. Would love feedback on all or any of this for those who’ve made it this far. I’ll be sharing more thoughts, especially on empathetic nudges and community management in bugzilla ASAP.

Many eyes make all bugs shallow – especially when the eyes get smarter

[Please bear with me for the next 24 hours – I’m moving to a new hosting company, and there may be some glitches here and there to work out. Thanks.]

My friend Diederik has been writing a number of cool posts over at his blog Network-labs.org. For example he has an awesome jetpack plugin that predicts the likelihood a bug will get fixed and a plugin that allows users to rate how badly a bug is being flamed (to spot counterproductive behaviours).

But he recently published a very cool post that uses data from Mozilla bug submissions over the past decade to demonstrate that bug submitters become more proficient over time. However, there are outliers who are a “drag” on the system. More importantly, I believe his data can be interpreted in another way. That, with some minor investment (particularly some simpler vetting screens prior to reaching bugzilla) bug submitters could learn faster.

For example, a landing screen that asks you if you’ve ever submitted a bug before might take newbies to a different page where the bugzilla process is explained in greater detail, the fact that this is not a support site is outlined, and some models of good “submissions” are shared (along with some words of encouragement). By segmenting newbies we might ease the work burden on those who have to vet the bugs.

I think this also has implications for cities and 311 systems (systems that essentially allow citizens to report problems – or bugs – with city infrastructure or services). There will inevitably be people who become heavy users of 311 – the goal is not to get frustrated with them, but to move them up a learning curve of submitting helpful and useful information as quickly as possible. Ignoring them will not work. Strategies for engagement, including understanding what is motivating them and why they continue to use the system in an ineffective way, could help save the city resources and help foster a larger army of citizens who use their spare social capital to make the city a better and more resilient space.

How to predict the "Fixability" of a Bugzilla Submission

bugzilla iconMy friend Diederik van Liere has written a very, very cool jet-pack add-on that calculates the probability a bug report will result in a fixed bug.

The skinny on it is that Diederik’s app bases its prediction on the bug reporter’s experience, their past success rate, the presence of a stack trace and whether the bug reporter is a Mozilla affiliate. These variables appear to be strong and positive predictors of whether a bug will be fixed. The add-on can be downloaded here and its underlying methodology is explained in this blog post.

One way the add-on could be helpful is that it would enable the mozilla community to focus its resources on the most promising bug reports. Volunteer coders with limited time who want to show up and and take ownership over a specific bug would probably find this add-on handy as it would help them spend their precious volunteer time on bugs that are likely well thought through, documented effectively and submitted by someone who will be accessible and able to provide them with input if necessary.

The danger of course, is that a tool like this might further enhance (what I imagine is) a power-law like distribution of bug submitters. The add-on would allow those who are already the most effective bug submitters to get still more attention while first time submitters or those who are still learning may not receive as much sufficient attention (coaching, feedback, support) to improve. Indeed, one powerful way the tool might be used (and which I’m about to talk to Diederik about) is to determine if there are classes of bug submitters who are least likely to be successful. If we can find some common traits among them it might be possible to identify ways to better support them and/or enable them to contribute to the community more effectively. Suddenly a group of people who have expressed interest but have been inadvertently marginalized (not on purpose) could be brought more effectively into the community. Such a group might be the lowest hanging fruit in finding the next million mozillians.