Thursday, October 18, 2007

Call for Participation in Openbravo 2.35 Acceptance Testing

One of the most frequent requests that we get from our community is an opportunity to participate in the Openbravo quality assurance process.

Well... here it is.

Starting with the upcoming 2.35 Acceptance Testing, we would like to take a small step in that direction and ask for a handful of volunteers to test some of the mechanics. If you are willing and able, you can notify us by sending an email to collaborate@openbravo.com.

Here is what we are asking:
  1. When Openbravo 2.35 code is frozen, we will give you early access to the installer through a private FTP server. You will essentially receive the release at the same time as our QA team.
  2. We will give you access to our QA portal, where you can see our test cases for the acceptance test.
  3. We will assign you a set of test cases and we expect that you will install the application on your server, run the test cases and report the outcome in the QA portal
  4. We will also assign you a set of bugs to verify (i.e. validate that bugs that are supposed to be fixed in 2.35 are indeed fixed)
  5. We will also welcome any other testing that you might be inclined to perform, including verifying bug fixes that are critical to you or your customers.
  6. If you have any questions or doubts during the process, we will be available on the IRC channel to answer.
The whole thing should start early next week (October 22nd) and take 2 or 3 days of your time, if everything goes well (does it ever?).

Please remember that this is going to be Acceptance Testing, not a full blown QA cycle. The purpose of Acceptance Testing is to validate that an already QA'ed release is good to go and that the last build didn't introduce any major regression (essentially: test the product as it is going to be shipped before your users do to avoid to be embarrassed later).
Because of that, we will stop the release only if one of the test cases fails with a significant bug or one of the critical bugs that we thought had been fixed still reproduces. Nonetheless, we do expect you to log all issues you find, including small bugs, as we will fix those in future releases.

What's in it for you? Well... practically not much, but you can be very proud to have helped the community and to have contributed to speed up the release of the best version ever of Openbravo. You will also be able to validate that bug fixes critical to you and your customers are indeed fixed.
Finally, you will also get a mention in our release notes, so there is some glory at stake.

So... why such a small step now? And why accepting only a handful of volunteers rather than the whole community?
Well... this is a factor of where we are in the release. Openbravo 2.3x went already through several rounds of full blown QA testing and 4 public releases (2.30, 2.31, 2.33 and 2.34). So Acceptance Testing is what we need to do next and, since the scope is quite limited, it would not make sense to involve too many people. Also, this is the first time we do this, and we would like to make sure that we can coordinate with a few people before we try with many in order to avoid wasting people's time.

For the next major release, Openbravo 2.4x, we have more ambitious plans which will include a public participation of the whole community, public status reporting, etc. We will be working on the definition of that process over the next few weeks. One of the key outstanding issues is being able to publish frequent releases to the community in order to get frequent feedback on fresh code: at the moment the build process is time consuming and manual intensive but we are working to address that.

If you have good ideas, feel free to post them as comments.

Tuesday, October 16, 2007

Openbravo Releases

Today it is exactly two months since I joined Openbravo and I thought that the best way to celebrate is to write my first blog entry.
What will I write about? Well.. Openbravo obviously, but also other topics that might be of interest to the Openbravo community.
And what better way to start than to cover a topic that has been very hot with our community in the recent weeks... release management and release maturity.

When Openbravo 2.30 was released, the project published it as an alpha release but that status was declared only in the news announcement. When we published 2.31, we published it as beta using the same announcement mechanism.
The success of these releases was so overwhelming that, when we eventually announced 2.33, we were not clear enough on whether it was suitable for production purposes or not.

As it turned out, that was a major mistake and that lack of clarity created a lot of confusion among our community. We tried to correct it with 2.34 which was published as beta, but at that point it was too late. The damage had been done.

Through this experience, we learned our lesson and we have now published an updated release policy. All the details are in that document but, in a nutshell, we intend to distinguish between Community Edition, which is intended to be stable and well tested but published twice a year, and Developer's Edition, which is intended to be published very frequently and includes all the latest development, including features in progress.

The Community Edition will go through a stabilization process, moving from alpha version, to one or more beta versions, and eventually to production. At all stages, the status of the release will be very clearly marked both in its name and in the release notes and - by default - users will not be directed to download the alpha or beta releases but the previous stable production release.

It is too late to apply this policy for 2.3x but we will start following it with 2.4x and use it for all future releases. We hope that this policy will greatly help our community by setting the right expectations on production usage and avoid some of the confusions and frustrations that users experienced in the past.

At the same time, the Developer's Edition will allow the community to be involved in our development process and to give us early feedback on the features we are working on. This will give us not only an opportunity to continuously fine tune our product based on your input, but also to reduce the stabilization cycle of the Community Edition.