Bug Fixes vs Feature Patches

Revisiting the 1.6 Road Map

• Implement a new JForm library package [complete].
• Implement a simple way of providing translation in JavaScript [complete].
• Implement new controller dispatchers for more robust request routing [complete].
• Implement a new access control system that needs to at least emulate what is in 1.5, allow adding of new groups and access levels, and allow you to set new "view" rules for at least articles [nearly complete].
• Implement and standardise several new event triggers [in progress].
• Implement a JContent class that will be used by content plug-ins and views [in progress].
• Upgrade to Mootools 1.2 [in progress].
• Finish the new extension updater work [in progress].
• Menu manager re-work -- added since it's broken in 1.6 [in progress].

After we release the alpha, each beta release will be time-boxed (we hope that not more than four are required). The following is a list of the features highly desired for the final distribution. Each of these features will need to be complete in order to be included in a beta release.  Some of these features will make it in the alpha, but none will prevent the alpha from being released if they are not ready.  Overall we will need significant help from the Joomla development community to complete any of these features:

• Implement unlimited depth categories (but not multi-mapping).
• Refactor the user management system and make it more extensible (eg, allow custom user fields).
• Implement a comments system (including pings and track-backs).
• Implement queued redirects (allows you to, for example, return to the previous page you were on after you edit something).
• Refactor parameters and make them more extensible (for example, plugins could allow you to add additional custom parameters to articles).
• Finish MVC-ing the Administrator components (we need lots of help here).
• Implement CAPTCHA helpers for any form.
• Implement systems whereby external authentication systems, such as LDAP, can map to our new Joomla user groups.
• Re-implement the ability to select multiple categories for some views in com_content (was in 1.0, got dropped in 1.5).
• Implement a database driven installation log.
• Refactor JError.
• Examine the PDF generation system in detail and see if we can make it work properly (otherwise we will look at dropping it if we can't make it work well).
• Localise the Invalid Token messages.
• Drop the Polls component because the quality of that extension is pretty bad and there are much better third-part alternatives available.
• Convert all layouts to semantic, XHTML Strict.
  
• Convert of ini-based "params" fields to use JSON instead of INI format (huge technical and performance improvements).  Note, the language files will remain in INI format.

It is important to understand that we will focus on the features listed above. There are no doubt lots of ideas for what could be implemented, but it is very important to understand that this is the list of features we will focus on.  The features that make it into the final Joomla 1.6 release will depend upon what the Joomla Development Community contributes.  Since some these features are already being worked on, we strongly advise everyone to post a message to the Joomla general development mailing list before you start working on something. If you're not already a member, please apply.

The policy document handles all details, just read the policy document on the development site for those interested in committing back to the project...let's stick to the code and make good things happen!

2008 was certainly a big year with the release of 1.5 in January.  I think this has been one of our most successful and ground breaking releases (comparable to Mambo 4.5.1 which really pushed us to a new level back in the good old days).  A new stability release will come out this month marking 1.5's first year of life.  But what's in store for 2009?  Well, just as 1.5 up'd the standard compared to 1.0, we believe 1.6 is going to do continue the trend.

Joomla will hold it's first Developer Coordinator Summit in Australia this month.  Anthony, Louis, Sam, Wilco and I are going to be thrashing out the details for finalising the feature list for the final release, and if we are lucky we might even be close to cutting our first alpha after the event (providing we haven't been spending too much time playing cards or chasing kangaroos).

Notwithstanding that there is some other really cool work going on, 1.6 is about two main new features.  The first is a that we've completely rebuilt the way extensions are stored in the database and this will ultimately make it easier for developers to make multi-installable packages.  I'll let Sam blog about this more at another time.

The other feature is giving Joomla a fully feldged Access Control System.  In November and December last year the work on this progressed a long way.  A new Access Control component in the Administrator (under Site in the Menubar) now gives you the ability to create new User Groups, new Access Levels (more than just Public, Registered and Special) and several varities of Rules that help you control what a user should or shouldn't be able to do.  The User Interface for the rule building and the terminology still needs work but at least it's functional now.

As it stands at the moment, the Access Control System will include the ability to:

• lock groups of users down to access particular functions, like managing each extension, managing menus, installing a particular type of extension, etc;
• lock groups of users to be able to add, edit or remove content in particular categories (in other words, you will be able to define your own variants for existing groups like Author, Editor and Publisher);
• lock groups or users down to be able to view content in new access levels that you define (for example, you could create a Staff group and a Staff access level so that your Staff users can see content restricted to their group).

The "granularity" of what you can do is still up in the air a bit but that's the general direction we are heading.  Suffice to say though, that in itself is a major improvement over 1.0 and 1.5.

Most Administrator components and the Admin Menu are now locked down to the ACL system.  While a lot of performance tuning is still required, in principle it seems to be working well.  We've also introduced the concept of a "root" user that is able to perform any function (he/she sits above the Access Control System).  This is just a User ID that is set in the configuration.php file so that in the event of an emergency (like you accidentally blow up all the rules), you can set one user to be login to the site and rescue it from certain doom.  It also gives you the ability to limit the permission that even a Super Administrator has which could be useful on some sites.

For developers, the API for Access Control has been given a complete overhaul and we've tried to make it as easy as possible to interface with it during the installation of an extension.  If you want to peak at the progress, take a look in /libraries/joomla/acl/ and also in /installation/sql/mysql/install.php (that last file will probably move somewhere else eventually).  There are a couple of workability issues to iron out but after that hopefully I can drill into some pretty cool stuff you can do with it next month.

Regarding upgrading your site, we aren't sure of what the process will look like but suffice to say it will be a whole lot easier than the leap from 1.0 to 1.5.  However, a version 1.0 site will have to go through a migration to version 1.5 in order to get to version 1.6 or beyond.

Finally, it's worth reminding you that Joomla 1.6 will require a host with PHP 5.2 so start making preparations for changing hosts (or nagging them to upgrade their PHP) now.

Oh, and we should be getting a redesign of the developer.joomla.org site this year to.  It's our turn at last :)

Over the past three years, Joomla! development has evolved. During the split from the Mambo project the Joomla! Core Team was fully responsible for overall development. As the project grew, the Core Team realized that additional structures where required to organize everything around the Joomla! project. Mid-2006, the Joomla! Core Team changed from a "developers only" team into a team that had coordinators for several focused areas. One of the roles was the role of "Development Coordinator." I was the first Development Coordinator within the Joomla! project and we only had general ideas on how this role should be fulfilled. The role has changed over time, and is still changing as the project evolves.

In August 2007, we drafted a plan to work on a final version of Joomla! 1.5. Being able to work on maintenance releases was key for releasing a stable version of Joomla! 1.5. After the split the development team worked on maintenance versions for Joomla! 1.0 and a full refactoring for Joomla! 1.5, but until then we never had released a new major version. The development team was simply too small, and with the focus on creating new logic we decided to create a new team with maintenance as their main responsibility. The Bug Squad has also evolved from a relative small and unstructured team, into a large group of active people that is very well organized (see the Joomla! maintenance procedures if you are interested how things work in the Bug Squad).

In the early stages of the Joomla! Bug Squad, the number of team members quickly increased, and Anthony Ferrara was assigned as the first team lead. As mentioned before, this team is still evolving and new changes have been put in place to make sure things keep on running smoothly. The challenge, from an organizational aspect, is to keep the barriers for participation as low as possible. An initial blog post titled "Lowering Barriers" that I wrote explained our motivation at that time (May 2008).

After creating the Bug Squad around Christmas 2007, I realized that the amount of work that goes into coordinating the maintenance is huge and it would be pretty impossible to also drive the development of new major or minor version. And so, Andrew Eddie was invited to work along me as a second Development Coordinator. As time progressed we also asked Anthony Ferrera and Samuel Moffatt to move into that role, so we can continue to spread the increasing workload.

We now have a situation that Joomla! development is organized in two teams; the Development Team and the Joomla! Bug Squad. We keep evolving and implementing necessary improvements in the two teams. Let's look at the changes we have implemented so far.

Bug Squad

Within the Bug Squad we also have implemented some changes. We abandoned the position of team leader and created a new position of Joomla! Bug Squad Team Coordinator. We asked Ian MacLennan and Mark Dexter to fill that role, and we are glad they accepted. Within this role, they take care of the day to day operation of the Joomla! Bug Squad and help new members to find their way into this team. Andrew Eddie, Anthony Ferrara and Samuel Moffatt represent the core team, and along with me, the four of us support Ian and Mark working on the overall coordination (in the role as Development Team Coordinators) working hard on moving forward.

Along with the JBS team coordination roles we also have defined the roles of co-maintainers. The co-maintainers are responsible for committing patches that have moved to "ready-to-commit" status. Before they commit the code changes they do a final code validation (standards, quality, docbook mark-ups etc.). Ian MacLennan and Kevin Devine will be responsible for this role in the bug squad.

Development team

Let's start with the Google Summer of Code. As you could have read in the September edition of the magazine we had a very successful version of this great event. As a result, we had ten successful projects. We invited Chantal Bisson, Dahn Le Phuoc and Ercan Özkaya to join the Development Working Group. We are very happy to announce that they each accepted our invitation, strengthening the Development Working Group. We expect to see them accomplish more great work.

Andrew Eddie, Johan Janssens and Louis Landry held the position of lead developers. Since we continue to work to lower the barriers, we really want to move from positions to roles. The position of lead developer has served a very good purpose for building 1.5. To evolve to a new situation, we decided already several months ago it was time to abandon these positions. We have not yet decided if there will be any new positions within the Development Team, this really depends on how we are going to move forward with Joomla! 1.6. We plan to share status information about this release, as soon as possible.

I want to thank the new team members, and the members that have accepted the new roles and wish them a lot of fun in those teams!

As most of you know, a critical security vulnerability affecting all Joomla versions below (and including) 1.5.5 was discovered on Tuesday, August 12th 2008.  What most of you don't know, is what went on behind the scenes that day.  A whole mass of people came together and immediately worked on all the tasks necessary to make 1.5.6 happen.   Experiencing this first hand was quite amazing...  Publishing a release is a process that normally has two weeks (and a team of people) devoted to it (for everything from selecting which remaining artifacts will be fixed, to translations, to clicking publish and everything in-between).  This all happened in a VERY short time.

Here's an abridged breakdown of how 1.5.6 came to be...

15:50 EST

Bug Squad member Marijke Stuivenberg points the squad to a reported vulnerability in Joomla 1.5.5.

15:55 EST

Bug Squad members Jennifer Mariott, Elin Waring, and Marijke (along with development coordinator Wilco Jansen, OSM Vice President Rob Schley and myself) verify that the vulnerability exists and the report is valid.

15:56 EST

All available development Work Group members, Bug Squad members and Core Team members are notified of the issue.

Bug Squad confirms that 1.5's SVN is stable and is ready for immediate release pending vulnerability fix.

Forum moderators are informed of and asked to remove references of this issue until release.

16:05 EST

Patch is generated and provided to Bug Squad for testing/confirmation of fix.

16:20 EST

Patch is confirmed to fix vulnerability.

Front page announcement is drafted.

16:30 EST

Patch is committed into SVN along with all preparations for release. 

Joomla 1.5 branch is frozen for release cycle.  Bug Squad begins testing sanity and operation of SVN.

16:46 EST

Security announcement (on developer.joomla.org) is drafted.

17:20 EST

Front page announcement provided to translators.

Joomlacode prepared for release.

17:30 EST

Bug Squad confirms sanity of SVN and that all release preparations are in place.

Package generation begins. 

17:50 EST

Full download packages generated.

18:05 EST

Packages provided to Bug Squad for validation and testing.

18:30 EST

Bug Squad confirms package sanity, final steps before release are completed.

18:40 EST

Front Page article and Developer security report published. 

Full download packages released.

19:30 EST

All patch downloads tested and published.  Release cycle completed.

Conclusion

Total time from report of vulnerability to initial release: 2 hours 50 minutes

Total time from report of vulnerability to completion of release cycle completion: 3 hours 40 minutes

Total number of people directly involved: between 20 and 30

A List Apart is calling all designers, developers, information architects, project managers, writers, editors, marketers, and everyone else who makes websites. It is time once again to pool information so as to begin sketching a true picture of the way our profession is practiced worldwide.

I encourage all Joomla! professionals to take the survey.  The results from last year are also available.