Mozdev.org

• Call for hg testers - several project owners stepped up and said they'd test things out. I haven't heard any reports yet on how things work.
• Setup commit logging - Updated my svn2mysql tool to vcslog2mysql to work with mercurial as well. Other tools that rely on the cvslog need to be updated to work with the new tables.
• Upgraded Mercurial to 1.0
• Update external links in mozdev heading
• Released secure update tool
• SSL cert research for new Mozdev websites - Started looking into multi-domain UCC certs to support new Mozdev subdomains
• Updated Mozdev policy to allow for fixing web errors

This week I'd like to finalize plans for OSCON, I need to do some work for Mozdev's sponsors, and I'll continue working on getting Mercurial setup as an option for projects.

I ended up spending a large part of last week fixing some architectural problems I created with the ProjectVCS/VCS classes and debugging some test cases, but overall the admin page for selecting a project's current VCS and initializing a repository is ready (announcement coming soon asking for beta testers).

Other tasks last week included fixing the Drupal spam module so it notifies the project owner rather than Mozdev admins and cleaning up the change password form to reduce the number of warnings.

This week I plan on getting logging of Mercurial commits into our VCS database log and figuring out how to best publish the new settings for projects that use a different VCS.

• Working on admin interface for new VCS' (bug#18960, bug#18958)
• Cleaned up spam comments from recent spam attack
• Disabled several forms that could be used for spamming
• Researching proper mime type for .rdf files - Determined that application/rdf+xml is the proper mime type for update.rdf and install.rdf files despite finding other documentation that text/xml is all that's supported (thanks Mossop!)
• Fixed password reset/change forms

I didn't quite get as much done last week as I had hoped so I'm still working on getting the admin interface for making VCS changes finished.

Much of last week was spent getting a test web interface setup for Mercurial. We also had a spam attack on our old notes system that required us to disable the script shortly after it was re-enabled. We're still trying to cleanup from that problem but it should be resolved shortly.

This week I plan on continue working on setting up Mercurial and working on the tools needed for supporting it.

• Triaging various site bugs after Apache 2.2/PHP 5 upgrade
• Putting together Mercurial setup plan
• Test suite improvements - testing various web pages on mozdev.org
• Moved secure update.rdf generation to production for testing - Thanks to bug#18732 being moved to production I was also able to get the update.rdf generation setup; we're still looking for testers, so please let me know if you're interested in helping.
• Trying to highlight Chatzilla extension support at Mozdev.org - We would like to see Chatzilla in our supported applications list, but its plugins aren't XPIs so support will probably need to be manually added.
• Backend changes to support stronger hashes for secure installations were completed - more work is needed to make support live, but the backend was put in place when I was working on supporting the updateInfoUrl for bug#18526

This week I plan to continue working on getting a Mercurial setup on vebzom while testing out the update.rdf generation.

Last week was mostly monopolized by testing/debugging after our Apache 2.2 and PHP 5 upgrade. There's still some bugs to iron out, but it appears that the main site and most project sites are running well.

Other tasks include:

• Creating project tagging policy/docs
• Generation of update.rdf files for secure updates - not quite ready for production (waiting on bug#18732)
• Setting up test suite for Mozdev.org code (not in production yet)

I didn't quite get to writing out the plan for adding additional VCSes to Mozdev.org due to the short week and testing/debugging of our new web stack. I plan on getting started on that this week but most of my attention is going to be focused on getting our new Apache/PHP setup stabilized.

Last week's big event was the launch of secure installs with our project overview page. In only 5 days we already have 27% (119/440) of our extensions registered with our system. Unfortunately we only have 9% of the registered files setup to enable secure downloads (ie, they have verified their file hash) - we'll have to figure out if this is a problem with our tools or process or if it will just improve as new files are registered.

Other tasks touched this week include:

• Received some clarification on database policy
• Improved download counters by adding functions to retrieve only the download count for a file (no HTML) as well as the total downloads for a project for the last 30 days.
• Helped test and monitor server load thanks to improvements from bug#18735
• Setup web-visible cache directory on our staging server which will be useful for projects to generate their own update.rdf files (or other web-accessible cache data/images/etc. Also wrote some docs/configs for implementing it in production
• Began working on generation of update.rdf files for secure updates

This week I'd like to finish up the update.rdf generation and begin working on the plan for setting up subversion at Mozdev.org.

• Continued testing of project overview/secure install page
• Testing lightweight web server and PHP accelerator
• Setting up web-visible cache directory for projects
• Fixed bug in Drupal mozdevauth module that displayed an error when editing a user's information in Drupal that was also a Mozdev CVS user
• Fixed the start of Spring and other season starts for Mozdev.org's seasonal themes

This week I hope to get the secure installation released to project owners and begin work on the secure update.rdf generation for mozdev.org projects. The high priority remains to try to help get mozdev.org server load handled during Firefox updates, but things seem to be progressing well on that front. Hopefully I'll be able to get a test subversion server setup soon and being work on integrating that into Mozdev.org's architecture.

• Ensuring the Spring theme works
• Created project overview page with InstallTrigger links - the initial design is done; we're going through a short testing period before we set it up for all projects
• Research subversion and Mercurial configuration for bug#11890

This week I'll be focusing on testing the lightweight web server to reduce Mozdev's load, trying to setup a new web-writable/visible save directory, and trying to deploy the secure installations.

The biggest topic of the past week has been mitigating load problems on Mozdev's server when Firefox releases an update. We now have a plan in place to reduce the load by using a lightweight web server to handle update requests. We're also going to help out the Mycroft project update their site to use a new cache directory that is web-visible.

Other tasks include:

• Started on project overview page - This will be the page where secure installations are performed from.
• Fixed bug with .php pages and $_SERVER['PATH_INFO'] in new Apache rewrite rules
• Fixed some notices that were making testing difficult and cleaned up a lot of functions/files/definitions in hovercraft/sandbox/php that were outdated
• Tying Mozdev projects to bugzilla products more tightly - Right now Mozdev projects are only tied to our Bugzilla products by the project and product name; I'm trying to improve this since there are some projects (only mozdev-related right now) where the two don't match and it's difficult to automatically generate the links, or even know if a bugzilla product is enabled for that matter.
• Fixed D.MD.o bug for recently-uploaded files - Files that were recently-uploaded that hadn't made it to a mirror yet could not be downloaded to to a bug that only served the first 8192 bytes of the file.

We are trying to get Mozdev's load problem resolved or at least improved before the next Firefox update (2.0.0.13) on March 25th so my focus will be testing our new features as they become available and helping with implementation.

Otherwise I'll continue working on the frontend for performing secure installations of extensions.

• Integrated extension list and tagged projects for an application
• Reduced server load for D.MD.o - I updated D.MD.o to serve update requests from any random mirror instead of the best mirror. I also improved the query that finds the best mirror to use better indexing which should reduce the time it takes to redirect a user and reduce load.
• Notified POs about move to Apache 2 and PHP 5 on March 31st
• Worked on mockups for a project-overview page which will be utilized as a template for making a project's download files page for providing secure installs
• Discussions on how to reduce server load and handle Firefox upgrades

• Download file registration/release backend (bug#12486, bug#17302) - the UI to allow projects to register and release their download files is almost complete. This will allow project owners to allow secure installations of their extensions and generation of update.rdf files once those features are complete.
  There were a lot of small backend fixes such as adding support for 'deleted' files in our cache, fixing some detection of when a file is replaced, and fixing a bug introduced over the previous weekend that broke our caching of new files for several projects.
• Reduce D.MD.o load - I was able to reduce the load on our servers by redirecting update requests to a random mirror instead of the user's best mirror.
  I found that 62% of users hitting our server for an update only requested the update.rdf file (or equivalent). Out of that 62%, only 1.5% actually retrieved an update after that. So we were spending a lot of time trying to figure out where to send the user for a very small file.
• Misc page cleanups (bug#18671, bug#18669, bug#18677, bug#18667)

This week I'm taking off tomorrow for my daughter's 1st birthday, but I hope to get a good start on the project download file interface that will allow secure installations of extensions. I'll also be done with handling a project's applications and will have that released and the documentation to go along with it.

• Finding supported applications automatically is deployed - Mozdev.org now has a list of applications that have extensions hosted with us. We're automatically tagging projects with the application names as well in order to make finding projects for an application (not just extensions) easier.
• Document how to make mailman default config less annoying - I've created a page detailing the steps we're considering to make administering lists easier that you can use to make your life easier now.
• Updated project search interface to search applications and extensions

This week I will spending some time ensuring our download system can handle FireFox updates (we had some problems handling the last FireFox 2.0.0.12 update), working on ideas for improving site navigation, and continuing on a download file registration system to be used by Project Owners for approving files and verifying uploads.

• Finding supported applications automatically - I have the portion that parses .xpi files' install manifests (install.rdf files) working - it currently tracks what extension an .xpi file is as well as what applications and version ranges it supports. The frontend for displaying the applications Mozdev.org projects support and the extensions for each application is almost complete; only some minor styling work left. An admin interface is also complete for managing information about the applications. Left to do is to link this data into the project search interface (bug#18525) and create a download file manager for project owners to be able to "release" a download file for public consumption.
• Updated the project search interface to support more search words - previously we only supported searching projects and descriptions with words over 4 characters or longer; we updated our database to support searching words 3 characters or longer - this should make the tool much more useful for finding projects.

This week I'll continue to work on the frontends for displaying supported applications and searching applications and extensions as well as the download file management interface (which will be crucial for secure installations as well).

• Released Project search interface - There is still a small bug to work out here, but having a search form for finding projects at Mozdev should be a huge help for users trying to find a project.
• Parse supported applications from install.rdf files - I made a lot of progress on being able to parse .xpi files and pull information out of the install.rdf file. I should be testing this week and hopefully have the parsing code in place towards the end of the week. After that I can work on creating a frontend to show what applications are supported by our projects and adding search capability to it.