Mozdev.org

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• setup mercurial commit logging last week
• upgraded (staging) mercurial to 1.0
• not much other mercurial work accomplished
• updated external links in the mozdev.org header
• released the secure update tool; fixed several minor bugs and updated documentation to make it a little easier to understand
• updated the mozdev.org policy to allow us to fix web errors (and fixed some minor errors for projects I had notified several weeks ago)
• working on updating application list so it can be more easily modified for sponsors
• will be setting up an ad system to track displays, views, clicks, etc for sponsors
• hope to finish up sponsor work by Wednesday, then work on testing SSL certs for Mercurial (and SVN)

Discussed sysadmin priorities

• cleaned up disk space sunday night
• log files have been moved to stats.mozdev.org for archival
• nagios notifications were disabled for drive stats; gjm will check into that

Web stats/Top 50 page

• still processing

Staging server migration

• mozilla said things are still moving for getting a VM there

Next meeting May 20th, 2008 @ 15:00 UTC in #mozdev

• Call for hg testers - several project owners stepped up and said they'd test things out. I haven't heard any reports yet on how things work.
• Setup commit logging - Updated my svn2mysql tool to vcslog2mysql to work with mercurial as well. Other tools that rely on the cvslog need to be updated to work with the new tables.
• Upgraded Mercurial to 1.0
• Update external links in mozdev heading
• Released secure update tool
• SSL cert research for new Mozdev websites - Started looking into multi-domain UCC certs to support new Mozdev subdomains
• Updated Mozdev policy to allow for fixing web errors

This week I'd like to finalize plans for OSCON, I need to do some work for Mozdev's sponsors, and I'll continue working on getting Mercurial setup as an option for projects.

Present: davidwboswell (David Boswell), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• hg testing released; two projects have asked for testing repos already
• secure update.rdf generation released; there's been some additional interest on the list, but no direct questions
• sent policy update to PO list; no objections so far
• setup admin interface to select active VCS and initialize hg repository
• left for hg: logging of commits to db, writing documentation, figuring out how to deal w/ source.html, figuring out what to do w/ /source/browse

Discussed sysadmin priorities

• connection limitations are in place for CGIs (bugzilla, cvsweb, mailman)
• bots/spiders have been hard on our server recently so this was very helpful
• several software packages were updated on both servers

Web stats/Top 50 page

• still ongoing

Staging server migration

• no updates

Additional SSL Certs

• hg.mozdev.org will be needed
• look into bugzilla.mozdev.org?
• in the future we'll need svn.mozdev.org
• wildcard cert? We're not sure about the techincal details w/ that yet
• probably cheaper to keep buying individual certs for the near future (4x $30/yr < $200/yr wildcard)

Next meeting May 13th, 2008 @ 15:00 UTC in #mozdev

I ended up spending a large part of last week fixing some architectural problems I created with the ProjectVCS/VCS classes and debugging some test cases, but overall the admin page for selecting a project's current VCS and initializing a repository is ready (announcement coming soon asking for beta testers).

Other tasks last week included fixing the Drupal spam module so it notifies the project owner rather than Mozdev admins and cleaning up the change password form to reduce the number of warnings.

This week I plan on getting logging of Mercurial commits into our VCS database log and figuring out how to best publish the new settings for projects that use a different VCS.

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)
Community present: ccaygill, djc, JesperHansen

Discussion was held publically in #mozdev

Discussed developer priorities

• worked on the backend for choosing VCS and initializing mercurial
• didn't get much time to work on the admin interface; working on that now
• some more cleanup from the spam a week ago
• research into what the proper mime type for .rdf files should be (application/rdf+xml)
• fixed password reset/change forms (missed from php5 upgrade)
• been short on time the past couple weeks due to being down a car and being driver for the family
• Doug will look into any recent errors that should be fixed

Discussed sysadmin priorities

• mostly trying to find spam sources and potential spam sources on our server
• some minor non-user-visible config changes
• doesn't look like there is any spam being generated from hosted php scripts

drupal admin messages going to webmaster@

• spam notifications were fixed to go to the site email (which is set to the project owner) rather than user 1 (which is always webmaster@)

Web stats/Top 50 page

• top 50 page is still blank; web stats are missing for april
• processing choked on new log file format; restarted this morning

email delay

• email was blocked from Saturday to this morning
• virus scanner died

Staging server migration

• no updates

Minor updates for unmaintained projects

• still waiting for Doug to write a policy and post to the PO list; will try to address this week

pay drupal developer to write wiki page on configuring discussed defaults

• the documentation would help with existing projects and new installs until we have the time to update the default install profile
• more discussion needed

Next meeting May 6th, 2008 @ 15:00 UTC in #mozdev

• Working on admin interface for new VCS' (bug#18960, bug#18958)
• Cleaned up spam comments from recent spam attack
• Disabled several forms that could be used for spamming
• Researching proper mime type for .rdf files - Determined that application/rdf+xml is the proper mime type for update.rdf and install.rdf files despite finding other documentation that text/xml is all that's supported (thanks Mossop!)
• Fixed password reset/change forms

I didn't quite get as much done last week as I had hoped so I'm still working on getting the admin interface for making VCS changes finished.

Many people who host the development of their extension on mozdev also distribute their work on addons.mozilla.org and not surprisingly we've received a number of requests to make managing projects in both locations easier for developers.

From these requests and from other discussions with project owners, we've known that more integration would help some developers but we haven't had any specific information about how many people would benefit. Well, today I had the idea that I should just go and look at how many add-ons on AMO have mozdev listed as their developer home page. The results for the 50 most popular add-ons are below:

I think there are a few interesting things to see here. First is that most people are hosting their own developer site, but of the project hosting sites mozdev is by far the most used (I have no idea if this ratio holds across all of AMO and would be interested in doing a more complete survey). There are also a number of projects that had hosted their project on mozdev at one time and are now hosting their own sites (although some of these are still using some of mozdev's developer tools). I think this number is encouraging since it's natural for projects to want their own site once they've reached a certain size and this fits in with our ideas of having mozdev serve as a community incubator.

To answer the original question though, it looks like we could make things easier for a significant fraction of the developer's on AMO by making it easier to manage projects in both places. There are a lot of potential things that could be done (have download files sync across sites so you just need to update once, share account information so people don't need to remember multiple passwords to deal with one extension...). If anyone has ideas for what would be helpful here, please let us know.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner), djc (autocopy extension owner)

Discussion was held publically in #mozdev

Discussed developer priorities

• spam attack last week has been cleaned up; old notes system is currently disabled
• mercurial is setup on staging at hg.vebzom.org; authentication is currently working against tigris database using existing CVS perms
• currently working on getting admin tools setup to select VCS and create repositories
• once admin tools are in place (hopefully this week) we can have devs test this on staging
• no response from community for secure updates testing; just going to release it as "beta" and deal with bugs later

Discussed sysadmin priorities

• spam attack ate up all the time last week
• logging of outgoing messages was broken since PHP 5 upgrade; fixed and improved
• gjm is monitoring abuse@mozdev now and replied to our upstreams about our recent spam attack
• some security patches were applied to servers
• copied production apache/nginx config to staging for Doug

Web stats/Top 50 page

• log analysis has been restarted (about a month behind)

Staging server migration

• no updates

Firefox updates server load handling

• handled firefox 2.0.0.14 release well

Cooperation between AMO and Mozdev.org on abandoned project adoption policy

• djc adopted auto-copy extension on mozdev.org bug would like to take over addon on AMO
• https://bugzilla.mozilla.org/show_bug.cgi?id=430048
• would be nice if Mozdev could work w/ Addons on this type of thing

ericjung would like to bring passwordmaker.org's mediawiki install back to mozdev.org

• should be possible; we'll need to work on the configuration a little bit

announce Mozdev.org sysadmin meetings?

• Doug will post the next meeting time in his minutes
• we should try to get the agenda for the upcoming week in a wiki for people to edit/comment

project status update form is being spammed

• gjm will post to sysadmin list to ensure it's safe to be disabled
• Doug will look into removing the form and links to it

Next meeting April 29th, 2008 @ 15:00 UTC

Much of last week was spent getting a test web interface setup for Mercurial. We also had a spam attack on our old notes system that required us to disable the script shortly after it was re-enabled. We're still trying to cleanup from that problem but it should be resolved shortly.

This week I plan on continue working on setting up Mercurial and working on the tools needed for supporting it.

The roadmap for mozdev.org was updated earlier this week. The motivation behind these changes was to refocus our efforts on functionality that will benefit Mozilla developers and to address the concerns we've been hearing about the usability of the site. More details about our thought process can be seen on the recent post about the proposed changes.

I wanted to point out some of the bigger changes that were made. We've been planning on adding at least one new version control option for a while and we had initially considered going with Subversion support first. In the roadmap, one of our new top priorities is to add Mercurial support (Subversion support is still planned, but it will come later). Since our mission is to support Mozilla developers, and not open source developers in general, we feel that it is important to stay in sync with Mozilla's development tools and provide other tools that are specific to the Mozilla community.

A number of tasks that should help with the usability of the site have also been added, including making edits to a project's web pages easier, making the creation of new projects a quicker process and redesigning the look of the site. We feel that there are many reasons why a developer working on Mozilla extensions or applications would choose mozdev.org over a general purpose hosting site, but there is no denying that the current design and layout of our site is a far cry from the usability of Google Code.

If there are any comments or suggestions about these changes, feel free to let us know.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• got mercurial plan together; started getting mercurial setup on vebzom
• deployed update.rdf generation, but need some testers
• various bug triaging
• we have some mod_rewrite bugs still lingering; trying to get my test suite fixed to track them down
• otherwise working on mercurial and update.rdf testing

Discussed sysadmin priorities

• still need to work on log rotation scripts
• fixing mod_rewrite bugs as they come up
• apache2 is serving http and https

Review roadmap changes

• MXR tool (replaces LXR) support, CVS, SVN, and hg
• updated roadmap will be announced to POs later today

Staging server migration

• no updates

Firefox updates server load handling

• no updates

• Triaging various site bugs after Apache 2.2/PHP 5 upgrade
• Putting together Mercurial setup plan
• Test suite improvements - testing various web pages on mozdev.org
• Moved secure update.rdf generation to production for testing - Thanks to bug#18732 being moved to production I was also able to get the update.rdf generation setup; we're still looking for testers, so please let me know if you're interested in helping.
• Trying to highlight Chatzilla extension support at Mozdev.org - We would like to see Chatzilla in our supported applications list, but its plugins aren't XPIs so support will probably need to be manually added.
• Backend changes to support stronger hashes for secure installations were completed - more work is needed to make support live, but the backend was put in place when I was working on supporting the updateInfoUrl for bug#18526

This week I plan to continue working on getting a Mercurial setup on vebzom while testing out the update.rdf generation.

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• testing apache 2.2/php 5 changes
• created project tagging policy/docs
• started update.rdf generation for secure updates
• started setting up test suite for mozdev.org code
• this week is planned to be: finish up the hg setup plan (in progress), start working on hg, and try to get some sysadmin time to close some bugs (apache 2.2 rewrite bugs, web-visible cache directory setup)
• asked about working on mercurial before svn; davidwboswell says an update to the roadmap is coming soon

Discussed sysadmin priorities

• apache 2.2 and php 5 upgrade is complete; working on ironing out some bugs
• https is being setup and tested

Firefox updates server load handling

• no updates

Staging server migration

• no news on VMs
• server move should be highest priority now that apache/php setup is done

Other projects

• if staging server setup drags on too long, sysadmin might move on to cvs perms with pam auth
• project creation automation would be a good item to work on as well
• openid would be a nice authentication mechanism for users; integrating this with other auth mechanisms isn't understood well right now

Code testing

• mostly just unit tests now that PHP 5 is avaialble (PHP 4 couldn't do mock objects which was very limiting)
• Doug will be writing some tests to verify certain web paths are working correctly
• not really focused on full integration testing or continuous integration right now (the entire web stack isn't in version control, so we can't know when changes are made)

Last week was mostly monopolized by testing/debugging after our Apache 2.2 and PHP 5 upgrade. There's still some bugs to iron out, but it appears that the main site and most project sites are running well.

Other tasks include:

• Creating project tagging policy/docs
• Generation of update.rdf files for secure updates - not quite ready for production (waiting on bug#18732)
• Setting up test suite for Mozdev.org code (not in production yet)

I didn't quite get to writing out the plan for adding additional VCSes to Mozdev.org due to the short week and testing/debugging of our new web stack. I plan on getting started on that this week but most of my attention is going to be focused on getting our new Apache/PHP setup stabilized.

Present: davidwboswell (David Boswell), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• project overview page/secure installs was generally well-received
• several minor bugs in file management, presentation, and extension parsing were handled quickly
• working on a tagging policy to help project owners know what to expect to be approved: http://www.mozdev.org/drupal/wiki/MozdevProjectTagging
• improved download counters offered by mozdev
• setup web-visible cache directory on the staging server w/ info for migration to production
• work on update.rdf generation is going well; script to generate files is mostly complete; need to add 'update info' url to file management and test the resulting update.rdf files; this will require the web-visible cache directory setup to finish
• should be starting on subversion setup plan this week and filing bugs using email I sent earlier to sysadmin@ as a base
• visiting family this weekend; offline Friday

Discussed sysadmin priorities

• nginx has been in production for about a week and has been handling load from updates fine
• web stats are currently broken (since the 25th) due to nginx setup and apache2/php5 update
• apache2/php5 is being setup in production today
• backups have not been occuring (reason isn't known yet); gjm is working w/ osuosl to get them working again

Firefox updates server load handling

• update was handled well; there were several spikes in bandwidth/requests but they were handled by nginx
• silfreed/gjm will continue to work on improving nginx setup by serving more files, but the solution worked incredibly well
• some projects get lots of 404 hits for their update.rdf requests; gjm will make a list of projects and silfreed will contact POs to make sure they're aware their users don't have an upgrade path

Staging server migration

• mozilla has said "yes" to 2U rack space and probably a VM
• osuosl is having disk capacity problems so a VM isn't immediately available
• first plans for for VM (hopefully from OSUOSL) is moving download master
• separating email and web traffic would probably be the next step
• figuring out how to split projects up across servers would also be useful from a security viewpoint (split mozdev.org projects from hosted projects)
• we'll still need to ask TWS to ship our server when we decide where its going (Mozilla or retirement)