Mozdev.org

Present: davidwboswell (David Boswell), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• hg testing released; two projects have asked for testing repos already
• secure update.rdf generation released; there's been some additional interest on the list, but no direct questions
• sent policy update to PO list; no objections so far
• setup admin interface to select active VCS and initialize hg repository
• left for hg: logging of commits to db, writing documentation, figuring out how to deal w/ source.html, figuring out what to do w/ /source/browse

Discussed sysadmin priorities

• connection limitations are in place for CGIs (bugzilla, cvsweb, mailman)
• bots/spiders have been hard on our server recently so this was very helpful
• several software packages were updated on both servers

Web stats/Top 50 page

• still ongoing

Staging server migration

• no updates

Additional SSL Certs

• hg.mozdev.org will be needed
• look into bugzilla.mozdev.org?
• in the future we'll need svn.mozdev.org
• wildcard cert? We're not sure about the techincal details w/ that yet
• probably cheaper to keep buying individual certs for the near future (4x $30/yr < $200/yr wildcard)

Next meeting May 13th, 2008 @ 15:00 UTC in #mozdev

I ended up spending a large part of last week fixing some architectural problems I created with the ProjectVCS/VCS classes and debugging some test cases, but overall the admin page for selecting a project's current VCS and initializing a repository is ready (announcement coming soon asking for beta testers).

Other tasks last week included fixing the Drupal spam module so it notifies the project owner rather than Mozdev admins and cleaning up the change password form to reduce the number of warnings.

This week I plan on getting logging of Mercurial commits into our VCS database log and figuring out how to best publish the new settings for projects that use a different VCS.

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)
Community present: ccaygill, djc, JesperHansen

Discussion was held publically in #mozdev

Discussed developer priorities

• worked on the backend for choosing VCS and initializing mercurial
• didn't get much time to work on the admin interface; working on that now
• some more cleanup from the spam a week ago
• research into what the proper mime type for .rdf files should be (application/rdf+xml)
• fixed password reset/change forms (missed from php5 upgrade)
• been short on time the past couple weeks due to being down a car and being driver for the family
• Doug will look into any recent errors that should be fixed

Discussed sysadmin priorities

• mostly trying to find spam sources and potential spam sources on our server
• some minor non-user-visible config changes
• doesn't look like there is any spam being generated from hosted php scripts

drupal admin messages going to webmaster@

• spam notifications were fixed to go to the site email (which is set to the project owner) rather than user 1 (which is always webmaster@)

Web stats/Top 50 page

• top 50 page is still blank; web stats are missing for april
• processing choked on new log file format; restarted this morning

email delay

• email was blocked from Saturday to this morning
• virus scanner died

Staging server migration

• no updates

Minor updates for unmaintained projects

• still waiting for Doug to write a policy and post to the PO list; will try to address this week

pay drupal developer to write wiki page on configuring discussed defaults

• the documentation would help with existing projects and new installs until we have the time to update the default install profile
• more discussion needed

Next meeting May 6th, 2008 @ 15:00 UTC in #mozdev

• Working on admin interface for new VCS' (bug#18960, bug#18958)
• Cleaned up spam comments from recent spam attack
• Disabled several forms that could be used for spamming
• Researching proper mime type for .rdf files - Determined that application/rdf+xml is the proper mime type for update.rdf and install.rdf files despite finding other documentation that text/xml is all that's supported (thanks Mossop!)
• Fixed password reset/change forms

I didn't quite get as much done last week as I had hoped so I'm still working on getting the admin interface for making VCS changes finished.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner), djc (autocopy extension owner)

Discussion was held publically in #mozdev

Discussed developer priorities

• spam attack last week has been cleaned up; old notes system is currently disabled
• mercurial is setup on staging at hg.vebzom.org; authentication is currently working against tigris database using existing CVS perms
• currently working on getting admin tools setup to select VCS and create repositories
• once admin tools are in place (hopefully this week) we can have devs test this on staging
• no response from community for secure updates testing; just going to release it as "beta" and deal with bugs later

Discussed sysadmin priorities

• spam attack ate up all the time last week
• logging of outgoing messages was broken since PHP 5 upgrade; fixed and improved
• gjm is monitoring abuse@mozdev now and replied to our upstreams about our recent spam attack
• some security patches were applied to servers
• copied production apache/nginx config to staging for Doug

Web stats/Top 50 page

• log analysis has been restarted (about a month behind)

Staging server migration

• no updates

Firefox updates server load handling

• handled firefox 2.0.0.14 release well

Cooperation between AMO and Mozdev.org on abandoned project adoption policy

• djc adopted auto-copy extension on mozdev.org bug would like to take over addon on AMO
• https://bugzilla.mozilla.org/show_bug.cgi?id=430048
• would be nice if Mozdev could work w/ Addons on this type of thing

ericjung would like to bring passwordmaker.org's mediawiki install back to mozdev.org

• should be possible; we'll need to work on the configuration a little bit

announce Mozdev.org sysadmin meetings?

• Doug will post the next meeting time in his minutes
• we should try to get the agenda for the upcoming week in a wiki for people to edit/comment

project status update form is being spammed

• gjm will post to sysadmin list to ensure it's safe to be disabled
• Doug will look into removing the form and links to it

Next meeting April 29th, 2008 @ 15:00 UTC

Much of last week was spent getting a test web interface setup for Mercurial. We also had a spam attack on our old notes system that required us to disable the script shortly after it was re-enabled. We're still trying to cleanup from that problem but it should be resolved shortly.

This week I plan on continue working on setting up Mercurial and working on the tools needed for supporting it.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• got mercurial plan together; started getting mercurial setup on vebzom
• deployed update.rdf generation, but need some testers
• various bug triaging
• we have some mod_rewrite bugs still lingering; trying to get my test suite fixed to track them down
• otherwise working on mercurial and update.rdf testing

Discussed sysadmin priorities

• still need to work on log rotation scripts
• fixing mod_rewrite bugs as they come up
• apache2 is serving http and https

Review roadmap changes

• MXR tool (replaces LXR) support, CVS, SVN, and hg
• updated roadmap will be announced to POs later today

Staging server migration

• no updates

Firefox updates server load handling

• no updates

• Triaging various site bugs after Apache 2.2/PHP 5 upgrade
• Putting together Mercurial setup plan
• Test suite improvements - testing various web pages on mozdev.org
• Moved secure update.rdf generation to production for testing - Thanks to bug#18732 being moved to production I was also able to get the update.rdf generation setup; we're still looking for testers, so please let me know if you're interested in helping.
• Trying to highlight Chatzilla extension support at Mozdev.org - We would like to see Chatzilla in our supported applications list, but its plugins aren't XPIs so support will probably need to be manually added.
• Backend changes to support stronger hashes for secure installations were completed - more work is needed to make support live, but the backend was put in place when I was working on supporting the updateInfoUrl for bug#18526

This week I plan to continue working on getting a Mercurial setup on vebzom while testing out the update.rdf generation.

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• testing apache 2.2/php 5 changes
• created project tagging policy/docs
• started update.rdf generation for secure updates
• started setting up test suite for mozdev.org code
• this week is planned to be: finish up the hg setup plan (in progress), start working on hg, and try to get some sysadmin time to close some bugs (apache 2.2 rewrite bugs, web-visible cache directory setup)
• asked about working on mercurial before svn; davidwboswell says an update to the roadmap is coming soon

Discussed sysadmin priorities

• apache 2.2 and php 5 upgrade is complete; working on ironing out some bugs
• https is being setup and tested

Firefox updates server load handling

• no updates

Staging server migration

• no news on VMs
• server move should be highest priority now that apache/php setup is done

Other projects

• if staging server setup drags on too long, sysadmin might move on to cvs perms with pam auth
• project creation automation would be a good item to work on as well
• openid would be a nice authentication mechanism for users; integrating this with other auth mechanisms isn't understood well right now

Code testing

• mostly just unit tests now that PHP 5 is avaialble (PHP 4 couldn't do mock objects which was very limiting)
• Doug will be writing some tests to verify certain web paths are working correctly
• not really focused on full integration testing or continuous integration right now (the entire web stack isn't in version control, so we can't know when changes are made)

Last week was mostly monopolized by testing/debugging after our Apache 2.2 and PHP 5 upgrade. There's still some bugs to iron out, but it appears that the main site and most project sites are running well.

Other tasks include:

• Creating project tagging policy/docs
• Generation of update.rdf files for secure updates - not quite ready for production (waiting on bug#18732)
• Setting up test suite for Mozdev.org code (not in production yet)

I didn't quite get to writing out the plan for adding additional VCSes to Mozdev.org due to the short week and testing/debugging of our new web stack. I plan on getting started on that this week but most of my attention is going to be focused on getting our new Apache/PHP setup stabilized.

Present: davidwboswell (David Boswell), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• project overview page/secure installs was generally well-received
• several minor bugs in file management, presentation, and extension parsing were handled quickly
• working on a tagging policy to help project owners know what to expect to be approved: http://www.mozdev.org/drupal/wiki/MozdevProjectTagging
• improved download counters offered by mozdev
• setup web-visible cache directory on the staging server w/ info for migration to production
• work on update.rdf generation is going well; script to generate files is mostly complete; need to add 'update info' url to file management and test the resulting update.rdf files; this will require the web-visible cache directory setup to finish
• should be starting on subversion setup plan this week and filing bugs using email I sent earlier to sysadmin@ as a base
• visiting family this weekend; offline Friday

Discussed sysadmin priorities

• nginx has been in production for about a week and has been handling load from updates fine
• web stats are currently broken (since the 25th) due to nginx setup and apache2/php5 update
• apache2/php5 is being setup in production today
• backups have not been occuring (reason isn't known yet); gjm is working w/ osuosl to get them working again

Firefox updates server load handling

• update was handled well; there were several spikes in bandwidth/requests but they were handled by nginx
• silfreed/gjm will continue to work on improving nginx setup by serving more files, but the solution worked incredibly well
• some projects get lots of 404 hits for their update.rdf requests; gjm will make a list of projects and silfreed will contact POs to make sure they're aware their users don't have an upgrade path

Staging server migration

• mozilla has said "yes" to 2U rack space and probably a VM
• osuosl is having disk capacity problems so a VM isn't immediately available
• first plans for for VM (hopefully from OSUOSL) is moving download master
• separating email and web traffic would probably be the next step
• figuring out how to split projects up across servers would also be useful from a security viewpoint (split mozdev.org projects from hosted projects)
• we'll still need to ask TWS to ship our server when we decide where its going (Mozilla or retirement)

Last week's big event was the launch of secure installs with our project overview page. In only 5 days we already have 27% (119/440) of our extensions registered with our system. Unfortunately we only have 9% of the registered files setup to enable secure downloads (ie, they have verified their file hash) - we'll have to figure out if this is a problem with our tools or process or if it will just improve as new files are registered.

Other tasks touched this week include:

• Received some clarification on database policy
• Improved download counters by adding functions to retrieve only the download count for a file (no HTML) as well as the total downloads for a project for the last 30 days.
• Helped test and monitor server load thanks to improvements from bug#18735
• Setup web-visible cache directory on our staging server which will be useful for projects to generate their own update.rdf files (or other web-accessible cache data/images/etc. Also wrote some docs/configs for implementing it in production
• Began working on generation of update.rdf files for secure updates

This week I'd like to finish up the update.rdf generation and begin working on the plan for setting up subversion at Mozdev.org.

Mozdev's project overview pages are now live and feature secure installations using InstallTrigger.

Right now we don't have the download graph or the project stats in place, but the main functionality is there:

• Project name and description
• Project tags - projects are tagged with supported applications automatically
• Links to project tools
• Project activity
• Project extensions and downloads with InstallTrigger links

We're really excited to have this type of page in place that gives a nice overview of a project and available tools as well as providing everyone a way to install extensions securely.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner), ccaygill (MyCroft project)

Discussion was held publically in #mozdev

Discussed developer priorities

• about ready to release the secure installation/project overview stuff; been working w/ a PO on testing things; trying to figure out a bug in the file release part, but otherwise things seem good
• plan to get the secure installation announced and in production this week
• did some testing of the lightweight web server last week and couldn't find any problems with the setup on vebzom
• fixed a bug in Drupal authentication module when editing mozdev cvs users
• changed the season starts for mozdev's themes to the equinoxes/solstices
• secure updates shouldn't take too long - it's mostly backend stuff that won't really have a UI anyway; hopefully only a week left

Database policy (bug#15661)

• we need to separate our policy decision from our technical limitations
• Doug will file a separate bug about needing to be able to monitor what our project usage is

Discussed sysadmin priorities

• preparations for apache2/php5 are underway
• setup php eaccelerator at the end of last week
• looking at spam filtering to reduce amount of cpu time used by increasing number of firewalled hosts

Firefox updates server load handling

• 2.0.0.13 release is scheduled for today
• still need to reduce amount of pages that get handled by apache/php

Staging server migration

• moco might be able to provide a VM; hopefully hear back later this week
• no news about VMs from OSUOSL yet

• Continued testing of project overview/secure install page
• Testing lightweight web server and PHP accelerator
• Setting up web-visible cache directory for projects
• Fixed bug in Drupal mozdevauth module that displayed an error when editing a user's information in Drupal that was also a Mozdev CVS user
• Fixed the start of Spring and other season starts for Mozdev.org's seasonal themes

This week I hope to get the secure installation released to project owners and begin work on the secure update.rdf generation for mozdev.org projects. The high priority remains to try to help get mozdev.org server load handled during Firefox updates, but things seem to be progressing well on that front. Hopefully I'll be able to get a test subversion server setup soon and being work on integrating that into Mozdev.org's architecture.