Just a quick note to tell y’all that I have moved my Firefox-related posts off Planet Plone, so if you want to subscribe to everything I write about — not just Plone-related stuff — take a pick from the feeds offered at the main site.

¹ If you can read Japanese, the Plone advent calendar can be found here. Since there has been a lot of Plone-related posts in the advent calendar¹ so far, I thought we’d go back and look at the new landscape of web development that has emerged in the last couple of years. It is a good time to teach yourself something new, or just dig deeper into the technologies you work with every day.

Most of us learned HTML and CSS the piecemeal, iterative way — we found our way around the standards as part of various projects, and gradually got better at it. Especially those coming from a coding background found it easier to not properly teach ourselves the “front-end” of web technologies, and often learned through copy & paste programming.

The browser world has come a long way since I taught myself the basics back in 1996 — and sometimes I forget how far we have come since then. Internet Explorer came and went, and the open web triumphed. Designing a site in 2009 that has no special support for IE6 is actually a feasible idea, something that seemed very remote just a few years ago.

So if I had to start teaching myself web technologies today, I would start with the following articles — which should be a good starting point even if you consider yourself a seasoned web developer — the web is very different from what it was just a few years ago:

• Dive into HTML5 — you can use HTML5 features, today. Excellent guide from Mark Pilgrim, still under development.
• Why Programmers Suck at CSS Design — a great, entertaining primer on CSS for everyone.
• CSS Techniques I Wish I Knew When I Started Designing Websites — a good list of stuff you still have to know about when it comes to CSS.
• Make Your Mockup in Markup — are you still designing web sites in Photoshop? Think again.
• jQuery Tutorials — every site uses JavaScript to enhance the user experience these days, and jQuery is a vital tool in your belt. Great for both quick prototyping and large-scale web sites.
• Teach yourself a grid system for layout — I of course recommend the Deco Grid System (which is part of the upcoming Plone 4 and 5 releases), but any grid system will do if you prefer others. Deco is just easier, less weird, and light-weight.
• Finally, two useful tools for designing and prototyping web sites: Mockingbird for creating wireframes, and Color Scheme Designer for the æsthetically challenged among us.

Hopefully, these links can teach you something new and exciting for 2010.

Happy holidays, wherever you are this time of year! With the final release of Plone 4 out soon, and Plone 5 development well underway, 2010 looks to be Plone’s best year yet!

Since starting work with the Firefox User Experience team here at Mozilla less than a year ago, I have fallen in love with my job again. Now, I want you to join in on the fun.

Mozilla Labs recently announced that they are looking for a UI designer that is ready to work on some of the most interesting & challenging problems of the future of the web.

Some of the things I personally think are exceptionally cool about being a UI designer at Mozilla:

• You get to — in fact, you have to — do all your work in the open. No secrets, instant feedback, and lots of good ideas from people in the community.
• The people building Firefox and the Mozilla platform really care about usability and user experience. You never hear That will be hard to build or I don’t think that’s worth it. The engineers figure out what to do, and make it happen. Every time.
• There’s very little ego around. A sharp focus on our key products means that there’s no bullshit politics. The company is small enough that you can get to know everyone, but big enough to change the world.
• You get to work on the cutting edge of web innovation, and experimenting with technology that will decide the future of identity on the web, how we can carry our online life with us across browsers, and new ways to let our users take control of the web.
• As part of Mozilla Labs in particular, you get to do a lot of experimental, exploratory work with concepts that haven’t even been invented yet.
• Your work — if you do it right — will improve the daily life of 300 million people, an astounding 25% of the people using the web.
• Mozilla works on making the web better, for everyone. They have the long-term view, and you get to change the world, every day. This is not just something we say, it’s something we do.

So, brush up your portfolio & résumé over the holidays, and apply for the job. Tell them Limi sent you.

Update, summer 2011: The Firefox team has decided to pursue these types of improvements via SPDY and HTTP Pipelining instead, but I will keep this original article about Resource Packages online.

What if there was a backwards compatible way to transfer all of the resources that are used on every single page in your site — CSS, JS, images, anything else — in a single HTTP request at the start of the first visit to the page? This is what Resource Package support in browsers will let you do.

Update: There's now a proper Resource Package specification, and any details there override whatever is written here. This article is a bit more conversational, and is useful to understand what we’re trying to do. If you’re looking for specifics — like what formats we support and how we parse — please refer to the formal specification instead of this article.

When it comes to browser performance, it’s widely known that a lot of the time is spent waiting for HTTP requests. You are probably familiar with the issue; a well-known optimization technique is to reduce the number of HTTP requests that are done for a given web site, since browsers only do 2–6 requests in parallel. This is why techniques like image spriting exist.

There are problems with image spriting, though. In addition to potentially severe memory penalties, they obfuscate the code — “What is this icon at 704px, exactly?” — and every time you add a new icon, you have to update the sprite file, which adds to the maintenance burden.

Some images can’t be sprited (think about YouTube, which easily serves up 40 JPEG thumbnails on a given page), and there’s also other resources like JavaScript & CSS, which — while possible to combine — at the very least need one file each. You can see how this quickly saturates the available parallel HTTP pipelines.

Even if bandwidth is getting better, and the internet is getting faster, latency are actually getting worse in a lot of cases. With mobile internet browsing, and to some extent US domestic cable internet and DSL, the round-trip time for a single request can be slow, even if it downloads relatively fast once the transfer starts.

While there are lots of workarounds to solve this class of problems, we suggest a standard approach that all browsers makers can easily implement, and that is backwards compatible with browsers that do not support it. We also want a solution that works for all types of resources, not only image bitmaps, and one that doesn’t require any new tools, file types or protocols.

Goals

This proposal has the following goals:

• Make it possible to serve all the resources (images, stylesheets, javascript) required by a page in a single HTTP request, freeing up the other parallel requests to fetch resources that are page-specific.
• Be as simple to implement as possible, so anyone with a passing familiarity with HTML should be able to perform the optimization.
• Be entirely transparent to browsers that do not support it.
• Avoid retransmission of existing resources.
• Use existing tools that are widely used on all platforms.
• Support the “80% use case” over adding a lot of complexity to the spec.

Non-goals

Some explicit non-goals of this proposal:

• Invent new file formats
• Invent new compression formats

Implementation

Our goal is to be compression-format independent, but the obvious candidates for support are:

• tar.gz, aka. .tgz files — MIME type: application/x-tar-gz
• zip files — MIME type: application/zip

While both zip and tar.gz files do not have not the most elegant or efficient packing format out there, they have the following very desirable traits:

• Easily available reference implementations.
• Can be unpacked even in partial state — which means that we can stream the file, and put CSS and JavaScript first in the archive, and they will unpacked and made available before the entire file has been downloaded.
• Excellent toolchain support, available on all major platforms, so it’s easy for web developers to use.

Other formats, like bzip may be more efficient in terms of file size, but they aren’t streamable and can’t be unpacked as partial files — so they are considered unfit for this particular purpose.

We propose this markup to signal a zipped resource package:

<link rel="resource-package" 
      type="application/zip" 
      href="site-resources.zip" />

The default MIME type for a resource package will be application/x-tar-gz, and you can omit it in documents where it is valid, like in HTML5, where an equivalent would be:

<link rel="resource-package" 
      href="site-resources.tar.gz" />

This will tell the browser to download this file first, and use the resources contained in the file instead of the referenced images, style sheets and javascript files — or for that matter, any other file. Browsers should prefer the files in the resource package, and do individual requests for images that are not contained in the package.

A given browser will probably block downloading any resources until the lists of files that are available in resource packages have been accounted for — or there may be a way to do opportunistic requests or similar, we leave this up to the browser vendor unless there’s a compelling reason to specify how this should work.

Older browsers that do not support resource packages will simply ignore this tag, and fetch the files normally, with one HTTP request for each.

Path handling

Paths will be rendered relative to where the resource package is located, so you can supply additional directories inside the resource package to mimic existing site structure.

The resource package is referenced as follows in a page somewhere on the site www.example.com:

<link rel="resource-package" 
      type="application/zip" 
      href="/static/site-resources.zip" />

The Zip file has this internal structure:

javascript/jquery.js
css/reset.css
css/grid.css
css/main.css
images/save.png
images/info.png

In this example, the resolved path to the main.css file would be http://www.example.com/static/css/main.css. Notice how the path inside the resource package is added to the path where the actual file is located.

File listing using inline HTML

The most efficient way to declare what’s in the resource package without blocking to wait for the first part of it to load, is to do it inline in the resource package <link> tag itself:

<link rel="resource-package" 
      type="application/zip" 
      href="/static/site-resources.zip"
      content="javascript/jquery.js;
               css/reset.css;
               css/grid.css;
               css/main.css;
               images/save.png;
               images/info.png" />

The only problem with the above is that it will break validation for existing specifications like HTML 4 and XHTML 1.x, so we also support an alternate syntax, (ab)using the title attribute to get the same result:

<link rel="resource-package" 
      type="application/zip" 
      href="/static/site-resources.zip"
      title="javascript/jquery.js;
             css/reset.css;
             css/grid.css;
             css/main.css;
             images/save.png;
             images/info.png" />

This makes the file work with the existing validators and older standards.

Fallback

There should be no compatibility issues with old browsers, as they will just load the individual files instead of the resource package.

Browsers that don’t implement this will seem slow in comparison to other browsers. Luckily, it should be a simple addition to any of the modern browsers.

File listing using a manifest file

You can also give the browser the ability to know what files are in the resource package file without reading the entire file first, by adding a manifest file that can contain this information. This file can be supplied as a separate file (useful if combining with Offline Resources), or as the first file in the file itself. Of course, this will be slower to get than defining the contents inline in the HTML, but can be easier and cleaner to implement, especially if you're using offline resource support already.

Example manifest.txt file:

javascript/jquery.js
styles/reset.css
styles/grid.css
styles/main.css
images/save.png
images/info.png

• This file must be the first file in the archive.
• This file must be named manifest.txt when supplied as part of the archive.

If this simple format looks familiar, that’s not a coincidence. Initially, we were looking at using either an XML or JSON format to specify this, but we believe it’s easier to add a couple of new abilities to the offline resource specification instead. When using resource packages with Offline Resources (which are also part of the HTML5 spec), we’d like it to be easy to extend the rules, so the offline manifest with resource package support could look like this:

CACHE: resources=/static/siteresources.zip
javascript/jquery.js
styles/reset.css
styles/grid.css
styles/main.css
images/save.png
images/info.png

# The above section lists the files in siteresources.zip.
# To start a new section, do:
CACHE:
images/outside-package.png

The only thing we’d need to add to the HTML5 offline spec is that it should ignore anything on the same line after CACHE: if it doesn’t know how to handle it. This means that you could potentially put the resource package definitions in your Offline Resources manifest — we would also support doing it the other way around, and put the Offline Resources manifest inside the resource package.

This isn’t a requirement for implementing the initial version of resource packages, however — but could be an easy way to add support for it to the offline resource specification. If there’s a better way to do it, let me know.

Duplicate files and override behavior

If a resource is defined twice on the same path — e.g. using multiple resource-packages — the file defined last takes priority. This enables an application with offline mode to synchronize changes without downloading the entire resource package again.

For example, on the first sync, the browser sees a resource package like this:

<link rel="resource-package" href="/documents/offline-0.zip" />

offline-0.zip contains:

manifest.txt 
doc1.html 
doc2.html 

Offline the application would be able to access documents at /documents/doc1.html and /documents/doc2.html.

The user then performs another sync and the browser now sees:

<link rel="resource-package" href="/documents/offline-0.zip" />
<link rel="resource-package" href="/documents/offline-1.zip" />

offline-1.zip contains:

manifest.txt 
doc1.html 
doc3.html

doc1.html and doc3.html would now be read from the offline-1.zip, doc2.html would continue to be read from offline-0.zip.

Even for use outside of the offline applications space, it allows pages to easily override the site look and feel with section-specific images and styling. A page could serve up:

<link rel="resource-package" href="/static/main-theme.zip" />
<link rel="resource-package" href="/static/section-theme.zip" />

Resources in section-theme.zip would take precedence over the content in main-theme.zip — making it easy to do overrides without replacing the entire resource package.

Two simple examples

It’s not hard to see where Resource Packages could be useful in existing sites, two obvious categories would be:

Supply the core layout & functionality of a site

Typically, you would ship over all the CSS, JS and images that are used on every page in the site. These could be cached quite aggressively, and even use ETags to invalidate the resource package when needed.

The thumbnail search result case

Consider a typical YouTube search results page. It contains 20-40 thumbnails of videos, and there’s no easy way to add all these images into an image sprite, since the long tail of search results would vary a lot. Resource Packages would let you build a zip of search result thumbnails on the fly, and ship them all over in one HTTP request. It would require some CPU power, but would be much faster for the end-user. This wouldn’t have to be cached, or could be cached on a per-search basis.

Other approaches

There are several other approaches that could solve parts of this problem, but they all have issues with current browsers and graceful degradation and/or are trying to solve a slightly different problem:

HTTP pipelining

This is a more aggressive way of utilizing the HTTP keep-alive mode, but is not implemented correctly by all web servers. Proxies have a hard time with it, some browsers also do, so it’s not really working unless you want to be aggressive and/or whitelist/blacklist certain servers.

It also causes a “head-of-line blocking” issue, where if you request X and Y on the same connection and X is slow, then Y is slowed too.

Multipart MIME

Hard for integrators, requires special packing that isn’t trivial to do, and has poor browser support.

JAR files or anything using data: URLs

No reasonable fallback mode, as the file name is embedded in the href/src link, and browsers that doesn’t support it just won’t render it.

SPDY

While this effort from Google aims to make everything faster, it is largely orthogonal to what we’re trying to do with Resource Packages. It also requires you to retrofit both web browsers and web servers to make it work, which means it will take quite a while before this will be in common use. Resource Packages work without any changes to the web server software, and will work as soon as any browser supports it — with no adverse effects to the browsers that don’t.

Additional notes

• The Zip format doesn’t have MIME type support, so this will have to be solved by the browser based on filename extensions or other heuristics. We don’t believe this to be a problem, since browsers already have to do this.
• All the resources in the package will have the same headers (expiry, ETags, etc.) as the resource package itself. If you need different expiry dates or other caching settings, you should specify multiple resource files with different cache headers.
• You can specify a charset in the resource package definition. If unspecified, it is assumed that any non-binary files inside are UTF-8.

Next steps

We have sent this out to the major browser vendors for feedback, and we will be implementing this in the next upcoming release of Firefox — which tentatively has the version number 3.7, but this may change.

FAQ

Does zipping up multiple optimized PNGs or other files work with zip? Can it potentially increase file size or lead to a high unpacking CPU overhead?

Zip automatically chooses the best of deflation or no compression. Images will usually not be compressed, since they already are, text files like CSS/JS will be. In general, CPU impact from unzipping is negligible, even on slow devices.

How does this affect mobile devices, which have limited CPUs?

More realistic concerns are cache ability and bandwidth — as well as the latency on mobile networks — and memory. A lot of mobile browsers only keeps things in the browser cache at all if the individual file is something like 20kB or less. For returning visitors, you suddenly need to download one large file again, instead of having multiple small files locally.

In general, mobile browsers clear their caches quite aggressively — although with the resource package spec, one would hope that they would implement more optimal handling and prioritize caching these, since they more likely to be valuable for browsing performance than another random image/CSS/JS file in a site.

How would Resource Packages work with CDNs?

There would be no special handling, these mirrors would just carry the resource package file like any other file they are supplying.

How would you manage priority in the resource package? It would be useful to decide which files get downloaded first.

The priority is managed by the order they are added to the resource package. If you want a specific order, it’s trivial to specify this on the command line, so we aren’t adding any special syntax for this. Also, we have to do it this way to take advantages of the ability to unpack and display resources while the file is still downloading.

I worry about reduced parallelism. Lots of sites make heavy use of resource sharding across many hostnames to take advantage of multiple connections. Won’t this be a problem?

Sites usually use multiple hostnames to get around per-host connection limits, which are almost entirely a latency issue — not a bandwidth issue. Resource packages make multiple hostnames unnecessary, because it solves the latency issue a different way.

Acknowledgments

• Mike Solomon from YouTube for encouraging me to propose a solution to this issue.
• David Baron & Elika Etemad from Mozilla for comments on the implementation feasibility, and for helping identify prior art.
• Vladimir Vukićević & Jonas Sicking from Mozilla for help with adapting the Offline Resources standard to handle Resource Packages.
• Dion Almaer & Ben Galbraith from Palm, Steve Souders, Gregg Tavares & Alex Russell from Google & the Chrome team for feedback on the proposal from an implementer’s perspective.
• Ben Mathews from Facebook for feedback on compression formats.
• Laurence Rowe from Jarn for suggestions on how to handle duplicates/overrides.

Improvements & feedback

If you have any suggestions on how to improve this proposal, send me an email at limi@mozilla.com or even better, comment in the open thread over at Mozilla’s dev.platform forum. It has been filed as bug #529208 in Bugzilla for those of you that want to monitor its progress.

Proposal State: Ready for prototype implementation

Revision 7: Feb 22^nd, 2010 — Added new information on compression formats, both zip and tar.gz files are now part of the spec.

Revision 6: Feb 12^th, 2010 — Added a defined behavior for what happens when resources are defined twice.

Revision 5: January 10^th, 2010 — added inline definition of resource package content in a manner that is compatible with HTML4/XHTML validators.

Revision 4: Nov 16^th, 2009 — first published & widely circulated version, added Offline Resources support

Revision 3: Nov 10^th, 2009

Revision 2: Sep 1^st, 2009

Revision 1: Jun 15^th, 2009

From the dept. of status update slacking, the two previous weeks:

Plone

• The new theme for Plone 4 was merged, along with all the other improvement branches that were proposed for this release. Impressive work by everyone involved in Plone 4! It’s looking to be a great release.
• Completed my keynote presentation. More than a week before I’m actually presenting. I know, crazy.
• Worked with Eric Steele and Israel Saeta Pérez to improve the roadmap on plone.org, which is now fully converted to Trac. Still needs better descriptions, but the hard part is done. Thanks, guys!
• Worked with Jon Stahl to identify some opportunities around improving plone.org and our evaluator experience. Looking forward to taking a crack at this with Joel Burton after the conference is over.

Goals for the coming week

• Finish my Plone Conference talk on advanced CSS layouts at Plone Conference 2009.
• Get Deco in shape to demo it at the conference keynote.
• Finish article on how to improve the social aspects of the conference for people that are new to the community.

Firefox

• Prioritized the UX work we want to land in Firefox 3.7. There’s now a list at the top of the UX index.
• Did some investigations on how Chrome handles its installer experience on Windows as part of making the experience better, similar to the proposed Mac installer improvements. Findings:
  • Chrome Frame requires admin privileges to install, Chrome itself does not. Counter-intuitively.
  • Tested the new Firefox installers on a couple of locked-down corporate computers, and it turns out you can install it in much the same way as Chrome does their install.
  • We should probably still try to make the install experience better on Windows, but doing the same as Chrome is probably not a good idea — their installer failed on several test computers.
• Worked with Alex Faaborg to get the first draft of site-centric preferences improvements out, he has posted them internally for now, but look out for them soon. Props to Alex for doing the legwork on the mock-ups.
• Was going to produce a screencast to demonstrate the effect of the prioritized tab loading in Firefox 3.6 with Paul O’Shannessy, but there were some loose ends to tie up first, so we agreed to do it early next week instead.
• Worked with Jinghua to get things ready for Taskfox user testing. We now have a consent form, and a rough outline of the upcoming user studies.
• Got closure on the remaining Taskfox issues — props to Blair for fixing the last remaining bugs that were in the way of user testing. There’s a new build you can (and should!) test.
• Did some investigation around the status of inline PDF display for 3.7. It’s 2009, we should really have this by now. There were patent issues in the past, but need to figure out the current situation.
• Had a great conversation with John Lilly about Firefox strategy, where UX should focus — as well as demoing some of the things we want to do in Firefox 3.7 & 4.0. Will talk about this in the UX team meeting this week.
• Finally got back to the Resource Package work again. Hoping to wrap up a version with the local storage manifest variant with Vlad this week.

Goals for the coming week

• Finish the third draft of the Resource Package spec, hopefully with input from Vlad, jst, bz.
• Continue refining the preferences redesign with Faaborg.
• Write up scripts for the user studies of Taskfox, hopefully with Jinghua’s valuable input.
• Work with the rest of the UX team on splitting our list of UX focus areas for Firefox 3.7 into more detailed requirements, and file bugs so it can be tracked.
• (Aspirational, 4.0) Work on the tab sidebar mock-ups.
• Wrap up the things that can be completed before I head to the Plone Conference in Budapest.

Other

• Falling down stairs on your back/shoulders/elbows should be avoided — or at least be drunk if you do so, it’s more embarrassing when you’re not. Ask me how I know. Ow.
• Educate world on glorious feeling of using en-dashes and hair spaces in headlines. Dashing!
• Hopefully getting together with Mike & Nicholas this week to save Mac window management from itself. Stay tuned.
• Around 400 people registered for this year’s Plone Conference. It’ll be awesome, can’t wait.

Firefox

• Published an updated article on the decisions for Mac installer improvements, and we already have two volunteers looking at the implementation. Gotta love our community.
• Published an article describing how we’ll do tab matching in the Location Bar for 3.7.
• Gave feedback on the first and second iterations of the Mac theme improvements for Firefox 3.7, expect Stephen Horlander to have a blog post about it soon.
• Updated the Projects list to be up to date with everything I’m currently working on.
• Did some process planning to improve longer-term UX vision, details to follow.
• Sketched out some ideas on the Prefs improvements.
• Gathered ideas around session restore improvements.

Goals for the coming week

(some carry-overs & and ongoing tasks from last week)

• Finish the third draft of the Resource Package spec, hopefully with input from Vlad, jst, bz.
• Get closure on next steps for Taskfox.
• Work with Stephen Horlander on refining the new Mac theme for 3.7 & 4.0.
• (Aspirational, 3.7) Start working on a first UI mock-up for how the prefs reorg could look like. Coordinate with the add-on manager redesign.
• (Aspirational, 4.0) Work on the tab sidebar mock-ups.
• (Research) Get data from the tab experiments from Test Pilot, analyze what they mean.

Plone

• Fixed up the remaining issues with the new theme for Plone 4, and submitted it to the framework team for review.
• Fixed most of the issues raised by Martin Aspeli.

Goals for the coming week

• Work with Eric Steele to improve the roadmap on plone.org.
• Prepare keynote & layout talk for Plone Conference 2009.
• Publish an article on how to improve the social aspects of the conference for people that are new to the community.
• Get back into Deco development with Rob Gietema.

Firefox

• Led a session on reorganizing the preferences window, posted the notes to the wiki.
• Looked into excessive CPU usage on multiple idle Gmail sessions, found to be a real issue in repaint + garbage collection, filed as bug 515215.
• Expanded the UX index with help from Boriss, added description and references for all the areas we’re looking at.
• Worked with Boriss and Faaborg to record material for a short screencast on the new UI in 3.7 and 4.0. Faaborg to do further edits and get back to us.
• Presented on “Improving the Download Manager” at Design Lunch with the goal of getting more input what people like and want to keep from the current approach. Great discussion with lots of new ideas, which were added to the wiki.
• Spoke to Chris Beard about next steps for Taskfox — we need to decide whether Labs should own it from here on and use the remaining bugs as a way to bootstrap, or whether Blair should just fix the minor issues so we can start some user testing. Expecting resolution sometime this week now that people are back from travels/vacations.
• Did more tests of Prioritized tab loading using zpao’s extension — help us test it! (More background info)
• Put up a blog post about improving the Mac install experience, got coverage at Daring Fireball, TUAW, OSNews, as well as the French MacGeneration.
• Helped clarify and direct the conversations around the proposed 3.7 and 4.0 design directions on the list (link)
• Published the video from the all-hands session on 3.7 & 4.0 design directions.

Goals for the coming week

• Put up a blog post describing how we’ll tab matching in the URL bar for 3.7
• Publish an updated article on the decisions for Mac installer improvements.
• Make sure the Projects list is up to date with everything I’m currently working on.
• Finish the third draft of the Resource Package spec, hopefully with input from Vlad, jst, bz.
• Get closure on next steps for Taskfox.
• Work with Stephen Horlander on refining the new Mac theme for 3.7 & 4.0.
• (Research) Get data from the tab experiments from Jetpack, analyze what they mean.
• (Research) Gather loose thoughts around session restore improvements.
• (Aspirational) Start working on a first UI mock-up for how the prefs reorg could look like. Coordinate with the add-on manager redesign.
• (Aspirational) Work on the tab sidebar mock-ups.

Plone

• Improved the new theme for the upcoming Plone 4 — one of the major changes outside of the actual CSS work was to revert to using XHTML for now, and do a proper HTML5 implementation for Plone 5 to minimize the impact to existing themes.
• Did some initial planning with Jon Stahl & Joel Burton for a sprint to improve the evaluator experience of Plone — aka. approachability (installing, getting started, etc).
• Dug up some old notes on how to build the next-generation LinguaPlone for multilingual content. Huge thanks to Jonathan Lewis for sending me his copy, since these had since disappeared from the OpenPlans/CoActivate site.

Goals for the coming week

• Prepare keynote & layout talk for Plone Conference 2009.
• Publish an article on how to improve the social aspects of the conference for people that are new to the community.
• More improvements to the Plone 4 theme based on Martin Aspeli’s feedback
• Get back into Deco development with Rob Gietema.

In short—

• The standard feed that you might be subscribed to at the moment will continue to show you all new content from this site.
• There are two new feeds dedicated to only showing content related to Plone and Firefox, respectively.
• Planet Plone will continue to carry all the content updates, unless I get asked to set it up otherwise — since there is a significant overlap in what is interesting to them and what’s going on with Firefox.
• I haven’t decided what to do with Planet Mozilla yet, since it seems to contain a wider range of posts on stuff that isn’t directly related to Mozilla and Firefox. Maybe I’ll keep it with the existing setup for a while unless I hear otherwise. Feel free to complain if this is causing too much off-topic banter.
• The new Firefox-only feed is being used at the recently created, and more focused, Planet Firefox. A recommended subscription if you want to keep tabs (hah!) on the future of your favorite browser.

I will also post weekly updates for both the Firefox and Plone work I do. I’ve been slacking on the Plone front in particular lately, mea culpa.

Keep calm, and carry on.

Wow.

I had no idea that so many people were even reading this site, and was again reminded just how amazing the communities around Plone and Google are.

During the last 28 hours after announcing that I am no longer with Google, I have received:

• Over 150 email messages from friends and strangers expressing support and asking what happened, and what I’m doing next.
• Over 30 emails from former Google team members, and people from other teams saying “WTF, dude — I heard about your story, this is not supposed to happen at a place like Google. That’s f—ed up.”
• Four offers of job interviews with major companies in Silicon Valley — not counting the startups I have never heard of.
• Requests from five journalists from sites and magazines I have heard of asking me for comments. And no, I don’t think it's appropriate for me to discuss this with the media, even anonymously — but thanks for asking, you’ve all been very polite and respectful.
• Two invitations to have lunch at Google. I appreciate the humor in the middle of all this, guys.
• Countless IMs, IRC conversations, Facebook messages, and phone calls checking that I’m OK, that I can stay in the US, et cetera.

I’m humbled, and not just a little bit touched by all the support and attention I have received. I’m doing fine, I’m staying in San Francisco for the foreseeable future, and I have a lot of plans and options at the moment, and I’m excited to move on.

A special thank you to the people from the Google User Experience team for inviting me to their gathering in the Mission yesterday, discussing what happened, and what they can do to make sure it doesn’t happen again to someone else in the organization. You’re all amazing people, and I miss working with you already.

Unless something very unexpected happens, this will be the last message about my former Google employment to this site. Thanks for all the support, and if you have any questions, I’m happy to answer them.

For the past couple of years, I have been working as an interaction designer at Google in Mountain View, California.

Because of manager conflicts, I am no longer with Google. Their handling of the matter has been no less than astoundingly amateurish and highly unprofessional, but I won’t air their dirty laundry here — it’s not worth any more of my time.

Suffice to say, I’ll be surprised if they manage to hold on to their top talent in the User Experience division for more than a year or two. That part of the organization is slowly rotting from the top down, and people have already started leaving. The wasted potential is truly beyond anything I have ever seen in an organization that is otherwise really well run, and has its priorities straight.

I have had the pleasure to work with an astounding number of amazingly talented individuals at Google, and I will miss them dearly. And there are some truly great managers at Google — hopefully I have let them know — I just never got to work with them. To all the designers and researchers I worked with at Google: I really enjoyed meeting and working with you all, you taught me humility, data-driven decision making and overall greatness — please stay in touch. To all the managers that attempted to help resolve the difficult situation I was in: Thanks for the respect, kindness and energy to fight the good fight. Keep it up.

Google is a fundamentally sound company, they have their priorities straight, and they will do well in the foreseeable future. I had a great time there, but it’s time to move on.

Hello, Plone

So, what’s next? Less commuting. More time with friends. Taking care of my ill father. More sunny days outside. More music, travel, sports cars, and guitars. And, of course, Plone.

With the work going into the upcoming Plone 4 release, I’m more excited about the future than I’ve ever been. Some of the best developers I have ever worked with — including Google engineers — are working on the next-generation Plone, and it’s truly in a position to change the world of content management.

To me, this is the most exciting year in my life. I’m looking forward to spend my time doing what I love, work with the people I admire, and to live and breathe the best open source community on the planet. It’s like coming home.

Now, if you’ll excuse me, I have a week of sleep to catch up on. Let’s change the world, again.

We’ll pick as many as possible of the highest ranked questions and answer them at the end of the keynote. Now, go ahead and ask those questions you always wanted to have answered.

Ask your questions here!

Known from TurboGears — and more recently, his keynote What Django can learn from Zope — Mark Ramm was invited to the Plone Conference to discuss the Python ecosystem and how to expand reuse and collaboration between the different Python web frameworks.

I remember meeting Mark for the first time when Alan Runyan and I delivered the keynote at the Python Conference 2006 in Dallas. We had several great conversations during the conference, but our paths haven’t crossed since — mostly because I’m not really a Python coder, so we don’t attend the same conferences.

I’m looking forward to meeting him again, and I believe he has a valuable perspective on Plone, Zope, Repoze and WSGI that should be enlightening and interesting.

As usual when it comes to conferences, it’s the conversations over a beer after the day’s program is over that really count. We’re excited to welcome you to our conference, Mark!

If you’re in South America — especially if you’re in Brazil — I encourage you to check out the program for the conference.

My talk will be “The State of Plone and Lessons from Managing a Top 2% Open Source Project” — and will happen at 2:40pm on Friday Sep 19.

I’m also looking forward to meeting fellow keynote speaker Bruce Eckel, long-time Python advocate and author of the books “Thinking in Java” and “Thinking in Python.”

With the Plone Conference 2008 coming up on Oct 8–10, it’ll be a busy few months — but very exciting and fun.

While talking to Hanno Schlichting earlier today about his fantastic efforts at fixing bugs and cleaning the Plone issue tracker — combined, the two of us have managed to resolve 250 bugs from the Plone 3.x maintenance release in the space of a few days — Hanno mentioned that he’d finally managed to unplug his TV for about two weeks after not having a TV for most of his life until the recent last few months.

It reminded me about the following excerpt from Clay Shirky’s speech on cognitive surplus — he’s talking to a TV reporter about his new book, and she asks him about Wikipedia:

She shook her head and said, ‘Where do people find the time?’ That was her question. And I just kind of snapped. And I said, ‘No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you’ve been masking for 50 years.’ So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project — every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in — that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it’s a back-of-the-envelope calculation, but it’s the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that’s 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, ‘Where do they find the time?’ when they’re looking at things like Wikipedia don’t understand how tiny that entire project is, as a carve-out of this asset that’s finally being dragged into what Tim calls an architecture of participation.

Now, the interesting thing about a surplus like that is that society doesn’t know what to do with it at first — hence the gin, hence the sitcoms. Because if people knew what to do with a surplus with reference to the existing social institutions, then it wouldn’t be a surplus, would it? It’s precisely when no one has any idea how to deploy something that people have to start experimenting with it, in order for the surplus to get integrated, and the course of that integration can transform society.

The early phase for taking advantage of this cognitive surplus, the phase I think we’re still in, is all special cases. The physics of participation is much more like the physics of weather than it is like the physics of gravity. We know all the forces that combine to make these kinds of things work: there’s an interesting community over here, there’s an interesting sharing model over there, those people are collaborating on open source software. But despite knowing the inputs, we can’t predict the outputs yet because there’s so much complexity.

I’ve been television free since I was 18 years old and moved to my own apartment. That’s part of the reason why something like Plone could happen, a 250+ company worldwide phenomenon. Hanno got rid of his TV when he was 20. He's the most prolific contributor to Plone worldwide.

Get rid of your TV. Change the world, don’t just sit and watch.

…or, as Hanno said: “If you want to be the best — act accordingly.”

In the articles Simplifying the content authoring experience, Improved handling of rich media and Composite pages, listings and content proxies, we have laid out an approach to radically simplify the content authoring experience in Plone.

We have in one fell swoop eliminated the need for:

• Collections as a separate type,
• Images as a separate type,
• Composite page types,
• Proxy (represent one object in two locations) type,
• Default pages,
• The need for a separate "Contents" tab,
• Folder as a separate type.

We have made it substantially easier to handle:

• Rich (“opaque”) media like video, audio and Flash,
• Content reuse, i.e. generating new content by reusing other content (example: book with 3 different audiences),
• Application-specific functionality like forms/polls/maps inside a page.

We have made it possible to:

• Share “snippets of Plone” (widgets) to other systems — blogs, iGoogle, Facebook — you name it.

…all while making the UI massively simpler and easier to work with.

And if that wasn’t enough, the infrastructure improvements can be done gradually over a number of small releases in the 3.x series, and then culminate in a Plone 4.0 or 5.0 where we make the entire picture come together in a coherent new user experience. And we mostly don’t need to invent new infrastructure, instead we can build on work done in the Plone 3.x releases.

Next Steps

Where do we go from here? I want you all to help with fleshing out the plan, as well as contributing code and testing resources to make this happen. The next steps will be:

• Identify which individual components are needed,
• Figure out which of them can be built in parallel, and which ones can potentially be included in the 3.x releases,
• Identify champions for the different areas,
• Schedule sprints for big-picture integration sessions.

We hope to have the first of these sprints at the Plone Conference 2008 — and I hope we’ll see you there! Everyone is welcome this time around, and we’re trying to set up a range of sprint tasks, so that developers of all skill levels can participate in different ways.

I hope you will join us in taking Plone to the next level!