Progress Report: 13 April 2009

[denise]

Okay, let's just go with by-the-numbers for the intro:

Days to open beta launch: 17
Bugs blocking open beta without a patch: 17
Total open bugs: 319
Total resolved bugs: 498
Bugs resolved this week: 58

1. THE PEOPLE THING

So as many of you have noticed, we've been expanding our closed beta this week significantly, as we try to gather information about load figures so we know what kind of hardware build we're going to need at launch. In order to do that, we've been handing out invites in a bunch of ways: to random people who are subscribed to our mailing lists, to random people who have set and validated an email address on their OpenID account, to existing users for them to hand out, etc, etc.

We'll be continuing doing that this week as we continue to stress-test the system. We're up to over a thousand personal accounts, and our load graphs are starting to look like an actual active site's, complete with peaks and troughs. (In the past 24 hours, for instance, we've averaged nearly 1Mbps of traffic and peaked at 1.74Mbps -- and for the week, our average traffic was 1.06Mbps and our peak was 2Mbps. You can safely ignore this if you don't know what that means.) We're happy to report that we haven't -- knock wood -- had any major load-related issues yet, even given the fact that we don't have all of our production hardware set up yet. This is encouraging!

We're sending out between 50-75 invites a day as we try to push that traffic upwards. Most days, we're using the "validated OpenID accounts" method of distributing invites, and we're choosing from that set randomly every day.

2. THE INVITE THING

I do have to stress that we are still in closed beta. This means a few things:

• Not everything is done yet. Most notably, the journal import function still has a few glitches, the automatic crossposting system isn't finished yet, and the style system is not fully complete. All of this will be done by our open-beta launch date on April 30.
  
  
• Our payment system is not yet finished. This means that you can't give us money, either to create an account or to pay for your already-created account.
  
  
• We are being exceedingly careful about how many invites we give to people to hand out. We want to make very, very sure that we have the resources to support the amount of usage we're getting, and the best way for us to do that is to be very cautious for the next three weeks about how many accounts we give out per day.

We know that a lot of people are frustrated by having to wait for an invite, or are feeling excluded by the fact that their friends have been able to create accounts and haven't been able to invite them. This sucks, and we know it. We absolutely don't intend for invite codes to be elitist or exclusionary; they're a means by which we can control our growth to make sure that the site is stable and reliable for all users, to make sure that we aren't growing faster than we can support, and to make sure that we don't wind up overrun with spam accounts.

You can see some of our thoughts and reasoning about invite codes, why we're using them, and how we plan on making sure that they enhance the community experience rather than detracting from it here:

Account Types and Costs

Rest assured, as soon as April 30 rolls around, everyone with a Dreamwidth account will have a ton of invite codes to give out, everyone who's subscribed to one of our mailing lists or has set and validated an email address on their OpenID account will get an invite code that they can use if they want to, and if you absolutely can't find an invite code to use, you'll be able to create a paid account for as little as $3 for one month (and, if you choose not to keep paying us after that, the account will continue to exist as a free account after it expires).

3. THE BUSINESS THING

We've also seen some confusion about our business model. Specifically, we've seen people confusing us with the Organization for Transformative Works (OTW), which is a nonprofit organization. While many of our volunteers do also volunteer with the OTW (to the best of anyone's knowledge, we are the only two majority-female Open Source projects on the Internet, and both of our organizations are really committed to training and mentoring our developers), Dreamwidth Studios, LLC, is not formally affiliated or associated with the OTW.

Dreamwidth Studios, LLC, the company that owns and operates both dreamwidth.org the hosted service and the Dreamwidth Studios open-source codebase, is a Maryland-based Limited Liability Company wholly owned by myself and Mark Smith. You can read our Operating Agreement, the document we filed with the state of Maryland to form our company, here:

Dreamwidth Studios, LLC Operating Agreement

We are not a nonprofit or a member-owned collective. (We thought about organizing as a nonprofit or a member-owned collective at the very beginning, but there were a few things that made us decide against doing it that way. We do eventually plan on creating a nonprofit organization to own the code, while the for-profit corporation continues to operate the hosted service, much like how the Mozilla Foundation and the Mozilla Corporation work; that's a long-term goal, though.) You can read some more about our operating structure, along with what we plan on doing with any profit that Dreamwidth makes, here:

Operating Structure

Because we are not a nonprofit organization that runs on member donations, and because we are absolutely committed to creating and maintaining a sustainable business without ever accepting third-party advertising, we will rely on subscription fees from our users in order to subsidize the service for everyone. In exchange for those payments, we'll offer our paid users extra features -- both the features that cost us extra money to offer (because if something costs us money to offer, we'll reserve it for the people who have paid us money to support it), and features that are really shiny (as a form of saying thank-you). This week, as part of our efforts to share with you some of our business reasoning, I posted a list of what features will be available to paid users, both at open beta launch and the features we hope to add within the first year or so of our operation:

Paid Accounts

(As a side note, the dw_biz community is your place to go if you have a question about the business end of things. We'll be regularly posting about the process of running the business as a business, and we've got a Q&A thread running right now that we'll post a summary of later on this week.)

4. THE COMMUNITY THING

As people are moving into our closed beta, communities are not far behind! This week we've been expanding our "official" Dreamwidth communities -- which we will be posting about, writing better descriptions for, and adding to the site map before we hit open beta -- and our users are creating communities of their own that you might be interested in as well.

For an official way to browse through nifty communities and/or promote your new community, check out the Dreamwidth community promotion comm:

dw_community_promo

It has a sister community for syndicated feeds, at:

dw_feed_promo

Some Dreamwidth-specific unofficial communities that might be of interest:

getting_started: An unofficial community for sharing information and resources about Dreamwidth.
dreamchasers: Find your friends from another service, or make new friends on Dreamwidth.
dwrocks: For things you really love about DW. (Also includes an unofficial slogan contest.)
dwstones: For things about DW that you think could be better.

None of those communities are official, but they're all Dreamwidth-related in some way.

There's also been an effort to maintain a "community directory" of sorts on the Wiki, for people who are looking for communities (until we can get better community-search and community-directory tools written). So, if you're looking for something to match your interests, or if you have a community you want to add, check it out:

Community List

5. THE DOCUMENTATION THING

rho, our documentation admin, has asked to include some information on our documentation efforts, so I shall turn the update over to her:

For those of you who don't know me, I'm rho and I'm heading up our documentation project, which includes all our FAQs and guides as well as all the text on main site pages. I asked denise if I could have a few paragraphs of space here to let you all know what's going on with documentation and answer a few questions that I've been getting.

* FAQs, Guides, and Knowledge Bases

We're going to be having three different types of documentation on Dreamwidth: FAQs, Guides and Knowledge Bases. The idea behind the split is that the different types will be able to serve different problems. The FAQs will be simple answers to simple questions like "what are tags?" or "how do I post a poll?"

The guides will be more in depth, taking a given topic and discussing it in some detail. For instance:

"How do I protect my privacy on Dreamwidth?"

Then, the Knowledge Bases will be more like a complete technical manual. This is where we'll put all the obscure technical bits that we want to document somewhere but that don't really belong in our FAQs and Guides.

By splitting things up like this, we want to make things easier to find. If you're asking how to create a poll, you probably just want a link to the Poll Creator, so that's what the FAQ will give you. You probably don't care about how to write out the code for a poll manually, or just how many characters of text you're allowed to have in a poll question, so we won't make wade through that to find the answer you were looking for. Instead, we'll put that information in a Knowledge Base. It'll be there for the people who want it, but it won't get in the way for everyone else.

* The FAQ isn't finished yet!

As we're bringing more and more people into closed beta, I'm seeing more people talking about how something or other isn't documented yet, or how some FAQ is incomplete and ought to have some bit of information or other added to it. Mainly this is just because we haven't finished writing it yet! If you see something that isn't documented anywhere yet, then we almost certainly know that it isn't documented yet and most probably have someone working on it. Please do bear with us just a little while longer. By the time we reach open beta, we're hoping to have almost all the FAQs and a select handful of guides up and ready, and we'll be carrying on with adding more and more stuff as we proceed.

That said, though, the FAQ team have been kicking some serious behind, and we have written up a lot of stuff already. If you have a question, you could do worse than start out by checking the FAQ.

* Terminology

I've seen a few people wondering about some of the new terminology that we're using on Dreamwidth -- things like Subscribing and Access, for instance. We know that the choices that we've made aren't going to suit everyone. We spent a lot of time brainstorming about what to call things, accepted that there weren't any possibilities that didn't have at least a few issues with them, and went with what we figured were the best options. If you're lost, rest assured that we'll have a "Dreamwidth for LiveJournal users" Guide before Open Beta, which will include a terminology section.

We're also planning on revisiting terminology in about 6 months to a year, after we see what terms people start using in the wild.

* Site text

The really exciting news from my end this week is that we've finished updating all the main site text! That's over 3700 lines of text in total, and we've been through all of them and updated them for clarity, style, and terminology. This was a truly mammoth task, and I really can't express just how proud I am of my team for getting it finished almost 3 weeks ahead of deadline. Of course, we aren't going to be resting on our laurels. Now that we've done that, the cruel slave-driver (i.e. me) has directed the team to go back through everything all over again but using an even finer toothed comb this time. We're confident that our updated version of the text is good. We want to make it even better.

Of course, it's always possible that one or two errors have slipped through. If you spot any typos or any pages still talking about Friends Lists or LiveJournal, for instance, please do let us know!

* Getting Involved

A few people are also asking me what they can do to get involved with documentation or proof reading. At the moment, we aren't looking to bring in any extra people, but we will be doing after Open Beta. If you're interested, subscribe to the dw_docs community. In particular, you should read a post I made the other day about getting more people involved:

Bringing in more people

Thanks for listening. I'll now hand you back to Denise for your regularly scheduled wall of text!

6. THE SITE COPY THING

And now that it's me again, I have to just reiterate something rho said in her point up there: the site copy team really has done an amazing amount of work in an incredibly short period of time. Not only were they rewriting everything, and doing so through an administrative system that was not designed with usability in mind, they've also been keeping track of every instance where we needed to make a code update to pass more valuable information into the translation system. And now they're going back and doing it all over again, concentrating on things like readability and clarity, as well as making sure that we haven't used any visio-spatial language (language that automatically assumes our "default" is aimed at a sighted user accessing the site through a graphical browser).

I'd also like to thank zvi for calling the visio-spatial language problem to our attention, and ivorygates for taking point on assembling information and guidelines on how we can avoid it.

7. THE ACCESSIBILITY THING

And, speaking of accessibility, we have some exciting news there, as well. This week, our accessibility coordinator rb proposed that Dreamwidth support WAI-ARIA landmark roles, which is a way to describe parts of the site in such a fashion that assistive technology programs can understand which part of the page is the bit they want: "this part of the web page is the banner", "this part is the main text", "this part is the menu", etc. This is a relatively-new standard, but many browsers are just starting to support it, and we want to make sure that we're in on the ground floor:

WAI-ARIA spec

jadelennox, also part of the accessibility team, is working with rb to take the technical spec and implement it on Dreamwidth. (jadelennox is the programmer who's taking point on a lot of our accessibility efforts, including the effort to make sure that people can define meaningful and useful ALT text for all userpics.) I'm so excited that we're doing this, you have no idea.

8. THE PROGRAMMING THING

As I mentioned earlier, we resolved 58 bugs this week, including first-time contributions from baggyeyes, cesy, dani_the_girl, and piranha. There are also a number of first-time contributors who have patches waiting for review, or people who have just gotten started with a hosted development environment and are learning with us. Welcome to the development team, folks!

As part of our commitment to training new contributors, we opened up a companion community to our main development community, dw_dev:

dw_dev_training

This is the place to go when you're just starting out if dw_dev is a bit too confusing for you. We'll be adding more resources and tutorials as time goes on.

Meanwhile, we've posted about our development priorities for the rest of the year, along with our milestones list and the goals we'd like to complete by each one:

Development Roadmap

Our "code freeze" for open beta -- the date past which we will not check in any patches for anything but open-beta-blocking items -- is April 15. This doesn't mean that if you submit a patch past April 15, it won't get reviewed and checked in. It just means that it won't get reviewed and checked in until after April 30 unless we feel that the issue is critical enough to be included in our open-beta milestone.

We're also pleased to announce that this week, we finally completed our task of removing all advertising-related code from the codebase. (We'd been doing it a bit at a time, but mark finally got fed up with having to work around it.) I hope you'll all join me in doing a little dance of joy that all advertising support is officially and completely gone, gone, gone.

9. THE SYSADMIN THING

Poor mark spent a whole week saying that he'd get to coding tomorrow, and then tomorrow would come around and he'd have to do something else sysadmin-related. However, we had a very exciting sysadmin week as a result. As part of our efforts to get our production setup all ready for people to move in, we deployed a number of backend helper systems in order to finish our production build. This includes:

• SSL: We finished setting up our SSL (secure sockets layer) configuration, so people can (for instance) log into the site at the secure login page and make secure payments (once we get the payment system finished). While your password is never sent unencrypted even if you don't log in on the SSL page -- we use a Javascript-based challenge/response system with a one-way temporary hash of your password, even on non-SSL login forms -- this will help us continue to protect your personal data.
  
  
• MogileFS: MogileFS (an anagram of "omg files") is a distributed filesystem that makes it easy for us to store and access files such as userpics. We switched over all of our userpics to being served by MogileFS this week, which will give us more control about how we store and handle them.
  
  
• Gearman: Gearman (an anagram of "manager") is a "worker" system that lets us assign tasks around our network of machines, so heavy tasks can be done by machines that aren't as busy at the moment. It lets us run computationally heavy tasks in a more sensible way.

As always, all of our ops configuration files are available in the dw-ops code repository, for people running other sites to examine and use as they see fit:

dw-ops on Mercurial

10. THE PAYMENTS THING

mark did get to do some coding, though; he and janinedog spent a bunch of time working on our payment system, both the backend (the part that does the work of processing the payments, applying the paid time, expiring paid accounts that have chosen not to renew, etc) and the frontend (the "shop" pages that will show to people who are trying to make a payment. It's still not finished -- the actual part that processes the payment isn't ready yet -- but we're getting closer and closer.

When we launch on April 30, you'll be able to make a payment for an existing Dreamwidth account or pay us to create a new account. You'll also be able to buy paid time for a friend, if you know their Dreamwidth username (if they have one) or if you know their email address (if you're paying for them to create an account). We'll be expanding the payment system shortly after open-beta launch to include easier ways of sending gifts to your friends.

We'll also be implementing a credit-type system: instead of buying single-use gift certificates that have to be used all at once and forefit any unused balance if it's applied to a payment, you'll be able to buy Dreamwidth credits or points or whatever we wind up calling them, and then apply those credits/points to your purchase. That way, you'll be able to (for instance) buy $10 worth of credits/points and use them $1 at a time.

11. THE STYLES THING

Our Styles team has been burning the midnight oil this week, and we're getting closer and closer to the pretty!

This week afuna did a lot of work on the customization wizard, getting the major properties into the wizard (and getting them logically organized). This is part of our end goal of making it possible for you to customize your journal's look and feel without having to know anything about the S2 programming language, and only knowing minimal amounts of HTML and CSS.

Individual members of the styles team have been working, meanwhile, on porting our styles to the new core2 system, and we hope to have several additional styles checked into the codebase, and live on the site shortly thereafter, this week. Once this happens, we'll be looking to designers to make pretty themes for each of the styles!

If you're interested in doing styles work, check out the dw_styles community.

12. THE OVERWHELMED THING

As you can all imagine, mark and I have been receiving an incredible number of comments, emails, private messages, smoke signals, and other contacts. We love hearing your feedback, thoughts, and suggestions for Dreamwidth, but there are only so many hours in our day!

We know that we don't have a very good process set up for contacting us right now, and that's because we're still getting all of our contact methods into place. In the meantime, we're asking that people use comments to dw_news posts as the preferred method of contacting us. Please don't rely on email, the site private messaging function, comments to our journals, or Support requests; we can't guarantee we'll get to those in any reasonable time frame. (The one exception to this: if you're having problems with the journal import function, in that case we do ask that you open a support request, in the Importer category, so we can track common issues.)

We love you all, and we really love hearing what you have to say about Dreamwidth, whether that's a "yay Dreamwidth!", a problem that you're having, something you think isn't done right, or something you'd like to see in the future. We just can't guarantee that we'll be able to respond to everyone as we start picking up steam.

13. THE 411 THING

If you're looking for information on what's going on with Dreamwidth, we're already using a few of our official news channels. In particular:

dw_news: The place where we post important site announcements and our weekly progress. Look here first.

dw_biz: The place where we post about our business plans and development goals.

dw_maintenance: The place where we post about site-wide maintenance issues or downtime.

Dreamwidth on Twitter: Semi-regular updates on what we're up to, including announcements of downtime.

For more real-time discussion, commentary, brainstorming, and the ability to consign things to hell -- or at least our irc bot "hEll" -- come join us in irc:

irc.dwscoalition.org, port 6667, channel #dw


Questions? Comments? Things we really should know? Comment here, using your Dreamwidth account or via OpenID.

We'll see you next week for our next update!