![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png){kind=small}
![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png){kind=small}
Entry tags:
Progress Report: 9 March 2009
Oh, God, it's Monday again. I distinctly do not remember authorizing the repeat of Monday. (Or at least it was Monday when I started this update. It might not be Monday in my time zone once I finish it.)
This week we provide our update in slightly different form. Because the majority of our work this past week has gone to development (we're about a month away from when we really want to launch to open beta!), this week our progress report is going to be a listing off of all the bugs that we resolved, with what they were intended to fix and why we think that's important to fix. I'll intersperse it with additional clarification on bugs that I have a lot to say about.
I'm not only doing this because the majority of our effors this week were code-related; I'm doing this because some feedback we've heard is that the development process is rather opaque, and people don't know what changes are being made. So I hope that you will find this guided tour through this week's development efforts of interest. If nothing else, it'll be a good look into our thought processes as we go. We'll be back to our usual update format next week.
We resolved 75 bugs this week; 66 of them were actual fixes or patches, as opposed to things we decided we didn't need to fix, things we decided to defer until later, things that were duplicates of other items, reported issues that we couldn't reproduce using the same steps that the tester used, etc. One of those patches was a very, very tiny privacy-related fix that we decline to describe to this mailing list, because we'd like to give LiveJournal a chance to apply the patch first.
To the videotape.
1. "Old character encoding" no longer needed: patch by Ryan (bug 108)
CLEANUP: This patch removes old backwards compatability functionality from the LJ codebase. Because we're starting off with all of our data in the UTF-8 text encoding, we don't need to let people convert other text encodings to UTF-8. Eliminating unnecessary functionality = cleaner code = happier Dreamwidth! Also of note: this was Ryan's first patch to us. Thanks for joining our code team, Ryan!
2. "Insert image" popup in HTML editor broken (bug 308)
BUGFIX: This patch fixes the display of the "insert image" popup box on update.bml. It was broken due to interaction problems with our site scheme. Now it works! Patch by yours truly, but Afuna figured out the problem; I just typed it in.
3. <dt> in /site/index.bml needs to be wrapped in <dl> (bug 436)
CLEANUP: This patch corrects some missing HTML tags on the Site Map. Patch by yours truly, who wrote the original page and often forgets the little niceties of HTML.
4. Add sysban to Admin index (bug 461)
ADMIN, USABILITY: Adds a link to the sysban tool -- one of our administrative tools that will help us block and filter abusive uses of the site -- to the page with all the other admin tools. Patch by yours truly.
5. Add table of contents to faq.bml (bug 427)
DOCUMENTATION, USABILITY: To make it easier for users to find the specific FAQ category they're looking for, this patch adds a very basic table of contents to the top of the page containing links to all of our FAQs. Patch by Rho, our fabulous doc admin.
*
COMMERCIAL BREAK: You may notice, if you click through to the FAQ page on Dreamwidth itself, that our FAQ team is starting to fill out the contents of certain FAQs. This is because our FAQ team is made of 100% scientific-grade Awesome.
You may also notice that the resulting FAQs occasionally have little notes in them saying things like "need to check this" and the like. This is either because I am made of less Awesome than the FAQ team, and have not yet answered some of their questions, or because we haven't actually fixed the behavior that they're documenting yet.
The FAQ team is still made of Awesome, though.
*
6. Allow self-messaging (bug 463)
USABILITY: So, true story: I was trying to test something on my development environment, and I needed a private message in my Inbox to test on. Logically speaking, of course I log in and go to send a message to myself. Imagine my surprise when I get an error message! (Imagine my surprise when that error message says "Stop trying to message yourself, we're not that kind of service.") So as soon as I finished boggling, I submitted this patch to take out that error check and allow people to message themselves. I guess we *are* that kind of site?
7. Change data-gathering so that "gender" is only recorded (and recordable) for accounts of type P (personal account) (bug 20)
CLEANUP: This corrects our internal statistics so that the only account types that can specify (or decline to specify) their gender on the Edit Profile page are Personal and OpenID accounts. This will give us a much cleaner picture of our demographics; otherwise, community and syndicated feeds would be lumped into the numbers and skew our picture of our userbase. (This data is one of the many things we'll use to make intelligent decisions about the site's future development efforts.) Patch by Dre.
8. Configure login rate limiting (bug 455)
ADMIN: This patch sets up the login rate limiting for our service, which governs the number of failed logins you can have in a certain window. This helps to prevent people from trying to brute-force or dictionary-attack a specific account, trying passwords over and over again until one works. I'll confess, we kinda picked the number arbitrarily; we'll tweak the settings if legit use winds up tripping it too often. Patch by Janine.
9. Configure Thread Expander (bug 459)
PAID USERS, USABILITY: This sets up the "expand all comments in a thread" JavaScript for paid/premium/seed accounts. (The functionality was already there, we just hadn't specifically enabled it yet.) Right now, this just replicates the way the function works on LJ, but oh, do we have plans for this. Patch by Afuna.
10. Configure video embedding subdomain for production (bug 423)
CONFIG: For security reasons, video embeds (such as YouTube videos, Flash animations, etc) are served from a different subdomain, in order to isolate the scripting involved. This is so that in case someone manages to sneak a malicious embedded object past our code sentries, it's still limited in the damage it can potentially do. Config change by Mark.
*
COMMERCIAL BREAK: Mark did lots and lots and lots of things this week that were config-related, hardware-related, setup-related, or otherwise Not Appearing In This Update. I figure that if we could just clone Mark, we'd be all set.
*
11. Correct stylistic flaw in doc/template.bml.txt (bug 444)
CLEANUP: Now, look, I love Mark. We all love Mark. But we all also have to accept the simple fact ... Mark is as fanatical about certain matters of programming style as I am about those Russell Stover pectin jellybeans that only appear in the drugstores around Easter. We try not to hold it against him. This patch corrects a stylistic error in the file we provide for programmers as an example of how to build a user-facing webpage using our templating system. Patch by yours truly. After Mark gently chided me for the style error that I had just copied from the old template.
12. Create LJ::User is_individual function (392)
ADMIN, CLEANUP: This is a backend patch that defines a function to identify if the user being referenced is *either* a personal journal account or an OpenID identity account. This has the potential to simplify a bunch of redundant calls all over the codebase; we'll be slowly moving the old code over to using the new function. Patch by Afuna. Who then realized she'd forgotten to put it in alphabetical order after I spent four hours last week organizing that whole nine-thousand-line file, and promptly fixed it before I had to give her the sad puppy dog eyes.
13. Create site scheme look files from Photoshop mockups (bug 104)
USABILITY, LOOK & FEEL: This was the bug that had been open for a while for people to work on creating the site scheme template files from the mockups of the design (designed for us by the ever-fabulous grrliz). We know we've still got a few wrinkles to iron out, but since the functionality is now all there, we closed this bug in favor of opening new ones to fix the outstanding issues. Patches by Jesse and Janine.
14. Create support categories on dreamwidth.org (bug 227)
ADMIN, CONFIG: Not that we're ready to start accepting support requests yet -- far from it; all feedback to us should still use the same channels you've been using all along -- but this item was to configure the categories for when we are ready, and it was completed by the long-suffering Mark.
15. creating filters with invalid names (bug 367)
USABILITY, CLEANUP: When Mark first did the watching/trusting friend split, and reimplemented custom access filters that you can lock your posts to, he was insanely strict about what characters could be used in filter names. (Enforcing strictness when dealing with user data is one of many, many ways you can help avoid introducing security holes.) Turns out that it doesn't need to be that strict, so we just went with the same rules that the LiveJournal codebase uses for what's permitted in filter names -- which has the added benefit of not breaking cross-site compatability. Fix by Mark.
*
COMMERCIAL BREAK: I wanted to actually take a second here and specifically give a shout-out to Mark for the watching/trusting friends split, which is an incredibly involved change that he's been incredibly diligent about not only updating, but *documenting.* (Mark is one of those true rarities of nature: a programmer who writes fantastic documentation.) I also wanted to acknowledge Janine, who has done a truly heroic amount of work to find all of the places on the site that used to use friend-related code functions and update them to us the new watching/trusting code functions. They've done incredible work on the system.
*
16. Define and implement new user defaults (bug 372)
USABILITY: Patch to set all of the defaults for a newly-created account -- users can change these at any time, but we pre-populate the account with what we think are some sane defaults. Patch by Janine.
17. Document all user caps (bug 458)
CLEANUP: In his house at R'lyeh Great Cthulhu lies dreaming. In the code at Dreamwidth, great mysteries lie sleeping. Right now, as-is, the whole model of what users can and can't do, what features they can use, what features are on and off for the site as a whole, is governed by an arcane, esoteric, undocumented set of capabilities and flags, which can only be turned on and off by listing them in the textfile configuration files. We thought it would be a lot easier to set these capabilities and flags, not to mention deciding what the settings should be, if there were actually a complete list of what they *were*. Anywhere. At all. Thanks to Foxfirefey, now there is. Thanks for giving us a bonus to our SAN roll, Fey.
18-22. English-strip various files (bugs 382, 260, 409, 385, 380)
CLEANUP, DOCUMENTATION: 'English-stripping' refers to the process of removing the hardcoded English text on the site and instead putting it into the translation system, where the site copy team will be able to change it on-the-fly and where other sites using the Dreamwidth code will be able to translate the site into other languages. These bugs fix up some places that had not already been English-stripped in: dreamwidth.org/openid/, dreamwidth.org/editpics.bml, dreamwidth.org/stats.bml, dreamwidth.org/tools/recent_comments.bml, and dreamwidth.org/manage/profile/. Patches by me, Sophie, and Isabeau, and a special call-out to Isabeau's first (and second!) patch commit. Thanks for joining the dev team, Isabeau!
*
COMMERCIAL BREAK / PUBLIC SERVICE ANNOUNCEMENT: We won't be offering DW in other languages, for a lot of reasons, but mostly involving the fact that the translation system is sub-optimally usable. However, I come bearing a fun little trick: if you ever want to see whether or not a page (on Dreamwidth, LiveJournal, or any LJ-based clone site running code from any time past mid-2002 or so) has been English-stripped, add ?uselang=debug to the end of the URL, marvel at all the variables that suddenly appear, and look for any English text. (We've got bugs open to fully complete all of the English-stripping we still need to do, so over the next few weeks we'll be getting rid of all of those.)
*
23. Error handling for WTF backend (bug 368)
CLEANUP: As part of our split of 'friends' into watching vs trusting, we needed to gracefully handle error conditions if somebody tried to do something that didn't make sense. Now we do! Patch by Mark, who I bet is really tired of looking at friend-related code by now.
24. File inserts overwritten on code push (357)
ADMIN, BUGFIX: Man, just the thought of explaining this one makes me feel like I have to lie down. Suffice it to say that it's another case of something that was designed for the short-term quick fix sticking around way longer than it was originally intended to. Mark sprinkled the appropriate magic pixie dust over the right sections of the config until we can update the whole system to do it differently.
25. Fix dumpsql.pl (bug 466)
CLEANUP: This is a fix to a backend database-related script to remove references to outdated perl libraries we don't use anymore. Patch by Pauamma.
26. Fix incorrect comment in cgi-bin/ljfeed.pl (bug 440)
CLEANUP: Ever heard the saying "if the code and the comments disagree, they are both probably wrong?" This patch makes the code and the comments agree again, thus saving some future observer from going "...wait a minute, THAT'S NOT WHAT IT SAYS." (For the record, the code won.) Patch by yours truly.
27. Fix missing </div> on /site/index.bml (bug 435)
CLEANUP, BUGFIX: And now I will write "I will check my HTML output to make sure that I have not accidentally left out a closing tag" on the chalkboard a thousand times. Missing </div> was causing display issues on the site map; Yours Truly fixed the error, which was her own silly fault in the first place.
28. Full text of PMs (private messages) in email (bug 345)
USABILITY: Remember a while back when LiveJournal switched to sending you the full text of the message when someone sent you a private message, instead of just saying "hey, someone sent you a message"? This patch does that on Dreamwidth, too. (Next step: fix the double-escaping of things like quote marks and apostrophes.) Patch by Phoenix, who was a little startled, but tremendously pleased, to discover that rather than the complicated change we thought it'd be, it was, in fact, a single line's difference.
29. (I&I) Cross-site use of <lj user> tags (bug 92)
INTEROPERABILITY: This is one of my favorite changes we did this week. In order to facilitate cross-site linking, and not make people have to manually construct the <lj user> tags if they're talking about someone on (say) LiveJournal or InsaneJournal or Inksome, instead of someone on Dreamwidth, we now support the "site" extension to the <lj user> tag (which, yes, will be renamed to get rid of the 'lj', though we'll continue to support the old way as well). If you want to talk about your friend "exampleusername" on LJ, for instance, all you have to do is say <lj user=exampleusername site=livejournal.com>. Patch by Mark, with consultation from Afuna.
*
COMMERCIAL BREAK: With the cross-site <lj user> tags, we'll even try to use the other site's lj-user head icon where possible, and -- for extra win -- the importer automatically updates all the <lj user> tags in entries you import so you're not referring to the wrong place once you've imported your entries. We believe that it's the little touches and attention to detail that shows we take our craft and our product seriously. We also believe that it's the little touches and attention to detail that shows you guys that we are passionate users of the service too. (I will totally confess that half of the bugfixes I have made are to fix tiny things that just *annoy me*.)
*
30. Implement delete_trust_group method (bug 370)
CLEANUP: Adds the backend function to delete a particular trust group. Patch by Mark.
31. Incorrect invite-code regex in htdocs/create.bml (bug 351)
BUGFIX: This fixes an inherited problem -- back when LJ had invite codes, it used to do some magic to check, when you were signing up for an account, that you weren't accidentally entering the invite code where the username should be. We inherited that check in our code, but it didn't check for the actual patterns of *our* invite codes, which are different than LJ's used to be. Patch by Janine.
32. Kill /news.bml (bug 425)
CLEANUP: Removes an old legacy way of getting to the News journal, because it was implemented poorly (and relied on S1, which we don't have). We figure that since we have links to the News journal on the site map (and will have a link on the front page), we can get away with not having the shorter URL, especially since fixing the implementation to do the Right Thing would have been way too much effort. Patch by Yours Truly.
33. List OpenID users in some order (bug 386)
CLEANUP, INTEROPERABILITY: Previously, on the profile, OpenID users would sort by their invisible backend name (ext_NNN), not by their OpenID URL. Now it sorts alphabetically on displayed name (which is the OpenID URL). Patch by Afuna.
34. Make editgroups.bml work (bug 289)
USABILITY: Finish up migrating the editgroups.bml page (which is now editfilters.bml) to use the new watch/trust split. Patch by Janine.
35. Make spellchecker recognize Dreamwidth-specific terminology (bug 406)
USABILITY: Lets us specify site-specific terminology, so we can have the spellchecker (more on which in a bit) recognize things like Dreamwidth and DW as properly-spelled. Limited terminology right now; we'll see what kind of stuff to add once y'all move in and start generating new vocabulary, like LJ users generated the terms 'flist' and 'flock'. Patch by Afuna.
*
COMMERCIAL BREAK: You'll hear me say "once y'all move in", "in another six months", "once we can see what the usage patterns are like", and the like an awful lot. When we set our initial priorities and goals, and when we have to make a decision now about what site usage should be, we've been using two things. One is our design personas list. The other is our understanding of how people used (and use) LJ (which also contributed to how we built our design personas). This is a good and useful shorthand, but it ignores the fact that Dreamwidth is not LJ, and won't try to be. We hope that from the very beginning we can build our own identity, culture, and usage patterns. We just don't know what those might *be* yet.
So a lot of things are up in the air: we've made decisions on how to handle them for now, but we don't know if they're going to work with or against the way our users wind up using the service. This is why we promise you that we're always going to watch and listen to how you're *actually* using the site, not the way we think you should be using the site.
*
36. Manage Filters page does not alphabetize lists (bug 450)
USABILITY: In updating the Manage Filters page to use the watch/trust split, we forgot to sort the users in/not in filters by username, and not by user id. Now it sorts by username! Patch by Janine.
37. Manage | Mobile won't save settings (bug 453)
BUGFIX: Thanks to a bug in our error checking, people could specify whatever they wanted on the Mobile tab of the Manage Settings page (which is not a problem) but the changes wouldn't save. Now they do. Patch by Janine.
38. Migrate navigation strip options to /manage/settings (bug 438)
USABILITY: Zvi pointed out to us that despite us having changed the navigation strip so that it's a user viewing preference (follows the logged-in user) and not a journal customization preference (follows the journal preference), or a shambling hybrid of the two, we didn't move the configuration options to a logical place. We moved it, and consolidated the options down to a more sane subset. (We'd flirted with making the navstrip options totally customizable, which I thought was cool, but 9 options were way too overwhelming.) Patch by Afuna.
39. Missing strings on /manage/circle/, making the page mostly impossible to use (bug 457)
BUGFIX: When English-Stripping Goes Horribly Wrong. Fixes the page to use the correct translation strings. Patch by yours truly.
40. Need new trust group method (bug 442)
USABILITY: Adds a function to return the full watch and trust lists for a given user, to be used in code elsewhere. Patch by Mark.
41. Need to add reserved usernames (bug 365)
ADMIN: True story! I wanted to extend the reserved-usernames list (the usernames that people can't register, because they're used by the system in some way), but I also wanted to make it easy to edit or change on the fly if necessary, without having to commit and push a patch. I suggested how to do it, Janine explained why it wasn't the best way, I suggested another way, Janine and I kept talking about how to best implement it... it went on for about 45 minutes. Mark got tired of hearing us take a simple problem and over-spec it, so he implemented this in nine minutes. And bragged about it in Changelog, too. Moral of the story: if I try to overcomplicate things, Mark will eventually get annoyed, stop what he was doing, and implement a fix for my bug in under 10 minutes! Maybe that wasn't the lesson he wanted me to learn. Patch by Mark. Oh, and thanks to Sophie for helping me get the required formatting for the list itself.
42. OpenID menus missing items (bug 447)
USABILITY: Last week, due to some usability feedback from our testers, we removed the menu items leading to site features OpenID accounts couldn't access or use, so we didn't tempt them with things that would only error when they clicked. However, I forgot to take into account that OpenID users will have a reading page, which means they'll probably want to select a style; I was just thinking "oh, hey, they don't have a journal!" This put the style-related pages back into the menu. Patch by yours truly, who spent a lot of time this week cleaning up after herself.
43. OpenID validation email should be customized for OpenID accounts (bug 311)
USABILITY: If an OpenID user changed or set their email address, the email validation would refer to the account as the ext_NNN internal username. This patch updates that to instead use the OpenID URL: exampleusername.livejournal.com, for instance. Patch by yours truly.
*
COMMERCIAL BREAK: One of the things that we're really committed to doing is making the site experience much better for OpenID identity users. Everybody always forgets about them when they're designing, and the code never really was updated to take full advantage of things, so OpenID users wind up feeling like they've been permanently branded second-class citizens of the site. We don't like that, so we're doing a lot to improve it.
The quick, little fixes like this will be done by open beta. The slightly larger and more involved fixes, which include things like allowing people to "upgrade" their OpenID accounts into full accounts after a certain length of time, allowing people to merge their Dreamwidth account and any OpenID identity they might own, etc, will all follow shortly thereafter. We love our OpenID users, too.
*
44. Profile page text messages use "LJ" as the subject (bug 364)
CLEANUP, DOCUMENTATION: More English-stripping, in this case a hardcoded "LJ". Patch by Afuna.
45. Redo menu items to be a separate file for site schemes to use (bug 321)
ADMIN, USABILITY: I actually want to geek out here for a second. The *details* of what this patch does (splits out our menu items into a function that any site scheme designer can use, which will mean that we only have to change things in *one* location, rather than across every single site scheme; this means that we can provide a much larger number of site scheme options without dramatically increasing our code maintenance burden) aren't the important part, although is is cool. What I find most exciting about this patch is that one of our programmers, Sophie, was the one who identified the opportunity to make a massive admin-usability and code maintenance improvement with just a little bit of effort. She solved a problem that wasn't even on our radar yet, and did it in a way that'll directly enable us to dramatically increase the number of options we can offer our userbase. That, right there, is why we believe so strongly in Open Source, and why we, as a project, are so committed to mentoring new programmers, empowering and assisting our developers, and being open to patches and improvements from all corners. Thanks, Sophie.
46. Reduce maxlength of text field for feed account creation (bug 469)
USABILITY: Restrict feed accounts (syndicated/RSS accounts) maximum name length to 20 characters, not the full 25 characters that personal accounts can have. This is to make room for us to add _feed to the end of all syndicated accounts automatically. Patch by rho.
*
COMMERCIAL BREAK: We've increased the username text length to 25 characters, but for syndicated feeds created on Dreamwidth, we've made a simple change: they no longer appear on the syndicated.dreamwidth.org/* URL namespace, but instead are all created with _feed on the end of the account name. This helps to preserve the user namespace for users, and avoids situations where (for instance) someone might create a syndicated feed of your LiveJournal before you create an account on Dreamwidth and use the username you'd want on Dreamwidth, which would prevent you from registering that username. Rather than having to rename those accounts manually when that happened, or ask people to not accidentally "use up" someone else's username from elsewhere, we just add _feed to the end of all feed accounts, and don't let users create personal accounts ending in _feed -- problem solved in a technical way, not a social way or with a manual administrative workaround! (We picked _feed instead of something like "feed_*" at the beginning of the username because we've mostly seen people doing it at the end of the account on LJ, to the extent that there is any community consensus.)
*
47-50. Removing various unused pages/site "guts" otherwise shown to users (bugs 456, 377, 426, 354)
CLEANUP: Patches to remove incredibly old files ($LJHOME/doc/todo.txt, which was apparently the minutes of a 2002-era Danga Interactive staff meeting, patch by me; remove code for a feature that hasn't worked on LJ since 2002 and couldn't be made to work as-is without massive work (dreamwidth.org/meme.bml), patch by me; stop showing users the internal backend guts of the FAQ system in some cases (the FAQ number was in the <title> attribute of the resulting FAQ page, which is something users don't need to know), patch by phoenix; and finally, temporarily disable the Schools directory, until we can figure out how to make some usability fixes, clean up the very, very out-of-date and 'dirty' data shipped with the LJ codebase, and work out an admin system that doesn't require hundreds of person-hours a month to handle, patch by me.
51. S2 stylesheets randomly returning 0 bytes (bug 439)
BUGFIX: For those of you who've been using the Dreamwidth site in beta, and who had pasted in a custom style, I'm sure you noticed that sometimes, randomly, your journal style would fail to load and the page would appear unstyled. It finally annoyed Mark enough that he grabbed his debug tools to figure out what was going on and fixed the problem. Patch by Mark.
52. Set up email posts (bug 376)
PAID USERS, USABILITY: This bug was more of a task: set up and configure the ability to make a post by emailing it into the site. Took Mark a bunch of struggling with (and swearing) at our mail transport agent, but he emerged from the field of battle, victorious. I should also note that in the process, my POP sessions to my @dwscoalition.org email stopped working. "Mark!" I said. "You broke it!" He gave me that "...this will not end well" look (I can totally read Mark's looks over the internet by now) and said "What server do you have it trying to POP from?" Yeah, okay, it *does* work better if you have it set to the right mail server. (Not as bad as the time he once spent two hours helping me troubleshoot why mail was failing to my domain, only for me to sheepishly discover that I'd accidentally let the domain lapse. But close.) Config change and mail system wrangling by Mark.
53. Site scheme fails when smaller text size set (bug 183)
USABILITY, ACCESSABILITY: We know that our site scheme behaves badly at larger or smaller font sizes; this was our first change to try to address the problem, and we're still working on making it better. Patch by Jesse.
54. Site-scheme search type elements aren't listed correctly (bug 412)
BUGFIX: This actually didn't require a code change, just reloading the translation strings from the database, but oh, the research it took to figure that out.
55. Site-scheme, English-strip, and modernize /admin/console/index.bml (bug 437)
USABILITY, CLEANUP: This is one of the entries on the "tiny things that have always annoyed me" list: the Admin Console now uses the site scheme (instead of the blank white page with the image of the guy who always looked creepily like Brad, but was not in fact Brad), uses contemporary coding style, and outputs HTML that is not in <ALLCAPS> and ugly. NO MORE NAVIGATIONAL DEADEND ON THE ADMIN CONSOLE. Patch by yours truly.
56. Tell OpenID users to validate their accounts (bug 415)
USABILITY, INTEROPERABILITY: Right now (and we are changing this, but might not get to it immediately) OpenID users are treated as anonymous users for the purposes of comment screening, *unless* they have set and validated their email address, in which case they'd be treated as logged-in users. This patch adds some language on the /openid page to make that more explicit, as part of our efforts to improve the site experience for OpenID users. Patch by me; I'm embarrassed to say that I can't remember who originally suggested it, but I know one of our OpenID site testers did, and I'm thankful that they did, since I never personally used OpenID logins much and wouldn't have noticed it.
57. Text message disclaimer footer gets printed out multiple times (or 0 times) (bug 414)
BUGFIX: Fixes a bug where the text messaging interface really, really, *really* wanted you to know that the reliability of text messaging should not be trusted in dealing with emergencies. Afuna curbed its enthusiasm.
58. Top 1000 syn accounts page is empty (bug 422)
BUGFIX, CLEANUP: Update the "popular syndicated accounts" list to use the watch/trust friends split. Take a look! People are already using syndication, even during open beta, which is so cool: http://www.dreamwidth.org/syn/ . (Yeah, we know that page erroneously references syndication points -- our site copy team will correct that.) Patch by Mark.
59. Tropo: CSS-class, style /support/* (bug 417)
USABILITY, CLEANUP: As part of the default Dreamwidth.org site scheme, Tropospherical Red, we load a CSS file that explicitly strips a lot of default-assumptions that each browser makes about how things like tables should be rendered, because doing that gives us better ability to deal with cross-browser glitches. This has the unfortunate side effect of breaking anywhere on the site that relies solely on tables for layout ... which, given that Brad started programming on LJ in 1998 when the "MUST NOT USE TABLES" recommendation hadn't yet passed into common wisdom, is a lot of places. We've been updating various pages to at least CSS-class the tables so we can manipulate them via CSS; actually rewriting the HTML output to get rid of them entirely is going to be a massive undertaking. Patch by yours truly.
*
COMMERCIAL BREAK: We have a tremendous, tremendous advantage in that we're not starting from scratch to build Dreamwidth; we're working with a codebase that has ten years' worth of bugfixes, features, enhancements, and other improvements. We have a tremendous disadvantage in that we're not starting from scratch to build Dreamwidth; we're working with a codebase that has ten years' worth of byzantine cruft in a lot of places. A lot of what we've been doing since we started work on this project is a *lot* of backend work to rip out the most egregious cruft-built-up-over-time and modernize things, but the way the site actually builds and renders its HTML is among the hardest, because LJ uses a unique templating system known as BML.
BML -- nobody, Brad included, ever decided if it stood for Better Markup Language or Brad Markup Language -- is something Brad built from scratch back in 1997 for his first website venture, FreeVote.com, and he brought it with him when he started working on LiveJournal, because there really *weren't* any other options at the time.
BML has its strengths and its weaknesses; among the weaknesses is that it's not being actively developed to add new features, since it's reached the level of 'stable' enough for LiveJournal's purposes. We're currently evaluating the possibility of ripping out BML and replacing it with another Open Source templating system that will hopefully be much, much easier to use, update, and create site schemes for. Until we can do that, though -- and it's a massive undertaking that we'll probably have to do slowly over time -- we're probably not going to be doing extensive code-optimization to make sure that the code produce the 'right' HTML/CSS, since we'd just have to do it over again when we make the switch; we'll live with the workarounds for now.
*
60. Ugly error message when uploading a userpic of the wrong filetype (bug 160)
CLEANUP: This is one of those bugs that spontaneously fixed itself thanks to some other code or config change. I wish we could get more of those. Maybe the bugfixing fairies came in the middle of the night to pay us a visit.
61. update FOAF ljfeed.pl to use WTF (bug 441)
CLEANUP: FOAF (http://www.foaf-project.org/) is an open protocol to support the automatic discovery of your relationships with other people. With this bug, we updated it to use the new watch/trust friend split. LJ supports this protocol, as well as many other open data-access protocols, and we believe it's important for us to also support as many widely-adopted open protocols as we can, speaking to our commitment to interoperability and our principle that it should always be easy for you to get your data back out of our servers if you decide that you're not for us. Patch by Janine.
*
COMMERCIAL BREAK: One of the things we have on our radar -- not for this year, but we're aware of it -- is supporting Google's OpenSocial API, which is specifically designed to find your friends on other social networks if you want to. For maximum irony, OpenSocial was developed at Google by one Brad Fitzpatrick ... remember him?
*
62. update LJ::Spellcheck to work in Apache2 (bug 307)
CLEANUP, BUGFIX: Remember way up there in #35 I promised I'd talk more about the spellchecker? Here it is! I noticed a few weeks ago that the spellcheck option wasn't appearing on the beta site, thinking that it was some kind of configuration that was misconfigured. Turns out, Mark discovered, that our efforts to modernize the code and upgrade to a more recent version of Apache utterly broke the glue that held together the LJ application and the operating system's spellchecker. Afuna did some heroic work to figure out a replacement option and get it working under Apache2, so we'll soon have spellcheck again. And it will know the site's username.
63. Use common logic for user link bar on profile page and in S2 (bug 400)
CLEANUP, USABILITY: This patch eliminates duplicate code; the code for the user link bar (the standout box that appears to show you the little useful graphics that indicate what actions you can take, like next/forward/track this/add to memories/tag/etc) used to occur in two separate places, not shared; now it's just one place and they both share it. Patch by Afuna. (Who demonstrated her sense of humor in comments #1 and #2 to bug 400, and the entirety of bug 402, which just proves that a). we were getting a bit giddy at that point and b). we are all Star Trek geeks.)
64. Wrap user-generated content in CSS class to apply sane defaults (bug 264)
USABILITY: See #59 for why we include the CSS that strips the styling from many HTML tags. Users will want to use these HTML tags in their journals/profiles/comments, though, and they won't understand why they don't work (because the reset.css file removes any default definitions the browser provides). This is the start of an attempt to let users still use those tags and have them behave normally, without sacrificing the cross-browser benefits of reset.css. Patch by Jesse.
65. [community]/read isn't up (bug 387)
USABILITY, COMMUNITY: As part of the friends->watch/trust split, we changed how community 'membership' works in the backend of the codebase, because the old way of doing it was very much a workaround. (A nine-year workaround!) Because of the updates, community reading pages weren't being populated by public entries from comm members. This fixes that! Patch by Mark.
***
So, what have we learned from this tour through this week's codebase?
* We did a LOT of work this week.
* Most of it was in the realm of code cleanup: stuff designed to fix up and modernize the codebase we inherited, or stuff to fix bugs, either in the code we forked from or the code we've written ourselves.
* Of course, there was also a lot of "...this has ALWAYS ANNOYED ME and now I can FIX IT", which is, I will admit, one of the major satisfaction-bringing things about working on this project.
* Our beta testing team, led by the incredibly capable Domtheknight and full of people who really believe in poking in until it breaks, rocks house at finding things that we never would have noticed.
* The number of open-beta blocking items has finally, *finally* started to creep back downward, as we fix things we've identified as priority bugfixes for launch and people finish up projects that add the last remaining functionality we need to consider ourselves "ready for prime time".
* It really is dangerous when you teach the Suit how to code.
* Most importantly -- or at least, I hope you'd agree -- we are committed to making the site better and more usable for everybody, as much as we can.
I really can't say often enough that our Open Source programmers are doing an incredible job -- both the people who have been programming for years, and the people who are learning how to program, and how to participate in an Open Source project, with us.
***
So, where do we go from here? What's the plan for the next steps in Dreamwidth code development? I spent some time with the Wiki and Bugzilla this week and wrangled mostly everything into shape to answer that question.
Our Roadmap contains a brief summary of our prioritization processes, and includes Bugzilla links that will let you browse through what we have planned, all the way out to the end of 2009. After that ... well, we'll see what y'all will find most valuable and useful, now won't we?
***
Current bug count:
-- 200 open bugs
-- 270 resolved bugs
-- 46 blocking-open-beta bugs
-- 43 blocking-open-beta bugs without a patch waiting for review
***
We'll be back next week with an update in the more regular format, along with updates from areas other than development. I hope you've enjoyed this look through our development lens -- the kind of things we think about when we contemplate what items to prioritize for the various stages of our launch timeline.
The dw-discuss mailing list is your place to go for questions, comments, ideas, and discussion.
For semi-regular updates on what-we-did-today, check out our Dreamwidth Twitter account.
For more real-time discussion, commentary, and the ability to consign things to hell -- or at least, to our irc bot "hEll" -- come join us
in irc:
irc.dwscoalition.org, port 6667, channel #dw
Questions? Comments? Concerns? Donations of spare sleep? (I've been awake for 36 hours by this point!) Comment! (Remember, you can use your OpenID account, and we've even been making it better!)
We'll see you next week, where we will hopefully have passed the threshold of 300 resolved bugs and come out the other side. Just like I came out the other side of Monday in my timezone. Oops.
This week we provide our update in slightly different form. Because the majority of our work this past week has gone to development (we're about a month away from when we really want to launch to open beta!), this week our progress report is going to be a listing off of all the bugs that we resolved, with what they were intended to fix and why we think that's important to fix. I'll intersperse it with additional clarification on bugs that I have a lot to say about.
I'm not only doing this because the majority of our effors this week were code-related; I'm doing this because some feedback we've heard is that the development process is rather opaque, and people don't know what changes are being made. So I hope that you will find this guided tour through this week's development efforts of interest. If nothing else, it'll be a good look into our thought processes as we go. We'll be back to our usual update format next week.
We resolved 75 bugs this week; 66 of them were actual fixes or patches, as opposed to things we decided we didn't need to fix, things we decided to defer until later, things that were duplicates of other items, reported issues that we couldn't reproduce using the same steps that the tester used, etc. One of those patches was a very, very tiny privacy-related fix that we decline to describe to this mailing list, because we'd like to give LiveJournal a chance to apply the patch first.
To the videotape.
1. "Old character encoding" no longer needed: patch by Ryan (bug 108)
CLEANUP: This patch removes old backwards compatability functionality from the LJ codebase. Because we're starting off with all of our data in the UTF-8 text encoding, we don't need to let people convert other text encodings to UTF-8. Eliminating unnecessary functionality = cleaner code = happier Dreamwidth! Also of note: this was Ryan's first patch to us. Thanks for joining our code team, Ryan!
2. "Insert image" popup in HTML editor broken (bug 308)
BUGFIX: This patch fixes the display of the "insert image" popup box on update.bml. It was broken due to interaction problems with our site scheme. Now it works! Patch by yours truly, but Afuna figured out the problem; I just typed it in.
3. <dt> in /site/index.bml needs to be wrapped in <dl> (bug 436)
CLEANUP: This patch corrects some missing HTML tags on the Site Map. Patch by yours truly, who wrote the original page and often forgets the little niceties of HTML.
4. Add sysban to Admin index (bug 461)
ADMIN, USABILITY: Adds a link to the sysban tool -- one of our administrative tools that will help us block and filter abusive uses of the site -- to the page with all the other admin tools. Patch by yours truly.
5. Add table of contents to faq.bml (bug 427)
DOCUMENTATION, USABILITY: To make it easier for users to find the specific FAQ category they're looking for, this patch adds a very basic table of contents to the top of the page containing links to all of our FAQs. Patch by Rho, our fabulous doc admin.
*
COMMERCIAL BREAK: You may notice, if you click through to the FAQ page on Dreamwidth itself, that our FAQ team is starting to fill out the contents of certain FAQs. This is because our FAQ team is made of 100% scientific-grade Awesome.
You may also notice that the resulting FAQs occasionally have little notes in them saying things like "need to check this" and the like. This is either because I am made of less Awesome than the FAQ team, and have not yet answered some of their questions, or because we haven't actually fixed the behavior that they're documenting yet.
The FAQ team is still made of Awesome, though.
*
6. Allow self-messaging (bug 463)
USABILITY: So, true story: I was trying to test something on my development environment, and I needed a private message in my Inbox to test on. Logically speaking, of course I log in and go to send a message to myself. Imagine my surprise when I get an error message! (Imagine my surprise when that error message says "Stop trying to message yourself, we're not that kind of service.") So as soon as I finished boggling, I submitted this patch to take out that error check and allow people to message themselves. I guess we *are* that kind of site?
7. Change data-gathering so that "gender" is only recorded (and recordable) for accounts of type P (personal account) (bug 20)
CLEANUP: This corrects our internal statistics so that the only account types that can specify (or decline to specify) their gender on the Edit Profile page are Personal and OpenID accounts. This will give us a much cleaner picture of our demographics; otherwise, community and syndicated feeds would be lumped into the numbers and skew our picture of our userbase. (This data is one of the many things we'll use to make intelligent decisions about the site's future development efforts.) Patch by Dre.
8. Configure login rate limiting (bug 455)
ADMIN: This patch sets up the login rate limiting for our service, which governs the number of failed logins you can have in a certain window. This helps to prevent people from trying to brute-force or dictionary-attack a specific account, trying passwords over and over again until one works. I'll confess, we kinda picked the number arbitrarily; we'll tweak the settings if legit use winds up tripping it too often. Patch by Janine.
9. Configure Thread Expander (bug 459)
PAID USERS, USABILITY: This sets up the "expand all comments in a thread" JavaScript for paid/premium/seed accounts. (The functionality was already there, we just hadn't specifically enabled it yet.) Right now, this just replicates the way the function works on LJ, but oh, do we have plans for this. Patch by Afuna.
10. Configure video embedding subdomain for production (bug 423)
CONFIG: For security reasons, video embeds (such as YouTube videos, Flash animations, etc) are served from a different subdomain, in order to isolate the scripting involved. This is so that in case someone manages to sneak a malicious embedded object past our code sentries, it's still limited in the damage it can potentially do. Config change by Mark.
*
COMMERCIAL BREAK: Mark did lots and lots and lots of things this week that were config-related, hardware-related, setup-related, or otherwise Not Appearing In This Update. I figure that if we could just clone Mark, we'd be all set.
*
11. Correct stylistic flaw in doc/template.bml.txt (bug 444)
CLEANUP: Now, look, I love Mark. We all love Mark. But we all also have to accept the simple fact ... Mark is as fanatical about certain matters of programming style as I am about those Russell Stover pectin jellybeans that only appear in the drugstores around Easter. We try not to hold it against him. This patch corrects a stylistic error in the file we provide for programmers as an example of how to build a user-facing webpage using our templating system. Patch by yours truly. After Mark gently chided me for the style error that I had just copied from the old template.
12. Create LJ::User is_individual function (392)
ADMIN, CLEANUP: This is a backend patch that defines a function to identify if the user being referenced is *either* a personal journal account or an OpenID identity account. This has the potential to simplify a bunch of redundant calls all over the codebase; we'll be slowly moving the old code over to using the new function. Patch by Afuna. Who then realized she'd forgotten to put it in alphabetical order after I spent four hours last week organizing that whole nine-thousand-line file, and promptly fixed it before I had to give her the sad puppy dog eyes.
13. Create site scheme look files from Photoshop mockups (bug 104)
USABILITY, LOOK & FEEL: This was the bug that had been open for a while for people to work on creating the site scheme template files from the mockups of the design (designed for us by the ever-fabulous grrliz). We know we've still got a few wrinkles to iron out, but since the functionality is now all there, we closed this bug in favor of opening new ones to fix the outstanding issues. Patches by Jesse and Janine.
14. Create support categories on dreamwidth.org (bug 227)
ADMIN, CONFIG: Not that we're ready to start accepting support requests yet -- far from it; all feedback to us should still use the same channels you've been using all along -- but this item was to configure the categories for when we are ready, and it was completed by the long-suffering Mark.
15. creating filters with invalid names (bug 367)
USABILITY, CLEANUP: When Mark first did the watching/trusting friend split, and reimplemented custom access filters that you can lock your posts to, he was insanely strict about what characters could be used in filter names. (Enforcing strictness when dealing with user data is one of many, many ways you can help avoid introducing security holes.) Turns out that it doesn't need to be that strict, so we just went with the same rules that the LiveJournal codebase uses for what's permitted in filter names -- which has the added benefit of not breaking cross-site compatability. Fix by Mark.
*
COMMERCIAL BREAK: I wanted to actually take a second here and specifically give a shout-out to Mark for the watching/trusting friends split, which is an incredibly involved change that he's been incredibly diligent about not only updating, but *documenting.* (Mark is one of those true rarities of nature: a programmer who writes fantastic documentation.) I also wanted to acknowledge Janine, who has done a truly heroic amount of work to find all of the places on the site that used to use friend-related code functions and update them to us the new watching/trusting code functions. They've done incredible work on the system.
*
16. Define and implement new user defaults (bug 372)
USABILITY: Patch to set all of the defaults for a newly-created account -- users can change these at any time, but we pre-populate the account with what we think are some sane defaults. Patch by Janine.
17. Document all user caps (bug 458)
CLEANUP: In his house at R'lyeh Great Cthulhu lies dreaming. In the code at Dreamwidth, great mysteries lie sleeping. Right now, as-is, the whole model of what users can and can't do, what features they can use, what features are on and off for the site as a whole, is governed by an arcane, esoteric, undocumented set of capabilities and flags, which can only be turned on and off by listing them in the textfile configuration files. We thought it would be a lot easier to set these capabilities and flags, not to mention deciding what the settings should be, if there were actually a complete list of what they *were*. Anywhere. At all. Thanks to Foxfirefey, now there is. Thanks for giving us a bonus to our SAN roll, Fey.
18-22. English-strip various files (bugs 382, 260, 409, 385, 380)
CLEANUP, DOCUMENTATION: 'English-stripping' refers to the process of removing the hardcoded English text on the site and instead putting it into the translation system, where the site copy team will be able to change it on-the-fly and where other sites using the Dreamwidth code will be able to translate the site into other languages. These bugs fix up some places that had not already been English-stripped in: dreamwidth.org/openid/, dreamwidth.org/editpics.bml, dreamwidth.org/stats.bml, dreamwidth.org/tools/recent_comments.bml, and dreamwidth.org/manage/profile/. Patches by me, Sophie, and Isabeau, and a special call-out to Isabeau's first (and second!) patch commit. Thanks for joining the dev team, Isabeau!
*
COMMERCIAL BREAK / PUBLIC SERVICE ANNOUNCEMENT: We won't be offering DW in other languages, for a lot of reasons, but mostly involving the fact that the translation system is sub-optimally usable. However, I come bearing a fun little trick: if you ever want to see whether or not a page (on Dreamwidth, LiveJournal, or any LJ-based clone site running code from any time past mid-2002 or so) has been English-stripped, add ?uselang=debug to the end of the URL, marvel at all the variables that suddenly appear, and look for any English text. (We've got bugs open to fully complete all of the English-stripping we still need to do, so over the next few weeks we'll be getting rid of all of those.)
*
23. Error handling for WTF backend (bug 368)
CLEANUP: As part of our split of 'friends' into watching vs trusting, we needed to gracefully handle error conditions if somebody tried to do something that didn't make sense. Now we do! Patch by Mark, who I bet is really tired of looking at friend-related code by now.
24. File inserts overwritten on code push (357)
ADMIN, BUGFIX: Man, just the thought of explaining this one makes me feel like I have to lie down. Suffice it to say that it's another case of something that was designed for the short-term quick fix sticking around way longer than it was originally intended to. Mark sprinkled the appropriate magic pixie dust over the right sections of the config until we can update the whole system to do it differently.
25. Fix dumpsql.pl (bug 466)
CLEANUP: This is a fix to a backend database-related script to remove references to outdated perl libraries we don't use anymore. Patch by Pauamma.
26. Fix incorrect comment in cgi-bin/ljfeed.pl (bug 440)
CLEANUP: Ever heard the saying "if the code and the comments disagree, they are both probably wrong?" This patch makes the code and the comments agree again, thus saving some future observer from going "...wait a minute, THAT'S NOT WHAT IT SAYS." (For the record, the code won.) Patch by yours truly.
27. Fix missing </div> on /site/index.bml (bug 435)
CLEANUP, BUGFIX: And now I will write "I will check my HTML output to make sure that I have not accidentally left out a closing tag" on the chalkboard a thousand times. Missing </div> was causing display issues on the site map; Yours Truly fixed the error, which was her own silly fault in the first place.
28. Full text of PMs (private messages) in email (bug 345)
USABILITY: Remember a while back when LiveJournal switched to sending you the full text of the message when someone sent you a private message, instead of just saying "hey, someone sent you a message"? This patch does that on Dreamwidth, too. (Next step: fix the double-escaping of things like quote marks and apostrophes.) Patch by Phoenix, who was a little startled, but tremendously pleased, to discover that rather than the complicated change we thought it'd be, it was, in fact, a single line's difference.
29. (I&I) Cross-site use of <lj user> tags (bug 92)
INTEROPERABILITY: This is one of my favorite changes we did this week. In order to facilitate cross-site linking, and not make people have to manually construct the <lj user> tags if they're talking about someone on (say) LiveJournal or InsaneJournal or Inksome, instead of someone on Dreamwidth, we now support the "site" extension to the <lj user> tag (which, yes, will be renamed to get rid of the 'lj', though we'll continue to support the old way as well). If you want to talk about your friend "exampleusername" on LJ, for instance, all you have to do is say <lj user=exampleusername site=livejournal.com>. Patch by Mark, with consultation from Afuna.
*
COMMERCIAL BREAK: With the cross-site <lj user> tags, we'll even try to use the other site's lj-user head icon where possible, and -- for extra win -- the importer automatically updates all the <lj user> tags in entries you import so you're not referring to the wrong place once you've imported your entries. We believe that it's the little touches and attention to detail that shows we take our craft and our product seriously. We also believe that it's the little touches and attention to detail that shows you guys that we are passionate users of the service too. (I will totally confess that half of the bugfixes I have made are to fix tiny things that just *annoy me*.)
*
30. Implement delete_trust_group method (bug 370)
CLEANUP: Adds the backend function to delete a particular trust group. Patch by Mark.
31. Incorrect invite-code regex in htdocs/create.bml (bug 351)
BUGFIX: This fixes an inherited problem -- back when LJ had invite codes, it used to do some magic to check, when you were signing up for an account, that you weren't accidentally entering the invite code where the username should be. We inherited that check in our code, but it didn't check for the actual patterns of *our* invite codes, which are different than LJ's used to be. Patch by Janine.
32. Kill /news.bml (bug 425)
CLEANUP: Removes an old legacy way of getting to the News journal, because it was implemented poorly (and relied on S1, which we don't have). We figure that since we have links to the News journal on the site map (and will have a link on the front page), we can get away with not having the shorter URL, especially since fixing the implementation to do the Right Thing would have been way too much effort. Patch by Yours Truly.
33. List OpenID users in some order (bug 386)
CLEANUP, INTEROPERABILITY: Previously, on the profile, OpenID users would sort by their invisible backend name (ext_NNN), not by their OpenID URL. Now it sorts alphabetically on displayed name (which is the OpenID URL). Patch by Afuna.
34. Make editgroups.bml work (bug 289)
USABILITY: Finish up migrating the editgroups.bml page (which is now editfilters.bml) to use the new watch/trust split. Patch by Janine.
35. Make spellchecker recognize Dreamwidth-specific terminology (bug 406)
USABILITY: Lets us specify site-specific terminology, so we can have the spellchecker (more on which in a bit) recognize things like Dreamwidth and DW as properly-spelled. Limited terminology right now; we'll see what kind of stuff to add once y'all move in and start generating new vocabulary, like LJ users generated the terms 'flist' and 'flock'. Patch by Afuna.
*
COMMERCIAL BREAK: You'll hear me say "once y'all move in", "in another six months", "once we can see what the usage patterns are like", and the like an awful lot. When we set our initial priorities and goals, and when we have to make a decision now about what site usage should be, we've been using two things. One is our design personas list. The other is our understanding of how people used (and use) LJ (which also contributed to how we built our design personas). This is a good and useful shorthand, but it ignores the fact that Dreamwidth is not LJ, and won't try to be. We hope that from the very beginning we can build our own identity, culture, and usage patterns. We just don't know what those might *be* yet.
So a lot of things are up in the air: we've made decisions on how to handle them for now, but we don't know if they're going to work with or against the way our users wind up using the service. This is why we promise you that we're always going to watch and listen to how you're *actually* using the site, not the way we think you should be using the site.
*
36. Manage Filters page does not alphabetize lists (bug 450)
USABILITY: In updating the Manage Filters page to use the watch/trust split, we forgot to sort the users in/not in filters by username, and not by user id. Now it sorts by username! Patch by Janine.
37. Manage | Mobile won't save settings (bug 453)
BUGFIX: Thanks to a bug in our error checking, people could specify whatever they wanted on the Mobile tab of the Manage Settings page (which is not a problem) but the changes wouldn't save. Now they do. Patch by Janine.
38. Migrate navigation strip options to /manage/settings (bug 438)
USABILITY: Zvi pointed out to us that despite us having changed the navigation strip so that it's a user viewing preference (follows the logged-in user) and not a journal customization preference (follows the journal preference), or a shambling hybrid of the two, we didn't move the configuration options to a logical place. We moved it, and consolidated the options down to a more sane subset. (We'd flirted with making the navstrip options totally customizable, which I thought was cool, but 9 options were way too overwhelming.) Patch by Afuna.
39. Missing strings on /manage/circle/, making the page mostly impossible to use (bug 457)
BUGFIX: When English-Stripping Goes Horribly Wrong. Fixes the page to use the correct translation strings. Patch by yours truly.
40. Need new trust group method (bug 442)
USABILITY: Adds a function to return the full watch and trust lists for a given user, to be used in code elsewhere. Patch by Mark.
41. Need to add reserved usernames (bug 365)
ADMIN: True story! I wanted to extend the reserved-usernames list (the usernames that people can't register, because they're used by the system in some way), but I also wanted to make it easy to edit or change on the fly if necessary, without having to commit and push a patch. I suggested how to do it, Janine explained why it wasn't the best way, I suggested another way, Janine and I kept talking about how to best implement it... it went on for about 45 minutes. Mark got tired of hearing us take a simple problem and over-spec it, so he implemented this in nine minutes. And bragged about it in Changelog, too. Moral of the story: if I try to overcomplicate things, Mark will eventually get annoyed, stop what he was doing, and implement a fix for my bug in under 10 minutes! Maybe that wasn't the lesson he wanted me to learn. Patch by Mark. Oh, and thanks to Sophie for helping me get the required formatting for the list itself.
42. OpenID menus missing items (bug 447)
USABILITY: Last week, due to some usability feedback from our testers, we removed the menu items leading to site features OpenID accounts couldn't access or use, so we didn't tempt them with things that would only error when they clicked. However, I forgot to take into account that OpenID users will have a reading page, which means they'll probably want to select a style; I was just thinking "oh, hey, they don't have a journal!" This put the style-related pages back into the menu. Patch by yours truly, who spent a lot of time this week cleaning up after herself.
43. OpenID validation email should be customized for OpenID accounts (bug 311)
USABILITY: If an OpenID user changed or set their email address, the email validation would refer to the account as the ext_NNN internal username. This patch updates that to instead use the OpenID URL: exampleusername.livejournal.com, for instance. Patch by yours truly.
*
COMMERCIAL BREAK: One of the things that we're really committed to doing is making the site experience much better for OpenID identity users. Everybody always forgets about them when they're designing, and the code never really was updated to take full advantage of things, so OpenID users wind up feeling like they've been permanently branded second-class citizens of the site. We don't like that, so we're doing a lot to improve it.
The quick, little fixes like this will be done by open beta. The slightly larger and more involved fixes, which include things like allowing people to "upgrade" their OpenID accounts into full accounts after a certain length of time, allowing people to merge their Dreamwidth account and any OpenID identity they might own, etc, will all follow shortly thereafter. We love our OpenID users, too.
*
44. Profile page text messages use "LJ" as the subject (bug 364)
CLEANUP, DOCUMENTATION: More English-stripping, in this case a hardcoded "LJ". Patch by Afuna.
45. Redo menu items to be a separate file for site schemes to use (bug 321)
ADMIN, USABILITY: I actually want to geek out here for a second. The *details* of what this patch does (splits out our menu items into a function that any site scheme designer can use, which will mean that we only have to change things in *one* location, rather than across every single site scheme; this means that we can provide a much larger number of site scheme options without dramatically increasing our code maintenance burden) aren't the important part, although is is cool. What I find most exciting about this patch is that one of our programmers, Sophie, was the one who identified the opportunity to make a massive admin-usability and code maintenance improvement with just a little bit of effort. She solved a problem that wasn't even on our radar yet, and did it in a way that'll directly enable us to dramatically increase the number of options we can offer our userbase. That, right there, is why we believe so strongly in Open Source, and why we, as a project, are so committed to mentoring new programmers, empowering and assisting our developers, and being open to patches and improvements from all corners. Thanks, Sophie.
46. Reduce maxlength of text field for feed account creation (bug 469)
USABILITY: Restrict feed accounts (syndicated/RSS accounts) maximum name length to 20 characters, not the full 25 characters that personal accounts can have. This is to make room for us to add _feed to the end of all syndicated accounts automatically. Patch by rho.
*
COMMERCIAL BREAK: We've increased the username text length to 25 characters, but for syndicated feeds created on Dreamwidth, we've made a simple change: they no longer appear on the syndicated.dreamwidth.org/* URL namespace, but instead are all created with _feed on the end of the account name. This helps to preserve the user namespace for users, and avoids situations where (for instance) someone might create a syndicated feed of your LiveJournal before you create an account on Dreamwidth and use the username you'd want on Dreamwidth, which would prevent you from registering that username. Rather than having to rename those accounts manually when that happened, or ask people to not accidentally "use up" someone else's username from elsewhere, we just add _feed to the end of all feed accounts, and don't let users create personal accounts ending in _feed -- problem solved in a technical way, not a social way or with a manual administrative workaround! (We picked _feed instead of something like "feed_*" at the beginning of the username because we've mostly seen people doing it at the end of the account on LJ, to the extent that there is any community consensus.)
*
47-50. Removing various unused pages/site "guts" otherwise shown to users (bugs 456, 377, 426, 354)
CLEANUP: Patches to remove incredibly old files ($LJHOME/doc/todo.txt, which was apparently the minutes of a 2002-era Danga Interactive staff meeting, patch by me; remove code for a feature that hasn't worked on LJ since 2002 and couldn't be made to work as-is without massive work (dreamwidth.org/meme.bml), patch by me; stop showing users the internal backend guts of the FAQ system in some cases (the FAQ number was in the <title> attribute of the resulting FAQ page, which is something users don't need to know), patch by phoenix; and finally, temporarily disable the Schools directory, until we can figure out how to make some usability fixes, clean up the very, very out-of-date and 'dirty' data shipped with the LJ codebase, and work out an admin system that doesn't require hundreds of person-hours a month to handle, patch by me.
51. S2 stylesheets randomly returning 0 bytes (bug 439)
BUGFIX: For those of you who've been using the Dreamwidth site in beta, and who had pasted in a custom style, I'm sure you noticed that sometimes, randomly, your journal style would fail to load and the page would appear unstyled. It finally annoyed Mark enough that he grabbed his debug tools to figure out what was going on and fixed the problem. Patch by Mark.
52. Set up email posts (bug 376)
PAID USERS, USABILITY: This bug was more of a task: set up and configure the ability to make a post by emailing it into the site. Took Mark a bunch of struggling with (and swearing) at our mail transport agent, but he emerged from the field of battle, victorious. I should also note that in the process, my POP sessions to my @dwscoalition.org email stopped working. "Mark!" I said. "You broke it!" He gave me that "...this will not end well" look (I can totally read Mark's looks over the internet by now) and said "What server do you have it trying to POP from?" Yeah, okay, it *does* work better if you have it set to the right mail server. (Not as bad as the time he once spent two hours helping me troubleshoot why mail was failing to my domain, only for me to sheepishly discover that I'd accidentally let the domain lapse. But close.) Config change and mail system wrangling by Mark.
53. Site scheme fails when smaller text size set (bug 183)
USABILITY, ACCESSABILITY: We know that our site scheme behaves badly at larger or smaller font sizes; this was our first change to try to address the problem, and we're still working on making it better. Patch by Jesse.
54. Site-scheme search type elements aren't listed correctly (bug 412)
BUGFIX: This actually didn't require a code change, just reloading the translation strings from the database, but oh, the research it took to figure that out.
55. Site-scheme, English-strip, and modernize /admin/console/index.bml (bug 437)
USABILITY, CLEANUP: This is one of the entries on the "tiny things that have always annoyed me" list: the Admin Console now uses the site scheme (instead of the blank white page with the image of the guy who always looked creepily like Brad, but was not in fact Brad), uses contemporary coding style, and outputs HTML that is not in <ALLCAPS> and ugly. NO MORE NAVIGATIONAL DEADEND ON THE ADMIN CONSOLE. Patch by yours truly.
56. Tell OpenID users to validate their accounts (bug 415)
USABILITY, INTEROPERABILITY: Right now (and we are changing this, but might not get to it immediately) OpenID users are treated as anonymous users for the purposes of comment screening, *unless* they have set and validated their email address, in which case they'd be treated as logged-in users. This patch adds some language on the /openid page to make that more explicit, as part of our efforts to improve the site experience for OpenID users. Patch by me; I'm embarrassed to say that I can't remember who originally suggested it, but I know one of our OpenID site testers did, and I'm thankful that they did, since I never personally used OpenID logins much and wouldn't have noticed it.
57. Text message disclaimer footer gets printed out multiple times (or 0 times) (bug 414)
BUGFIX: Fixes a bug where the text messaging interface really, really, *really* wanted you to know that the reliability of text messaging should not be trusted in dealing with emergencies. Afuna curbed its enthusiasm.
58. Top 1000 syn accounts page is empty (bug 422)
BUGFIX, CLEANUP: Update the "popular syndicated accounts" list to use the watch/trust friends split. Take a look! People are already using syndication, even during open beta, which is so cool: http://www.dreamwidth.org/syn/ . (Yeah, we know that page erroneously references syndication points -- our site copy team will correct that.) Patch by Mark.
59. Tropo: CSS-class, style /support/* (bug 417)
USABILITY, CLEANUP: As part of the default Dreamwidth.org site scheme, Tropospherical Red, we load a CSS file that explicitly strips a lot of default-assumptions that each browser makes about how things like tables should be rendered, because doing that gives us better ability to deal with cross-browser glitches. This has the unfortunate side effect of breaking anywhere on the site that relies solely on tables for layout ... which, given that Brad started programming on LJ in 1998 when the "MUST NOT USE TABLES" recommendation hadn't yet passed into common wisdom, is a lot of places. We've been updating various pages to at least CSS-class the tables so we can manipulate them via CSS; actually rewriting the HTML output to get rid of them entirely is going to be a massive undertaking. Patch by yours truly.
*
COMMERCIAL BREAK: We have a tremendous, tremendous advantage in that we're not starting from scratch to build Dreamwidth; we're working with a codebase that has ten years' worth of bugfixes, features, enhancements, and other improvements. We have a tremendous disadvantage in that we're not starting from scratch to build Dreamwidth; we're working with a codebase that has ten years' worth of byzantine cruft in a lot of places. A lot of what we've been doing since we started work on this project is a *lot* of backend work to rip out the most egregious cruft-built-up-over-time and modernize things, but the way the site actually builds and renders its HTML is among the hardest, because LJ uses a unique templating system known as BML.
BML -- nobody, Brad included, ever decided if it stood for Better Markup Language or Brad Markup Language -- is something Brad built from scratch back in 1997 for his first website venture, FreeVote.com, and he brought it with him when he started working on LiveJournal, because there really *weren't* any other options at the time.
BML has its strengths and its weaknesses; among the weaknesses is that it's not being actively developed to add new features, since it's reached the level of 'stable' enough for LiveJournal's purposes. We're currently evaluating the possibility of ripping out BML and replacing it with another Open Source templating system that will hopefully be much, much easier to use, update, and create site schemes for. Until we can do that, though -- and it's a massive undertaking that we'll probably have to do slowly over time -- we're probably not going to be doing extensive code-optimization to make sure that the code produce the 'right' HTML/CSS, since we'd just have to do it over again when we make the switch; we'll live with the workarounds for now.
*
60. Ugly error message when uploading a userpic of the wrong filetype (bug 160)
CLEANUP: This is one of those bugs that spontaneously fixed itself thanks to some other code or config change. I wish we could get more of those. Maybe the bugfixing fairies came in the middle of the night to pay us a visit.
61. update FOAF ljfeed.pl to use WTF (bug 441)
CLEANUP: FOAF (http://www.foaf-project.org/) is an open protocol to support the automatic discovery of your relationships with other people. With this bug, we updated it to use the new watch/trust friend split. LJ supports this protocol, as well as many other open data-access protocols, and we believe it's important for us to also support as many widely-adopted open protocols as we can, speaking to our commitment to interoperability and our principle that it should always be easy for you to get your data back out of our servers if you decide that you're not for us. Patch by Janine.
*
COMMERCIAL BREAK: One of the things we have on our radar -- not for this year, but we're aware of it -- is supporting Google's OpenSocial API, which is specifically designed to find your friends on other social networks if you want to. For maximum irony, OpenSocial was developed at Google by one Brad Fitzpatrick ... remember him?
*
62. update LJ::Spellcheck to work in Apache2 (bug 307)
CLEANUP, BUGFIX: Remember way up there in #35 I promised I'd talk more about the spellchecker? Here it is! I noticed a few weeks ago that the spellcheck option wasn't appearing on the beta site, thinking that it was some kind of configuration that was misconfigured. Turns out, Mark discovered, that our efforts to modernize the code and upgrade to a more recent version of Apache utterly broke the glue that held together the LJ application and the operating system's spellchecker. Afuna did some heroic work to figure out a replacement option and get it working under Apache2, so we'll soon have spellcheck again. And it will know the site's username.
63. Use common logic for user link bar on profile page and in S2 (bug 400)
CLEANUP, USABILITY: This patch eliminates duplicate code; the code for the user link bar (the standout box that appears to show you the little useful graphics that indicate what actions you can take, like next/forward/track this/add to memories/tag/etc) used to occur in two separate places, not shared; now it's just one place and they both share it. Patch by Afuna. (Who demonstrated her sense of humor in comments #1 and #2 to bug 400, and the entirety of bug 402, which just proves that a). we were getting a bit giddy at that point and b). we are all Star Trek geeks.)
64. Wrap user-generated content in CSS class to apply sane defaults (bug 264)
USABILITY: See #59 for why we include the CSS that strips the styling from many HTML tags. Users will want to use these HTML tags in their journals/profiles/comments, though, and they won't understand why they don't work (because the reset.css file removes any default definitions the browser provides). This is the start of an attempt to let users still use those tags and have them behave normally, without sacrificing the cross-browser benefits of reset.css. Patch by Jesse.
65. [community]/read isn't up (bug 387)
USABILITY, COMMUNITY: As part of the friends->watch/trust split, we changed how community 'membership' works in the backend of the codebase, because the old way of doing it was very much a workaround. (A nine-year workaround!) Because of the updates, community reading pages weren't being populated by public entries from comm members. This fixes that! Patch by Mark.
***
So, what have we learned from this tour through this week's codebase?
* We did a LOT of work this week.
* Most of it was in the realm of code cleanup: stuff designed to fix up and modernize the codebase we inherited, or stuff to fix bugs, either in the code we forked from or the code we've written ourselves.
* Of course, there was also a lot of "...this has ALWAYS ANNOYED ME and now I can FIX IT", which is, I will admit, one of the major satisfaction-bringing things about working on this project.
* Our beta testing team, led by the incredibly capable Domtheknight and full of people who really believe in poking in until it breaks, rocks house at finding things that we never would have noticed.
* The number of open-beta blocking items has finally, *finally* started to creep back downward, as we fix things we've identified as priority bugfixes for launch and people finish up projects that add the last remaining functionality we need to consider ourselves "ready for prime time".
* It really is dangerous when you teach the Suit how to code.
* Most importantly -- or at least, I hope you'd agree -- we are committed to making the site better and more usable for everybody, as much as we can.
I really can't say often enough that our Open Source programmers are doing an incredible job -- both the people who have been programming for years, and the people who are learning how to program, and how to participate in an Open Source project, with us.
***
So, where do we go from here? What's the plan for the next steps in Dreamwidth code development? I spent some time with the Wiki and Bugzilla this week and wrangled mostly everything into shape to answer that question.
Our Roadmap contains a brief summary of our prioritization processes, and includes Bugzilla links that will let you browse through what we have planned, all the way out to the end of 2009. After that ... well, we'll see what y'all will find most valuable and useful, now won't we?
***
Current bug count:
-- 200 open bugs
-- 270 resolved bugs
-- 46 blocking-open-beta bugs
-- 43 blocking-open-beta bugs without a patch waiting for review
***
We'll be back next week with an update in the more regular format, along with updates from areas other than development. I hope you've enjoyed this look through our development lens -- the kind of things we think about when we contemplate what items to prioritize for the various stages of our launch timeline.
The dw-discuss mailing list is your place to go for questions, comments, ideas, and discussion.
For semi-regular updates on what-we-did-today, check out our Dreamwidth Twitter account.
For more real-time discussion, commentary, and the ability to consign things to hell -- or at least, to our irc bot "hEll" -- come join us
in irc:
irc.dwscoalition.org, port 6667, channel #dw
Questions? Comments? Concerns? Donations of spare sleep? (I've been awake for 36 hours by this point!) Comment! (Remember, you can use your OpenID account, and we've even been making it better!)
We'll see you next week, where we will hopefully have passed the threshold of 300 resolved bugs and come out the other side. Just like I came out the other side of Monday in my timezone. Oops.
no subject
no subject
no subject
So, 20 charaters plus "_feed"? *just checking*
Also, I love this format, and I know it took a whole heck of a lot of time to create, but seriously, if there were a changelog-digest like this ~weekly or so, I would subscribe to those Internets. *grin*
no subject
Yup, exactly! 20 maximum characters for the user-given name, and the last five is taken up by the "_feed" for a maximum of 25 actual characters.
no subject
(For example, "example_rss_feed" just looks silly.)
no subject
no subject
Me too... I love getting to actually see what the bugfixes are-- thank you so much for doing this. :)
no subject
... and that all of us, staff and users, are a bit obsessive.
We try to confine the obsession to useful stuff though, honestly!
r
(who has been playing way too much online pyramid solitaire lately...)
no subject
I dont have any extra spare sleep to donate but I'm sure I could find some strong coffee or Mt. Dew for a caffeine push....ooooo or chocolate! Ok, maybe not chocolate. *sneaks away with the chocolate*
no subject
PS. I love this format.
no subject
I do, too.
And it did highlight what a tremendous amount of work a large number of people are putting in to make this work.
Thank you, everyone!
no subject
no subject
Oh god. You're going to be able to add words to the new spellchecker? I love you! It may become actually useable!
46. Reduce maxlength of text field for feed account creation (bug 469)
I love this fix to bits *but* please can it be made explicit in the feed creation process? Otherwise I anticipate a lot of feeds called username_feed_feed created by people who are used to being careful about this sort of thing. I had a look but couldn't see it there at the moment - just sticking _feed at the end of the account name box should do it...
51. S2 stylesheets randomly returning 0 bytes (bug 439)
I thought this had been fixed! Thank you, Mark!
no subject
no subject
no subject
no subject
no subject
(Anonymous) 2009-03-10 10:19 am (UTC)(link)Item 58 of the progress report is a bit mean, btw. ;) The link redirects to the main page. I take it that means one needs to be logged in to view the top feeds. *is accountless*
-Moonie
no subject
(I look at that page, look at this update, see how much we've already done, and it makes me *so happy*)
no subject
no subject
no subject
So you should congratulate yourself a little, too. :D
no subject
Cough, cough, hneerr. :D
45. Redo menu items to be a separate file for site schemes to use (bug 321)
ADMIN, USABILITY: I actually want to geek out here for a second. The *details* of what this patch does (splits out our menu items into a function that any site scheme designer can use, which will mean that we only have to change things in *one* location, rather than across every single site scheme; this means that we can provide a much larger number of site scheme options without dramatically increasing our code maintenance burden) aren't the important part, although is is cool. What I find most exciting about this patch is that one of our programmers, Sophie, was the one who identified the opportunity to make a massive admin-usability and code maintenance improvement with just a little bit of effort. She solved a problem that wasn't even on our radar yet, and did it in a way that'll directly enable us to dramatically increase the number of options we can offer our userbase. That, right there, is why we believe so strongly in Open Source, and why we, as a project, are so committed to mentoring new programmers, empowering and assisting our developers, and being open to patches and improvements from all corners. Thanks, Sophie.
Hurrah for abstraction! Hurrah, Sophie!
BML -- nobody, Brad included, ever decided if it stood for Better Markup Language or Brad Markup Language
I LOL'D.
This is so awesome. We have a website!!
no subject
I toned it down a bit. ;)
no subject
no subject
re 53, the breaking at small/large font sizes. This is committed? Because tropo is still breaking at large font sizes. More specifically, when a larger-than-12 minimum font size is specified in the browser settings.
I know why this is happening and even have a suggestion for how to fix it. Should I open a bug?
no subject
no subject
I do feel very strongly that the default theme should not be fixed width. That will break too easily for too many casual visitors.
no subject
If we do end up having to create a second scheme, we'll make the accessible one the default, and let people choose the 'compact' one (or some other name) if they want.
But we're going to focus on not having to create a second scheme. I'd rather do it all in one.
no subject
(My first comment on Dreamwidth while logged in!! I think Jon Stewart is an appropriate icon. :)
no subject
no subject
no subject
no subject
It is a congratulatory comment, and an impressed one!
no subject
no subject