Dreamwidth Update: 2 February 2012

[denise]

Hello, Dreamwidth! It's been a busy month for us, and we have some nifty things to announce. Behind the cut, we bring you news on:

* Development
* Community Imports
* Claim your OpenID
* A note from the antispam team
* Invite Codes
* Abandoned Accounts
* Volunteering with DW
* State of the Servers

Development

You can see the development work that's been done since our last visit in our code tours:

5 Jan - 26 Jan
23 Jan - 1 Feb

Some of these improvements are live on the Dreamwidth site, and some will be live after the next code push. Live right now: you can now choose whether your journal's archive starts the week on a Monday instead of a Sunday; the keyword sorting on the Manage Icons page is case-insensitive so uppercased keywords don't get sorted before lowercased keywords; the Inbox has a separate filter for community membership requests so you can only view those notifications; a bug with logging out got fixed so you only have to click the log out button once; the "random icon" option when commenting is now a button instead of a text link; and (my total favorite; this has been bugging me for ages!) when previewing an entry with a poll in it, the poll will display as a mockup of an actual poll and not as text only. Live after the next code push: the ability to link to your Plurk, Tumblr, and Pinboard accounts on your profile; the ability to use Fanfiction.net and Pinboard accounts in the <user name=foo site=bar> tag; the addition of a link to manage your reading filters (in addition to the existing link to manage your posting filters) on the success page you get after making changes on the Manage Circle page; adding comment outline indicators to comments when viewing an entry in custom comment pages; and a bunch of new themes for choosing in the journal customization area.

Meanwhile, kareila, our stats goddess, has run the stats on contributors for this quarter. Since the end of October, we've welcomed six new code, layout, or theme contributors: chagrined, delladea, flatlanders, hope (who's done some design work for us before as well -- she's the one to thank for how awesome the beta update page is!), sevilemar, and stormy. We've had 43 unique contributors contribute to DW over the past year, which is awesome. If you're interested in becoming one of them, check out information later on in this update.

Community Imports

We'd previously enabled community importing for paid communities (either paid comms, or paid maintainer accounts) -- we limited it to paid accounts only at first in order to rate-limit the feature and to make sure we caught any glitches with a "soft" rollout.

Happily, we're pretty confident that the performance is as optimized as we can get it, so we've now opened up community importing to all accounts. To import a community, you must be the maintainer of the community on the site you're importing from, and the admin of the community on Dreamwidth. Visit Import Content, select the community you want to import into in the "Work as User" dropdown, input your username and password on the site you're importing from, and input the name of the community you want to import.

We expect that the import queue will get pretty busy over the next few days as people schedule imports of their communities, so please be patient if your import seems to be taking a while. You'll receive a message in your Dreamwidth Inbox when the job completes or fails (and if it fails there'll be an error message explaining why). Until you get this message, the job is still in queue, and it will run in time.

Claim your OpenID

We've rolled out the first draft of our "claim your OpenID" feature, and after some initial troubleshooting, we're ready to release it to a wider audience. This will allow you to update posts and comments that were attributed to your OpenID account so that they appear to have come from your DW account instead.

To claim your OpenID:

* Visit the Claim your OpenID page while logged in as the account you want to claim the OpenID with. This is the account that the posts and comments will update as having been posted by.

* Put in the URL of the OpenID account -- for instance, if you want to claim comments that show up as having been posted by exampleusername.livejournal.com, you'd enter http://exampleusername.livejournal.com.

* You will be forwarded to the remote site to verify your identity. (You will only get this step if you haven't previously told the remote site to always verify you to Dreamwidth. For instance, if you logged into Dreamwidth using your LiveJournal OpenID before and told LJ to always authorize DW to have access to your identity, this step will be skipped; the authentication will happen quietly and automatically in the background.)

* After a delay of about an hour (to give you a chance to change your mind) you'll get a confirmation link in email. As soon as you click that link, your OpenID will be claimed. The process is not undoable, so be sure you're logged into the right account, and enter the right OpenID URL, before you click the confirmation link. Clicking the link will activate the process.

* Once the process has been started, the worker will happily work its little way through the job. It's a pretty database-intensive operation, and it'll probably get to be pretty popular as soon as I post this, so it will take a while for everything to be updated.

You only have to claim your OpenID once. After you've claimed it, any new comments or posts that are imported will automatically be updated to use your DW account instead of your OpenID.


A few notes:

* The "comments posted" count on the profile won't be updated to reflect the new numbers. (That data comes from a different spot in the database.) We'll probably run something that will slowly update those numbers over time once the initial demand slows down.

* If your username on the remote site has an underscore (like_this) in it, and your OpenID account was created back before a certain point in our closed beta period, the claim will most likely not identify all your posts and comments. (Please note, you do not need to have logged into the site with your OpenID yourself: if someone imported a journal you'd commented in, the OpenID account was created automatically in order to properly attribute your comments.) mark is aware of the problem, but it's a very complex fix; it will take us a few months to solve the problem the correct way. In the meantime, if you're affected by this problem, we will soon be able to fix it manually for you. Open up a support request in the Account Payments category (which is really the "stuff staff needs to deal with" category!) from the account you want to claim the OpenID with, and we'll respond to you with further information once we're able to make the backend/admin tool that will handle this. (It may take a bit!)

* If your account on the remote site has been deleted and purged, or if you've renamed your account and chose not to forward your username, there's no way to authenticate against the remote site, and therefore you won't be able to claim your OpenID on Dreamwidth.

* If you receive an error saying "the provided URL doesn't declare its OpenID identity server", that means you'll need to edit your style on the remote site to include the proper header information. Instructions on doing this on LiveJournal can be found in their FAQ; the process should be roughly the same on InsaneJournal. Some people report needing to make a post to their journal after making the change in order for the change to take effect.

* Some people who did this earlier may have had problems with the last letter of their icon keywords getting chomped off. We've fixed the underlying bug (after a marathon debug session by mark), but if your account was affected, you'll need to reimport the journal (or have the admin reimport the community) that the comments were made in for the icons to display properly.

* Again, this process can't be undone. There is no way to unclaim an OpenID once you claim it. Be absolutely sure you're logged into the right accounts.

A note from the antispam team

The friendly neighborhood dw_antispam team would like to let you know that if you receive spam via private message, you should hit the "mark as spam" link on the site itself, in your inbox. That puts the message into the antispam system, where our antispam team can leap gleefully upon it while yelling battles cries as appropriate to their own cultural heritage, etc. (No, seriously, they leap. It's kind of terrifying.)

Please do not use the "mark as spam" function in your email (which may result in Dreamwidth emails being blacklisted by your email provider), or report the spammer to the Terms of Service team (who'd really prefer, on the whole, sitting down with a nice cuppa rather than leaping and hollering). If you report it to the Terms of Service team it will still get handled, but using the Mark as Spam link in the Dreamwidth inbox is the fastest way to make the spammer disappear.

Invite codes

As people have no doubt noticed, invite codes are still off! We currently believe we'll be leaving them off for a while, as long as the spammers continue to mostly leave us alone and the percentage of paid accounts remains at a sustainable level.

We're still having the problem where if you pay to create an account, you're then emailed a code that's "pre-loaded" with the paid time. Right now, you don't have an opportunity to apply that code to the account when you create it: fu is actively working on a fix for that and it should be fixed by our next code push, but in the meantime, if this has happened to you, open a support request in the Account Payments category with the link you received in email and we'll take care of manually applying the paid time for you. (If you don't have the link anymore, not to fear: just let us know the cart number of the order, or the full name the payment was made under and the amount of the payment.)

Meanwhile, some people have reported problems with reCAPTCHA, the service we use for the "human test" while creating an account. CAPTCHAs are awful for accessibility, but they do help to keep spammers out. We've found an alternate CAPTCHA provider that gives "plain language" text CAPTCHAs, which will be much more accessible (while, correspondingly, slightly easier for computers to solve, but the difference should only be slight). fu is working to implement this solution and we hope to have it by the next code push.

Text CAPTCHA will be potentially harder on non-English speakers, so we'll be keeping the option for you to use reCAPTCHA on your journal, instead of Text CAPTCHA, if you have CAPTCHAs enabled as an antispam measure, but we'll probably be switching to Text CAPTCHA for the site itself. Meanwhile, if you're having problems (or someone you know is having problems) creating an account because the CAPTCHA isn't showing for you, email support@dreamwidth.org with your desired username and date of birth and the lovely people there will hook you up.

Abandoned Accounts

We've gotten a few questions about accounts that look to be abandoned -- no posts, no comments -- asking if we ever take away accounts that people look to be "squatting". The short answer is no, we don't: generally once you register an account, it's yours until you choose to delete it. This is because we operate on a "first come, first served" basis: when you register an account, we won't take it away from you, no matter if you use it or not.

I say 'generally', because there are a few times when we'll make an exception. If someone is 'squatting' on hundreds of usernames and isn't using the accounts, we may sometimes rename the accounts to free up the usernames. (Please don't sit on hundreds of usernames that you don't intend to immediately use! It's rude to other people: usernames may be a first come, first served resource, but if you aren't intending to immediately use the account, sitting on usernames that others might want to get a chance to register is disrespectful to the community as a whole, and trading or reselling accounts is against the Terms of Service because a traded account is inherently insecure because the original owner can take it back at any time.)

Making those decisions is hard, and we don't want to set hard limits on what "acceptable" use is (it's possible that one person could have 20 accounts, all of which are being squatted with no activity at all in any of them, while another person might have 100 accounts but use each of them on a regular basis). So, things are very much handled on a case-by-case basis.

Still, the answer to "can I have this apparently-abandoned account" is almost always going to be 'no'. Unless there's mass username squatting going on, we won't do anything about accounts that were registered, then apparently abandoned: someone might be active on the site in ways that don't show up on a casual glance (someone with no posts, no comments, and nobody in their circle might still be using the account only to read journals by visiting them directly, for instance), and it's always possible someone might change their mind and come back. (We've gotten plenty of comments from people who created an account when we opened to open beta in April 2009, left it alone for a while, and then came back to us a few months ago and were delighted to find their account was still waiting for them!)

Volunteering with DW

Dreamwidth is a very small company -- there's just me and fu working full time, mark working part-time/evenings (and he and his partner have a newborn and a toddler, too!) and a few people working part-time on a contract basis. We're able to keep this place lurching along thanks to the efforts of a number of dedicated people contributing their time, effort, and energy to helping to make DW great.

If you'd like to pitch in, we'd love to have you. There's a number of different project teams for different areas of interest:

Development: Check out dw_dev and dw_dev_training for information, and the Development category of our volunteer wiki. We offer hosted developer environments so you don't have to install the code yourself (trust me, you don't want to have to install the code yourself), and the list of things that are available to work on is in our bug tracker. There's something for every experience level, from "complete beginner" to "been doing this for 20 years"! We do expect people to be fairly good at self-education, but we're always willing to answer questions and mentor.

Design: We can use help from both user experience designers and graphic designers. The community is dw_design, and you can get a list of the bugs that need a designer's attention with the needs-design Bugzilla query. We're happy to take sketches, wireframes, or full mockups to improve anything on the site; we know there are some sections of the site that haven't been touched since they were initially coded years and years ago, and working to unify everything with one single visual vocabulary is one of our ongoing projects.

Documentation: The eternal bane of any product (the docs are never up to date). Right now, we most need people to go through the FAQs and figure out what's not in them: what we've changed, features we've added, features that aren't documented, and things you can do that aren't documented anywhere. This effort is taking place in dw_docs.

Styles and themes: We're always looking for more pretty things for the Customize Journal area! If you've made a new style that you want to submit as a system style, or if you want to contribute a color theme, check out dreamscapes. For information about the style system, check out dw_styles. For general help in getting your style to do what you want it to do before you submit it, check out the unofficial style_system community.

Support: The support board is always in need of people to answer support questions! Most technical support is done by users, for users -- things that require more complex troubleshooting or action by site owners get forwarded further on to us, but questions about basic information and how things work get handled by volunteers. There's a basic screening process, so answers are checked over by more experienced volunteers before they're sent to the user for accuracy. If you're interested, check out dw_support and dw_support_training.

Suggestions: We know that we don't always know all the ways that people are using the site -- it's easy for us to get locked into ideas based on how we use the site, and we have some data about how people-in-general use the site, but there are a thousand different ways to use DW and the things people come up with never fail to astonish us. So, if you see something that would make Dreamwidth better, we want to hear about it. You can make a suggestion, which will then go into the moderation queue of the dw_suggestions community for discussion, refinement, and potential pitfalls. For more information about the process, and for what kind of discussion we're looking for, read the dw-suggestions User Guide. Anybody and everybody can review and discuss suggestions -- the final decision is ours, but we want to hear from you before we make that final decision!

State of the Servers

We've had some slowdowns in the evenings (US time) the last few days. This is related to our nightly maintenance and to the server load (it happens in the evenings because that's when a lot of our maintenance tasks run). mark and our backup sysadmin alierak are working on optimizing the servers, and we're starting in on some efforts to optimize the code. (And if you've got experience in optimizing web applications, we'd love to have you come and help! mark, who's our optimization guru, is being pulled in eight thousand different directions, and having a newborn around is not always conducive to hack time...)

Fixing this kind of problem takes a while: there are lots of little things we can do (and we're doing them) but load-related problems and optimization is a thousand-headed hydra. (Cut off one head and two grow back!) We'll stay on top of it, but if you notice 10-15 minutes of site slowness in the evenings, that's what's causing the problem.

Meanwhile, if you'd like to help out without knowing anything about code or optimization: one of our biggest load problems is entries that have a high number of comments -- loading those entries causes a big hit -- and we've been seeing an increase in the number of entries with lots of comments lately. If even a few high-traffic journals or communities started opening new posts when comments reached 2000 or 2500 instead of letting the post run until it hit the maximum number of comments, it would definitely help.

We're giving heavy attention to optimizing that particular use case, but depending on how the optimization work looks, we may have to tweak the number of comments an entry can receive downward temporarily. Keep an eye on dw_maintenance for more updates.

*

That's it from us for another update! As always, if you're having problems with Dreamwidth, Support can help you; for notices of site problems and downtime, check the Twitter status page; if you've got an idea to make the site better, you can make a suggestion.

I'll leave you with a nifty toy that exor674 coded, using public data, to let you find how you're connected to other users: Six Degrees. It's using data from the middle of January or thereabouts, so it's not completely up to date, but it's a fun toy to play with!

We'll see you in a few weeks for our next update!

Comments

[malurette]

Community Imports - OMG YES thank you so much for making it happen!!

[malurette]

Wait, what, how comes I am the first commenter here?

[rainne]

[ilyena_sylph]

♥ !!

Yes, that is all I have to say today.

Just. ♥

[anothercopycat]

I keep throwing money at the screen, but nothing happens.

Seriously, you guys are amazing. Never been more satisfied with a home change in my life.

[instantramen]

News post and suggestions on the same day? It's like a minor holiday!

[musyc]

And a code tour! XD

[moonreviews]

I think the "Claim Open ID" one is really interesting! I immediately did it, it might be useful because of all the importing from LJ :)

[wombatqueen]

Can't believe I have to ask, but exactly how do you do it? I tried using my account ID, and I got a stra
strange error message

[dragonfly]

I tried to claim my OpenID but LiveJournal gave me this error message:

"The site you just came from seems to want to verify an identity that you, as [info]dragonfly.dreamwidth.org, cannot provide. "

[denise]

You need to be logged in to the LJ account that corresponds to the OpenID you want to claim. That error message sounds like you're logged into LJ with your DW OpenID instead!

*looks around*

[disprove]

You're earning a lot of good will from your user base--it's well-deserved methinks, and so far, you've been using it wisely.

I hope the comms I follow Over There would just move already...

[stridin]

That picture of Mark and bb is just one of the cutest things I have ever seen.

[denise]

I KNOW. OMG SO CUTE.

[akito]

Thank you for all your hard work! It's still such a shocker to be part of a community with such easy to contact staff. I look forward to continuing my paid services with you guys! ('u') ♥

And congratulations to Mark on such a lovely newborn!

[trobadora]

Some people who did this earlier may have had problems with the last letter of their icon keywords getting chomped off.

A question - how can we find out if we've been affected by this?

[denise]

There's no easy test other than looking at the imported comments/entries, but it appears to only have affected people with particularly long keywords that contained at least one non-Latin-character-set character in them.

[kordella]

thank you so much for everything ♥ you guys are amazing!

[daphnie_1]

Awesome post! :D I am loving the 'open-ID' thing! It'll be nice to have all my comments and stuff all under the one roof :3

[crazed_delusion]

All this is amazing, so much love. Can't wait to get paid so I can renew my paid account♥

[telesilla]

I've been on DW for what seems like forever and yet, every time you post an update, I marvel again at DW's commitment to transparency. It's just fantastic to know what's going on and I really appreciate the effort you and the team make to keep your users informed. :)

[alchemy]

(Probably expected) Question on community importing: Has the restriction on communities with over 500,000 comments been lifted as well?

[mark]

Yeah, go for it. Just keep in mind they take a while -- some with millions take a few days to fully import. (We have to go slowly so as not to hit the remote sites too hard, and not to overload the databases on our end.)

[inoru_no_hoshi]

I love reading news posts so much I accidentally ignored IRC for 15 minutes. How's that for silly? :D

I may have, btw, snickered at the description of the antispam team. My mental image was awesome - there were swords and furred armor and everything. It was like a scene out of Conan the Barbarian! *amused*

And yay for volunteer calling! \o/ :D ♥

[geminianeyes]

Yes, that was the same image I had in my head too! :D

[hellburner]

Can I just throw my money at you?

Please.

Take all of my money.

[mark]

<3

[momijizukamori]

COME TO THE STYLES SIDE OF THE FORCE wrestle with the beast that is S2 with me.

I'd promise cookies but we need to solve that Food over IP problem first.

[inoru_no_hoshi]

COOKIES.

Oh wait. I'd have to learn to code. NO COOKIES FOR ME. ;_; xD

[guiltapalooza]

Ohhh yes please on having the comments posted count in the profile updated to reflect OpenID claiming!

Obviously it's a minor thing, but I'm proud of my comment counts, darn it.

I'm confused.

[ajeckaea]

"As people have no doubt noticed, invite codes are still off! We currently believe we'll be leaving them off for a while, as long as the spammers continue to mostly leave us alone and the percentage of paid accounts remains at a sustainable level."

What exactly does this mean? We can't invite anyone right now?

Re: I'm confused.

[instantramen]

It just means you don't need an invite code to create a new account.

[mercilynn]

The love I have for this post, I can't even begin to describe! Just. YOU GUYS ARE AMAZING!

There would be sparkle text but I'm sadly at work. SO CAPS IT IS. :D This is going to be awesome for moving our RP over here!

[prof]

Great changes! Thanks for the update.

If I might inquire: Are a la carte icon packs still on the development horizon somewhere? It's the #1 feature that most of my friends and I are hoping to see soon.

[mark]

Yes yes! It's my next big thing, now that OpenID Claiming is out. I have one more small thing in the middle then it's a la carte stuff. Well, I may be sidetracked by making stuff faster, but still. Yes.

[warriorscribe]

Being able to claim my OpenID posts is great, thank you!

And of course, thank you for being so transparent about everything you do. You guys are awesome, and I'm glad my comm moved here.

[denise]

You can thank mark directly -- he's the one who's been coding his fingers down to stubs working nonstop on things this month! I'm only the messenger. *G*

And we're glad to have you too. I hope you continue to enjoy it here!

[mistyzeo]

This is just super!