Hello, Dreamwidth! I come to you with an out-of-cycle news post to announce a super special end-of-year present to all y'all ... but while I've got you on the line, I might as well go through a few other things. Sorry, you're gonna have to wait for the good stuff ;)

Behind the cut:

* Development
* Dreamwidth 101
* Import delays
* Site stability
* Aid to the Philippines
* Holiday points bonus + paid account creation
* A request regarding official DW journal comments

... and ...

* HAPPY HOLIDAYS, DREAMWIDTH, HAVE A SHINY THING (soon)

Development

We start with the 8 Dec - 23 Dec code tour, done by ghoti and brainwane working together!

Most of the development on the list are backend bugfixes, new themes, theme bugfixes, fixes for the new Create Entries beta, and a few small frontend bug fixes and improvements. We've also added in a warning if you're trying to buy paid time for a deleted, purged, or suspended account, and a fix to remove entries from deleted journals from site search results immediately instead of waiting for the nightly search index.

(Speaking of themes: we're always looking for new themes and styles to be added to the existing choices. If you're a designer, or even if you just come up with something awesome for your own journal and want to offer it as a potential system theme, check out dreamscapes -- we offer bounties of Dreamwidth points for every theme and style that gets added.)

Some of these fixes are available on the live site, thanks to a few intermittent code pushes we've been doing throughout the week in order to tweak things for the GIANT INFLUX OF PEOPLE, oh my goodness, seriously, hi everybody! Which brings me to:

Dreamwidth 101

We're really happy to extend a Dreamwidth-style welcome-to-the-neighborhood to everyone who's joining us this week (and goodness, there are a lot of you all!) We have a Guide to Dreamwidth for LiveJournal users to help you find your feet.

A brief overview of some of the fun features we have that you may want to explore:

Finding stuff

* The Latest Things page, a feed of the public posts being made on the site. The tag cloud on top of the page contains the 100-most-used tags (calculated over the previous day's worth of posts, I believe), to help you find interesting content.

* We offer a site search option -- free users can search the entire site, paid users can search both the entire site and restrict the search to their own journal. (You can set your privacy settings for site search at the Privacy tab of Account Settings.) The search indexing runs at about a 12-hour delay, so content won't immediately show in the search results, but it's a fun way to find things -- or a useful way to come up with that one post you KNOW you read three weeks ago but can't remember where.

* For communities, dw_community_promo is a place for people to promote their communities. You can find lists of the most recently created and most recently active communities on the Communities page, and the Community Search will help you if you're looking for something in particular. If you're feeling lucky, you can check out a random active community. And in past news posts, we've done the Community Rec-O-Matic, where people ask for recommendations for communities that may interest them -- browsing those comments for suggestions might also help you find things. (Someone's running an unofficial round right now, but I don't want to overwhelm her personal journal by linking it here! We'll run one in the next news post, though -- I'd do it now, but I have a feeling the comments to this post are going to be full of people squeeing about the holiday present I'm totally teasing you by not mentioning yet.)


Display preference features

* On the Display tab of Account Settings, you can choose which style to automatically display entries and journals in, choosing from among the site skin, your own journal style, the "light mode" style designed to be more helpful for screenreader users and in low-bandwidth situations, or just defaulting to whatever is used for the journal in question. And once you've set that, you'll never need to add ?style=mine or ?style=light to the URL again -- but if you change your mind while you're looking at a page with the navigation strip enabled, the navstrip has an easy link to change the view with one click.

* Oh, and while I'm on the Display tab to talk about things, we've also got a Comment Hierarchy feature, which -- if enabled -- will add a numbered hierarchy to all the comments in your journal, making it clearer which comments are related to which. (I have a screenreader-using friend who says it took her eight years or so of using LJ and DW to figure out how sighted people were figuring out which comment was replying to which! Comment hierarchy is designed to help out screenreader and low-vision users, but it's proved surprisingly helpful for everybody else, too.)

Journal management features

* If you're like me and you wind up with eighty billion variations on the spelling of your tags, or only realize after a year and a half of using it that the keyword on your favorite icon is misspelled and suddenly it starts to drive you bonkers, you'll like these. The Manage Tags page will let you rename one tag to another tag or merge two tags together, and when you change an icon's keyword on the Edit Icons page, you'll be given an option to retroactively apply the change to all previous uses of the icons -- no more changing a keyword and having everything revert to default. (Unless you want it to.)

* Oh, and while I'm on the Edit Icons page: the icon upload form will let you upload multiple icons at once, instead of forcing you to do them one at a time, and you can also write a text description of what your icon looks like that will be read to screenreader users to help them get the sense of what the icon is. (You can see my icons for an example of how to describe your icons.)

* EDIT: HOW DID I FORGET THIS. On the Display tab of Manage Settings, you can set the URL of an entry in your journal to make "sticky" -- it will always appear on the top of the first page of your journal or community. No more messing around with forward dating if you want to have an introduction show up first!


Comment management features

If you've taken a look at the comments section of the entry page, you've probably noticed at least some of these!

* When you're viewing comments, there's a direct link to flat (chronological & unthreaded) mode and to view top-level comments (without also showing any replies) only.

* Paid users have an "expand all comments on page" link in that section, too, displayed once the comments begin to collapse.

* And you've probably already noticed that any non-threadstarting comment has a "thread from start" link at the bottom of the comment, which will bring you to the very top of the thread, no matter how deep in the thread the comment you were viewing is.

* Screened comments have also gotten an upgrade: replying to a screened comment doesn't unscreen it by default (although you can choose to unscreen it at the time you comment), and the person who made the screened comment you're replying to can see -- and answer -- your reply, allowing for entirely-screened comment conversations.

* And, finally, if you add ?comments=screened or ?comments=frozen to the URL of an entry page, you'll see only the screened or frozen comments. (That one, you do have to remember the URL parameters for -- but we'll put it on the page itself soon.)


...And, okay, this is a little thing, but I love it to death and it's been huge in how it's improved how I use the site: you can expand cut tags from your journal or from your reading page without having to load the entry in a separate page, by using the arrow next to the cut or the "expand all cuts" module that's available in the style system. I never knew I needed this so badly in my life until I had it.

Aside from the big ones mentioned in the migration guide, like the importer, the crossposter, and the split of 'friends' into 'I'm reading you' vs 'I want you to read my locked stuff', what other Dreamwidth features do people think newcomers should really know about? Sound off in the comments!

Import delays

As many people have noticed, our import queue is very, very busy right now. (At the time I'm writing, there are 958 jobs waiting to import entries, and 48 waiting to import comments -- and many of those entry imports will then go on to start a comment import once they're done with the entries.) At the rate we're processing imports, that's probably about two and a half days' worth of jobs to do -- meaning, if you started an import right now, it would probably take about two and a half days to complete. And, of course, more imports are being added to the queue as the ones already in the queue finish.

Mark's going to spend this week working on a few things that will hopefully improve the import process, allowing jobs to complete more quickly:

* better handling of the scheduling between entry imports and comment imports, to move jobs through the queue more efficiently, which will mean that your entry imports will complete much earlier than comment imports and comments will arrive up to a day or two later at times of peak load.

* optimization of the comment import process, since comment imports are the real bottleneck: they take more time, more processor power, and more database activity, so they're much slower to complete, especially if you have more than a hundred thousand or so comments to import.

In the meantime, we're asking everybody to be patient! Once you start an import job, you don't need to stay logged into your account or leave your computer running, and you can keep posting and commenting just the way you would otherwise. Just start the job and forget about it, and in a few days, your stuff will magically be moved in for you.

Think of it like how, during a cross-country move, the movers tell you to expect a three-day window for them to get there with your stuff and your furniture. We promise we'll be much more gentle with all the boxes marked 'fragile' than they generally are.

(It's not, unfortunately, something that throwing more hardware at can help much. I really wish it were! Most of the limitations have to do with database access speed and the speed at which we can access LiveJournal without being blocked for hitting the site too quickly. We can optimize the process on our end, and we'll be working on that, but there's a certain level past which you can't speed it up any more.)

Site stability

We've seen a lot of people asking us about whether or not Dreamwidth is capable of handling the increased traffic we're seeing this week, and we wanted to reassure you that everything's ticking along fine. The way we're set up, we're able to add additional server power very easily whenever we need it (we've already done that once this week, since we were very close to the point where we would need to even before the influx, and we're watching carefully to see whether we need to add more) and mark, our primary sysadmin, has years of experience in architecture and maintenance for high-performance web applications.

Of course there may always be periods where the site is temporarily slow, and (Flying Spaghetti Monster forbid) there's always the chance of a disaster -- although we do have disaster recovery and fallback procedures in place. But fear not, we watch site performance very carefully and do everything we can to keep things running smoothly.

Any site hiccups will be posted about in dw_maintenance, which is a very useful community you should keep on your reading page! We also use Twitter as a offsite status page in the event of site slowness or hiccups. We try to be really quick on updating both places, although sometimes we'll hit Twitter for a quick "working on it!" and save the dw_maintenance post for when one of us has a second to breathe.

Aid to the Philippines

As many of you know, Dreamwidth Employee #1, fu, is from the Philippines, which has recently experienced devastating flooding that has killed over a thousand people and left about half a million more without reliable access to food, shelter, and clean drinking water. Fu and her family are fine, but we'd like to encourage anyone who can to join us in donating to the relief efforts.

The Philippine Red Cross and UNICEF are both asking for donations to help the relief effort. Outside the Philippines, the United Nations Central Emergency Response Fund is also calling for aid.

Holiday points bonus + paid account creation

Last reminder, I promise, but I'm repeating it since we have so many newcomers to the neighborhood! For the remainder of 2011, all orders made in the Dreamwidth Shop will receive a 10% points bonus for future use. For instance, if you buy yourself a 12 month paid account (350 points), we'll give you 35 points to spend later once you complete your order. We're also donating 10% of our gross revenues for the month of December to the Ada Initiative, a nonprofit organization formed to help improve womens' participation in open technology and culture -- a cause that's so near and dear to our hearts that (full disclosure!) I volunteer my time on their board of directors.

Also, while I'm here: we've had a few people reporting problems with paying to create an account (rather than upgrading an account after they create it) while we have our open account creation for the rest of the year. This is a side-effect of invite codes being turned off -- usually, paying to create an account will give you an invite code that's preloaded with paid time to be applied when the code is used, and because you don't need a code to create an account, the process is skipping that step.

We've been too busy with other stuff to implement a bugfix for this, especially since we only do open account creation rarely, but the good news is, it's easily fixed. If this has happened to you, just submit a support request in the Account Payments category, while logged into the account that should've had the paid time applied to it. Include the account creation code that was emailed to you, and we'll make sure the paid time gets applied to your account. (If you don't still have the account creation code, just open a request and we'll ask you for some additional information to locate and verify your payment; Account Payments is a private, staff-only category.)

Finally, we've had many people ask us whether we can accept payments via PayPal. We can't, and not by our choice: last year, PayPal demanded that we censor our users' content as a condition of keeping our account with them open, and we chose upholding our Guiding Principles of free expression instead. Unfortunately, that means we can't accept PayPal for payments. We accept Visa, MasterCard, Discover, and American Express credit cards, Visa check/debit cards, Visa gift cards (Mastercard gift cards have a tendency to only work about 20% of the time), and check and money order in US dollars. If you can't make any of those work, we'll also take American Express travelers' checks (make an order to buy points in the amount of the traveler's check, then use the points over time) and cash in US funds if it makes it to us, although we don't recommend the cash because if it goes astray there's no way to trace it. And if all else fails, asking around will usually find you somebody who's willing to be your unofficial "PayPal proxy": you send them the money, and they use it to buy DW points with their credit card and transfer them to you.

A request regarding official DW journal comments

I've mentioned this in the comments to a few of our recent posts to dw_news and dw_maintenance, but I thought I'd also include it in a top-level section in this news update as well: we'd like to ask a personal favor of y'all.

As nearly everybody likely knows by now, the massive influx of new account creation and activity on Dreamwidth this week stems at least partially from a series of changes LiveJournal made to their product that a lot of people are unhappy about and/or have been personally harmed by, and from the dissatisfaction a lot of people have been feeling about the way the release process was handled. Without wanting to get into more of a discussion about the changes and the release process, we've noticed the beginnings of a trend that worries us in comments to dw_news and dw_maintenance, of people praising Dreamwidth by making negative comparisons to LJ or of people expressing their frustrations with LJ's actions.

The praise makes us feel good, of course, but the negative comparisons and the insults to LJ make us really sad, because no matter what decisions LJ's current management team make, the fact will never change that Dreamwidth literally could not exist without LJ. It was the ten years of LJ development, and the experience that mark and I gained in our years of working for LJ, that allow Dreamwidth to exist in the first place -- and we still have a lot of close friends and former coworkers working or volunteering for LJ as well who have been absolutely fabulous about working with us over time, especially during the past month so we could keep the crossposter and importer running through their Distributed Denial of Service attack mitigation. From the beginning, we've tried our best to be really clear that "Dreamwidth vs LiveJournal" doesn't have to be an adversarial relationship at all, and that just because the two sites are developing in different directions doesn't meant that one is better than the other (or that one or the other will be right for everybody -- the two sites are different, and different people will feel more comfortable on one or the other, and that's not only totally okay, it's actually what we had in mind from the beginning).

So, the favor we'd like to ask of y'all: please, when commenting in official DW journals, we'd like to ask you to keep your comments focused on DW itself, not on DW's actions as compared to LJ's actions or on LJ. We're not asking you to change what you're saying in your journals or in your communities! Just that in the official Dreamwidth communities, we'd like to focus on Dreamwidth's actions, benefits, and features in a positive way, and let us stand or fall on our own merits rather than in comparison to somebody else.

We're glad that we can offer a new home for people who aren't happy on LiveJournal anymore, and we're equally happy that LiveJournal can offer a home to people for whom Dreamwidth isn't the right answer. The work we've done to make Dreamwidth more interoperable with other sites (and the work we still have to do to advance those goals further!) is intended to make it so you don't have to lose your friends just because they use a different site than you do. We'd really like for the comments to official posts to maintain that same attitude of interoperability and respect for everybody's choices, so that nobody feels attacked for their choices (whatever those choices might be) just reading the comments for useful information.



And finally, what you've no doubt all been waiting for:

HAPPY HOLIDAYS, DREAMWIDTH, HAVE A SHINY THING (soon)

The increased traffic this week has prompted mark to speed up work on his current project, and we're pleased to pre-announce the introduction of community importing, coming next week. We know this is something a lot of people have been waiting for, so please read the rest of this section carefully!

To import a community, you'll need to be both the admin of the community on Dreamwidth that you're importing into, and the admin of the community on the remote site that you're importing from.

Like the personal journal import system, posts and comments from the remote site will still be controllable by the original poster; they will display as having been posted by the account on the remote site, using OpenID authentication, and anyone can sign in using their remote site's identity and continue to have the exact same amount of control over the posts.

Also like the personal journal import system, you can continue to use the community on Dreamwidth as much as you'd like while you're waiting for the import to finish. The system will pick up all posts and comments that were made to the community on the remote site up to the time when the job runs, but since we can't predict when a job will run and because communities are generally larger than personal journals (and therefore will take longer to import), if you intend to move entirely to Dreamwidth instead of maintaining mirrors on both sites and you want to be absolutely certain that you get everything, we recommend that you move the community activity to Dreamwidth first, then schedule the import and let your posts and comments follow you later. Or, you can do the import first, wait until you've confirmed your posts and comments have been moved, move the community activity, and then follow up with another import in a while once the backlog calms down a bit. (Doing a second import once you've already done one will not override anything: it just picks up anything new that was added on the remote site since your last import.)

And, finally, some slightly downer news: because of the traffic this will no doubt add to the import queue, and because (as mentioned above) the import queue is very, very backed up, we are restricting the use of the community import tool to paid accounts only for a limited time. We promise the feature will be available to everybody as soon as possible -- probably within a week or two, although if there's more demand than we were expecting, it may be longer -- but we're restricting it now in order to help manage some of the load. So, for right now, either the account doing the import or the community being imported into needs to be a paid, premium paid, or seed account.

We're also going to have to place a temporary restriction on imports from communities with lots of comments. (This is because the comment import process is more load than the entry import process.) When we release, if you try to import a community with more than 100,000 comments received, the system will cancel the comment import job (and only import your entries instead). We picked that number to be on the safe side. Again, we hope to be able to lift that restriction as soon as possible, and as soon as Mark has a chance to streamline the importer some more.

If your community has more than 100,000 comments received, or if you're willing to help save the queue a little and import entries only, fear not: you can import your entries now, and import your comments later. There won't be any duplication from importing again, and comments will automatically be put on the correct posts.

We'll tell you in a future dw_news post when we're able to lift these restrictions.

We were hoping to be able to release the feature tonight, but between the need for some last-minute troubleshooting and the fact that most of the DW team has family and holiday obligations that would slow us down on handling any problems that might crop up, we've decided it's better to delay a few days and make the feature public early next week. Keep an eye here on dw_news: we'll post again when it's ready for you to try out.


*

That's it for us for another news post! Because of the upcoming holidays, we'll likely be very in-and-out for the next few days -- all of our families do Christmas-ish stuff, and then of course there's New Year's Eve and New Year's Day. We'll be watching for any major issues, but non-time-sensitive issues will likely be a bit delayed! (Unless we're trying to hide from familial overload, in which case you might be seeing a lot of us.)

As always, if you're having problems with Dreamwidth, Support can help you; for notices of site problems and downtime, check the Twitter status page; if you've got an idea to make the site better, you can make a suggestion.

We'll see you in a few days for our next mini-update, and in the new year for our next full one.


EDIT 12/26 11:45pm EDT: Hey, guys, as a personal favor to me, before asking questions in the comments could you please try to read the existing comments? I know there are many, many comments to read through, but most of the questions I'm seeing now are repeats and duplicates, and there's only one of me to answer them all. Thanks. :)