Dreamwidth Update: 26 September, 2012

[denise]

Hello, Dreamwidth! It has been a busy busy week in these parts, and we are here to bring you a collection of new and interesting things you can play with and information about the awesome things we've been up to.

Behind the cut:

* Development
* Technical backend improvements
* S2 siteskinned comments beta
* Create Entries beta updated
* Site skins now HTML5
* New styles and themes
* Email Notification Woes
* Import Problems -- Fixed, But Tell Us Anyway
* Paid Time
* Miscellaneous Other Stuff

Development

We did a code push this weekend, so there is so much stuff new and improved that I have to share with you, omg. I'll go through some of the really awesome stuff in a bit, but if you want to see everything, we have code tours:

13 July - 25 Aug
26 Aug - 19 Sept

I also want to call out a welcome to new contributors carisma_sensei, dunvi, kunzite, and swaldman!

Some highlights of bugfixes and new features:

* Poll responses will now be sorted chronological by response order, not by the userid of the responder.

* Pages that have a "work as community" dropdown will also now have a statement of what user you're currently working as.

* Emails generated by actions taken in moderating communities will always include a copy of the entry that's being accepted (previously it only included the post if it was being rejected).

* Until we can take some more steps to make our morass of settings make more sense (it's definitely something we want to do!) there will be better linkage between the Manage Settings page for a community and the community posting settings page.

* We've expanded the Links List in your journal so you can now specify a title for each link (to be displayed on mouseover) in addition to the link and the link text.

* We made it clearer that the "Delete All" message in your inbox only applies to the particular subfolder you're looking at, if you're looking at a subfolder.

And many, many more, including:

Technical backend improvements

(Which sounds super, super dirty, I know.)

We've been working our fingers off for the past few months to make it possible to switch from our old version control system (Mercurial) to our new version control system (Git, hosted by GitHub). We also took the migration as a chance to fix up a ton of the process that annoyed the crap out of us (and made it hard to explain things to newcomers), and a bunch of stuff that worked great thirteen (!) years ago when Brad Fitzpatrick first started development on LiveJournal but that other people have come up with better ways of doing in the meantime.

Special shout-outs go to fu, mark, exor674, kareila, pauamma, foxfirefey, and ninetydegrees, all of whom did a ton of work in making the change possible, documenting the new process, or helping us to find glitches in the new process so we could fix them, and to cesy for a ton of useful advice. This changeover has been several months in the making, and I'm sure I've missed a few people who also helped out (and I apologize to anybody I left out!) It's been a really awesome example of everybody gamely pitching in and learning new stuff left and right in order to make the process better for everybody.

S2 siteskinned comments beta

One of our long-running backend cleanup tasks is, over time, to replace the existing templating language that builds the pages on the site with a newer, more modern templating language.

At the time we forked our code from LiveJournal, most of the pages on the site except for individual journals were generated by a custom templating language known as BML, which Brad invented for the site he ran before LJ (because at the time there really wasn't an alternative) and carried over to LJ. BML hasn't been under active development in years, though, because it got to the point where it was good enough for the site and it wasn't worth spending the time to make it better. In the intervening years, several other good templating languages have arisen, many of which have much more widespread adoption, better documentation, and more modern features. So, we're slowly converting pages from using BML to using Template Toolkit, the new templating language we've chosen.

Individual journals, however, are generated by a custom programming language called S2 (if you've ever created a custom style, on LJ or on Dreamwidth, you've probably dabbled a bit in S2). Except! If you had custom comment pages turned off in your journal, so that your entries and comment pages showed in the site skin instead of in your journal style, the entry or comment page was generated through BML and the site templating language.

(It was really, really, really old code, too. Like, really. Really, really, really old.)

Having the two separate ways to generate an entry page is super inefficient, because it means that when we change something in one, we have to change it in the other as well. (And, as people have no doubt noticed, we haven't been all that great at keeping them synchronized up.) So, we're working on making it so that the comment page is generated by S2, no matter if you're viewing it in the site skin or not. This has meant a lot of work to make the S2-generated entry/comment pages match the old site-skinned comment pages, and it's been a daunting task. exor674 did a lot of the backend changes necessary to even make it possible, with help from fu, and momijizukamori did most of the CSS and S2 work to create the actual views, with help from kunzite.

Changes like this are super huge, and there's always the chance that we missed something, so we're running the converted comment pages through our beta features system first, in order to make sure we didn't miss anything. If you'd like to help us out by testing, go to the Beta Features page and scroll to the third of the three beta tests, labeled "New S2 Comment Pages". Bugs or display weirdnesses can be reported in the entry for the S2 comment pages test in dw_beta, and you can see the current list of known/fixed issues that we've identified.

Create Entries beta updated

A while back we rolled out the beta version of our new Create Entries page, which will eventually become the page you use to update your journal. (We're redesigning it because the existing update page really won't work well for things we want to add in the future, like scheduled posts, draft posts, more integrated media hosting, and other spiffy new features.) It's a super huge undertaking, because as part of all this, we've had to pretty much reinvent a ton of stuff on the backend to make it possible. We've been tweaking it and fixing bugs with it based on your feedback for a while, as well as working on the last few things that haven't been implemented yet, and this push added one of those things that had been missing: the ability to edit your entries in the new form (as well as posting entries).

We still haven't implemented a rich text editor (which has proven annoyingly hard), draft entries of any type (including auto-saving of entries you're typing), or the "don't autoformat" button (although if that's the only thing keeping you from switching over, you can surround things that shouldn't be autoformatted in <raw-code>...</raw-code> tags), so if those things are really important to your use of Dreamwidth, you should hold off on enabling the beta version. But we've fixed a ton of the bugs that people have reported, too, so if you tried it out and ran into bugs with it, you might want to try again.

To enable the beta Create Entries page, go to the Beta Features page and scroll to the second of the three beta tests, labeled "New Create Entries Page". Once you've enabled it, visit the new page and hit the gear icon in the upper right hand corner of the page () to turn individual panels on and off, and to drag and drop the panels to rearrange them to your satisfaction. Bugs or display weirdnesses can be reported in the entry for the Create Entries editing page.

Site skins now HTML5

Thanks to the tireless efforts of deborah, all of our site skins are now written in HTML5, the newest version of the HTML spec. This is so that we can make the site better for people who access it with assistive technology, and also to take future advantage of the really flexible and powerful things that have been added to the markup language.

I did totally forget to let people know in advance of the codepush that any custom Greasemonkey or Stylish skins that people have created would probably break a little -- I'm sorry for forgetting to mention that! If you're using a custom skin, ninetydegrees commented with the CSS selectors that changed for you to use in changing your custom skins.

New styles and themes

This code push also included a screaming whackton (that's a technical term) of new styles and themes, thanks to the amazing work of the people in dreamscapes and momijizukamori and ninetydegrees, theme-patching machines. This push included over 100 new themes, and four entirely new styles: Ciel by carisma_sensei, Corinthian by momijizukamori, Librarian's Dream by branchandroot, and Paper Me by cimorene.

With this push, we've surpassed over 1000 available themes for your journal -- 1,120, to be precise. That's kind of freaking awesome.

There's an overview of the new stuff in dw_styles. If it's been a while since you've checked out the available themes and styles for your journal, we've added a "New" tab in the Select Journal Style area, which will hold all themes that have been added in the last two code pushes. You can also view the base styles, and if you click on the style name, it will show you all themes that are available for the style. Once you've picked a theme, the Customize Journal Style page will allow you to tweak it to your heart's content, from choosing where to put the sidebar (or how many sidebars you want, if any) to changing the colors to more.

Also, I'm pleased to announce the (belated) results of our polling for the new default theme for new accounts! Newly-created accounts past this point, until our next change, will use the "Early Edition" theme for the Five AM style. This won't affect accounts that have already been created, but newly-created accounts will use that style and theme for a while.

Email Notification Woes

Several Gmail users have let us know that Gmail appears to be marking email notifications from us as spam again. This happens from time to time: because of the volume of email we send out, and because of the @dreamwidth.org forwarding email addresses (which get a lot of spam, and therefore looks like we're sending a lot of spam), many providers will give us the side-eye of You Are Possibly A Spammer, You.

There are some things we can do whenever this happens, and we do do them, but because email is kind of a deeply flawed protocol (because it was created back in the day when if somebody was doing something stupid with it, their network admin could walk down the hall and whack them upside the back of the head and tell them to stop being stupid) it is super hard to make sure that it will work for everybody.

So, if you use Gmail and you've been missing email from us, check your spam folder: it's probably there. You can sometimes work around this by explicitly adding "dw_null@dreamwidth.org" to your Contacts, or by setting up a recipe in your inbox so that email from that address is delivered to a sub-folder of your inbox (we've been told that Gmail handles filtering recipes before they spam-filter, and assumes that any mail you've set up to divert into a separate folder is wanted mail). You might also want to contact them and let them know that their spam-filtering algorithm is depriving you of wanted mail, and make sure to check the "this is not spam" on any mail that has been diverted into your spam folders to help train their detectors.

Meanwhile, especially if you're using a huge webmail provider, please be careful not to mark email from us as spam, since many systems will learn from that over time. If you want to stop getting email notifications, you can change your settings on the Notifications tab of Account Settings. Likewise, if you get a spam comment to your journal or a spammy private message, use the "delete and mark as spam" option on the site (which will send it to the vigilant dw_antispam team) instead of marking the notification itself as spam inside your email program. (And the antispam team would also like to remind you that while it's best to report spam if you aren't sure if it's spam or not, people breaking community rules should just be deleted, not deleted & marked as spam!)

Import Problems -- Fixed, But Tell Us Anyway

A few weeks ago, people were reporting a problem where they'd request an import from another site and the system would tell them the import was successful, but comments weren't being imported.

We've found and fixed the thing that was causing the issue, but if you were one of the accounts that were affected and your comments still haven't been added to your account, we need you to tell us about it. mark was able to automatically fix some of the accounts that were affected, but not all.

If you're one of the people whose comments are still missing, or who got duplicated entries when doing an import, please open a support request in the Importer category to let us know. (I did contact a few people who had open support requests already, so if you already got a support reply from me, you don't need to do this again.)

Paid Time

It's been a while since we've said a public thank you to the people who make it possible for us to keep this place running! We have no advertisers, no venture capitalists, and no outside investors: our only source of income is your payments to us. That means that we're focused 100% on doing things that make you happy, not pleasing third parties who don't even use the site themselves.

If you'd like to show your support for us and to make it possible for us to do even more awesome things in the future, you can buy paid time, for yourself or for a friend. You can also buy extra icon spaces that you can use whenever your account is paid, or Dreamwidth Points that can be used in the shop in the future or given to a friend later.

A side note about paid time: in the past, we've transfered paid time from one account to another for anyone who asked, as a favor for the people who supported us. Unfortunately, paid time transfer has to be done manually right now, and the number of requests has really exploded lately -- it's started to take up a significant portion of my time. We're going to have to start placing some more restrictions on paid time transfer, so if you aren't sure whether you're going to keep using an account for a while, you may want to start buying paid time in smaller increments or buying a batch of Dreamwidth Points and then applying them to paid time one month at a time or so. (There's no real penalty for buying paid time in smaller batches: it costs roughly the same on a per-month basis, with a few pennies' difference here and there to make the numbers come out even.)

Miscellaneous Other Stuff

A collection of stuff that isn't involved enough to get its own header:

* mark and I will both be attending linux.conf.au 2013 in Canberra, Australia at the end of January! We'll both be speaking: he'll be delivering "Systems Administration: Server and Process Behavior" on Thursday, and I'll be delivering "Beyond Alt Text: What Every Project Should Know About Accessibility" on Friday. We've both been to LCA before and found it to be an awesome conference full of awesome people, so if you're in a place where you can come, you totally should.

* Back in December, we pledged 10% of our revenue that month to the Ada Initiative, a nonprofit organization that works to improve the representative of women in "open stuff", including open source software. You can read their blog post about our donation! (I believe in what they're doing so much that I donate my time to serve on their board of directors in an advisory capacity.) I went to their AdaCamp DC unconference a few months ago, and it was one of the most awesome conference experiences I've ever had. The Ada Initiative is holding an individual fundraising drive at the moment to allow them to keep doing awesome work; if you have some spare cash, please consider donating to the cause.

* One of the things we care about a lot is making Dreamwidth accessible to people who are using all kinds of assistive technology -- it's one of our primary goals for the site! This is something you can help with, too. If you have some spare time, visit your Manage Icons page, and take a few minutes to fill out the "Description" field for a few of your icons. That field is used as the "alt" text every time the icon is displayed, which is the text that screenreaders read out to people who are listening to pages instead of viewing them. Good descriptions are short and succinct -- you might explain what's happening in the icon, describe the mood the icon's intended to evoke, or transcribe the text. Check out my icons page for a sampling of some of the approaches.

* Because I've just been all super dorky all over our irc channel about how amazingly awesome all of our volunteer contributors are, I wanted to repeat that here! kareila has just compiled our quarterly(ish) list of code, theme, and styles contributors (and it's awesome to see how many people are currently active!) and you can see the list of people who have gotten "support points" for helping out in the public support area. Our antispam team (listed in the userinfo of dw_antispam) does amazing work to keep the site spammer-free (or at least spammer-repellant), and everybody who offers opinions in dw_accessibility helps keep us one of the more accessible sites on the internet. The designers in dreamscapes help make the site pretty, and everybody who's opted into one of our public beta tests and given feedback or reported bugs in dw_beta helps to make the site better and more reliable for everybody. If you're interested in helping out, or want to learn more about contributing, why not check out one of our official communities and see what strikes your fancy? We'd love to have you, and I like to think we're a pretty awesome project to work on.

*

That's it from us for another update! As always, if you're having problems with Dreamwidth, Support can help you; for notices of site problems and downtime, check the Twitter status page; if you've got an idea to make the site better, you can make a suggestion.

We'll see you soon for our next update!

Comments

[mugler]

Thank you guys for all your hard work, I really do appreciate it (especially considering you recently were kind enough to transfer paid time on one of my accounts).

I do have a couple questions about previous issues that were addressed in the other news post but I haven't seen mentioned since:

1) Is anything ever going to be done about the number of accounts that are being hoarded and/or are clearly left abandoned? In the past I didn't find it to be much of an issue, but given that everyone has been here for several months now and many others have been created, it's becoming extremely frustrating to find so many usernames just being sat on with no use, little to no comments, and in some cases no confirmation of their email/no way of contacting them about it. I recently created a new account for a character and was getting pretty frustrated at having to go through literally 10+ accounts that have 0-5 comments, no icons, never been posted in, and were made either before or during the big move to DW. I know there have been specific individuals who have been reported for hoarding usernames because they have lists that were clearly visible at the time, but beyond that the number of abandoned journals just seems to be piling up and it's pretty unfair for people who would be putting them to use.

2) I know you guys were kind enough to introduce icon slots, but are there ever going to be any other options? I ask because I did recently have a premium paid account and the only reason I purchased it was not for the length of time but for the icon space. I think I speak for a lot of RPers when I say that icon space is generally one of the big reasons we are willing to opt into the premium accounts rather than the length of time. However it still seems like such a huge jump from 3 months to 6 months, $6 to $25 and then 100 icons to 250. Would you ever consider doing either something similar to what LJ did (a package of userpics rather than individual slots) OR some sort of middle ground, maybe x amount of dollars for 1-3 month accounts with 250/more icons? You guys have been fantastic about being open to other options and additional features for everyone, and I do think especially for RPers it would be extremely useful to have another level in between. (Hopefully it would cut down on the need for manual transfers as well!)

[denise]

1) Generally, once someone registers a username, it's theirs to keep; we don't delete accounts for activity and we aren't interested in doing so (especially since any way that anyone has ever proposed identifying unused or abandoned accounts would also result in people who are legitimately using the site losing their accounts). Just because an account looks as though it's being unused doesn't mean that it is; there are plenty of people who look like they're not using the account but are actually active on the site, and plenty of people who have lost access to the internet through other circumstances (lack of internet, being overseas with no reliable coverage, military/Peace Corps deplyment, long illness or hospitalization, etc) but would be quite upset to find that their account had been deleted when they come back.

Account trading or selling is also a violation of the Terms of Service, above and beyond being a really bad idea (because a traded account can never again be fully secured), so even if you can contact someone who's registered a username you want, please don't accept a traded account.

2) We're very, very aware that people are interested in having more icons, believe me! However, icons are the single most costly part of offering the site and its features, and we've set our account features and levels very carefully to avoid overpromising or overcommitting.

I know that money is tight for a lot of people, but we are entirely user-supported, have no outside form of income such as advertising or venture capital, and operate on a fairly tight budget with pretty limited resources. (It's only recently that we've been able to pay me even slightly more than minimum wage to work full-time on the site, for instance, and Mark is currently unpaid no matter how much we'd love to be able to get him back to full-time work.) We can't give people everything they want for free: unlike many, many, many other sites out there, we aren't funded by advertising or venture capital and we're committed to long-term sustainability, not inflating our numbers and statistics in the short term in order to sell the site to someone else.

I've talked a lot in comments to previous dw_news and dw_biz posts about our business model, why paid accounts cost what they cost and are offered in the bundles they're offered, why we make the business decisions we make, and why things that work for LJ won't work for us -- you might be interested in reading back over those a bit! The most recent relevant discussions are probably in comments to the 21 July dw_news post and the icon add-on brainstorming post in dw_biz.

[hsumi]

Quick question about purchasing icon space;

Does this mean we have to have a normal paid account in order to buy permanent icon space, or can we have these "permanent" icons active even when we aren't using a paid account?

Basically, are we allowed to buy extra icons for a free account?

[denise]

Yes, the account needs to be paid to access any previously-bought icon slots -- otherwise it would mess up all our calculations about service usage and cost.

[gigglingkat]

Good descriptions are short and succinct -- you might explain what's happening in the icon, describe the mood the icon's intended to evoke, or transcribe the text.

I use lyrics as they tend to evoke an emotional response and I'm a music nerd. But I'm never sure if that's annoying or not.

Is there anyplace that I can ask?

[denise]

Ask five screenreader users, you'll get ten answers :) Though, the idea is to be both concise and descriptive -- all modern screenreaders should be able to skip past a particular item of description without reading it, but still, the shorter the better. You might want to put the lyrics in the "comments" field, and use the description for a short phrase or sentence describing the icon instead.

Is this a create entries bug?

[random_xtras]

Alright, when I use this Open ID I can usually edit my entries that I moved over here from LJ. But when I put on the beta for creating entries the journal no longer lets me edit my imported comments, saying that non members cannot make entries.

Is this supposed to happen? And if it is, what good is Open ID? (real question, not snark) If it is supposed to happen, how am I supposed to care for imported entries?

Re: Is this a create entries bug?

[zarhooie]

Open a support request, please!

[multifangeek]

Totally minor question and it's entirely possible I've been totally oblivious until now, but have the "share this entry" links always been on public entries? And is there a way to turn that link off?

[denise]

Yes, it always has been there, and no, there's no way to turn it off -- and even if it weren't there, it's functionally no different than someone opening up an email window and copying and pasting the link to the entry. :) (It's only on public entries, though.)

[primwood]

I seldom comment, but I wanted to thank you all for your efforts. You are an awesome bunch!

[albatrossxwing]

I have a bug. Here is what it looks like.

[fu]

Fixed, thanks for the info.

[triskelion]

i noticed i no longer get contextual pop up menus when i hover peoples icons, only when i hover the little icon thing next to the sn-- is that intentional?

[fu]

Nope not deliberate. Fixed now -- thanks!

[realpestilence]

When I try to read my friends page, half of it is covered by my icon/tags info, so I can't read everything. I was wondering if this is something that can be fixed on your end, or is it something to do with my settings? It just started today.

[denise]

I'm not seeing it happen, so it's probably due to invalid HTML in a protected entry that one of your circle posted. In which case it will stop happening when that entry drops off the front page of your reading page!

[bluebellanon]

Hello! I'm probably going to be importing sherlockbbc over here from LJ at some point in the future. :)

This might seem like a silly question, but is there anything I need to be aware of with it being such a big comm? Is there a best time/worst time for me to import or anything like that?

Sorry, pretty new to the importing thing and wanted to check.

[denise]

You should be okay! If you run into any problems, ask Support.

[kimkardashian]

where are all the hip comms located?

[jordannamorgan]

I see that this post is from late September, but the issue of comments not importing has apparently recurred. I just imported my journals from LJ a couple of days ago, and the commments DID NOT IMPORT, in spite of the importer's claims to the contrary.

I opened a support ticket about this (as I see many, many other people have), and so far it looks like none of us have gotten a reply. It isn't being listed as a known issue you're working on, either. I'd appreciate seeing this problem addressed, or at least acknowledged as something you're actively looking into.

Given LJ's latest fails, I'm looking at the possibility of switching to a paid account here... so if you want my money, now is the time to demonstrate to me that DW will give my concerns the attention LJ no longer does.

[denise]

We're behind on troubleshooting the import problems, and I'm on vacation until next Thursday, but as long as you've got a support request open, we will get back to you as soon as we can! Import problems can only be handled by a very few people, so they do tend to take longer.

[hlagol]

Thanks for working hard, I feel really good giving y'all my money. :D

[solerika]

Quick question regarding spam: I browse the Latest Things page sometimes, and lately I've been noticing an influx of what looks to be spam-type journals such as fashion5154, newguccitojp888, and shoppings555.

Should we report these even if they haven't left comment spam? And if so, is there a proper way of doing it?

[denise]

Yes, please do report them -- those sorts of journals are bad because search engines penalize sites that host things like that. Our antispam team mostly works with comment spam and we don't have a system for reporting journals like that automatically, but you can make a support request in the Terms of Service category and they'll handle it. (You don't need to report those three, I'm going to get 'em right now.)

[valtraid]

I just wanted to say, THANK YOU for everything. :) A part of me wished you guys had another Seed Account sale so that I could purchase an additional one, but I completely understand the rationale behind not offering it. (I should've just bought 2 when I had the opportunity!)

[clairshadows]

Hi I just want to report an antibug and how happy it makes me every time I notice it. My livejournal style has no previous button at the bottom of the page and as such I've trained myself to hit home once I get the bottom of my friends page. Dreamwidth happily does have this rather important feature and I truely, deeply appreciate it.

[discretionary]

hi there! i had a quick (as in verbose and lengthy) question re: moving paids between accounts since i know you suggested that you wanted to discourage doing it so much in the future and more so in relation to premium paids.

basically, the only way to get more than 100 icons is to buy permanent icon spaces per icon (which would need to be transferred, should you want to move those to another account) or to buy a premium (the shortest of which is 6 months of time).

i've had one premium account transferred and the advice i was given was to avoid having to do that in the future, just not buy as much time. i addressed this by suggesting it wasn't about the time, it was about having more icons and that was never really answered back, so i was wondering if you guys have any plants to bridge that gap?

i'd love to by-site have more icons and i'm happy to pay for that, but if you guys are going to be unwilling to move time around because the base amount of time is six months, i kind of feel like that's a problem that will crop up and is already cropping up for me, with this account. even if it were premiums at 2 month increments minimum, i feel like that's less of a worry or a problem over half a year minimum.

i guess, bringing up the permaicons too, will you be unwilling to move those around at-request after a while or will those always be able to be moved?

tl;dr: will there ever be shorter-length premium packages if you do intend to stop moving paids between accounts and does that desire to not move things around as often also extend to permaicons?

[rodriguezmigel]

Сделали бы ещё интерфейс на русском)))))

[denise]

Unfortunately, we can't offer the site in other languages right now. We're too small.

[prototypical]

This is probably the wrong place and I'm sorry, but I can't think of where else to ask -- is there a policy for seeing if a username was taken in a mass grab or was that something I imagined?

[denise]

We talked a bit a little while back about how to handle cases of "username squatting", and it's a very complex issue with no easy solution.

Generally, we don't take away usernames from people once they've registered them, even if they aren't being used or if the person has a lot of accounts, because doing anything like that invariably leads to very, very tough questions. Once a username has been registered, it's been registered, and the account is that person's to keep. The perception that many people have registered hundreds of popular usernames to "squat" on them so other people can't have them is actually really not much of a problem at all: we can look at how many people have a super high number of accounts (and how many of those are unused), and there really aren't very many people doing that at all.

I know it's frustrating when a username you want has been registered already and it appears that the account isn't being used, but it's almost impossible for someone to tell whether an account is being used or not just by looking at the profile, and any account that looks like it's not being used can be being used in ways that just aren't visible, or just isn't being used yet.

There have been a few cases where it's very clear that someone was registering accounts with little to no intent of ever using them, and we've taken some action in those cases. But generally, if a username has been registered, it's been registered, and you'll have to find an alternate username to use instead.