dw_news | Dreamwidth news: 10 June 2014

Hello, Dreamwidth! I bring you news and glad tidings of the new stuff that came out of last night's code push.

(I also bring yet another apology for the length of time in between this news post and the last; my ongoing RSI issues have been so bad that I've had to continue sharply limiting my typing time. The "good" news is that I'll be going in for surgery, or at least a first round, next month, so with a little bit of luck I'll be able to get back in the game a bit better after some time for recovery.)

Behind the cut:

* Development
* First steps towards our new responsive design
* Community admin redesign: admins take note
* Bugfix: deeply nested comments
* Apologies about thread tracking
* Advance warning: date/time checks at posting going away in future
* Embedding sources
* Spam (spam spam spam)

A reminder: Whenever a news post is posted, all notifications are delayed for a little while as the mail system sends out notifications of the announcement. Comment notifications may be delayed for up to an hour or two, due to the high volume of notifications generated after an update is posted to

dw_news. This was posted slightly before 0800 EST (see in your time zone). Please don't worry about missing notifications until at least 1000 EST.

Development

First off, some bad news: the period covered by this update saw a technical problem that, unfortunately, means we lost the contents of our bug tracking database. It was one of those things where one mistake compounded a bunch of times, and when the dust cleared, we realized that a misconfiguration meant the server backups had been wiped along with the server itself.

This does not mean that the same thing could happen to your data -- the bug tracking database was on an entirely separate server with an entirely different configuration than any user data. We also were able to reconstruct much of the data from other sources. We haven't, however, imported the old bug data into our new bug tracking setup. Instead, we're using this as an opportunity to start fresh: a lot of the items in the database had been in there for a while, and although we tried to stay on top of closing bugs that were no longer relevant or were reporting things that had been fixed by another change, there were still a lot of things in the database that weren't relevant to the state of Dreamwidth today.

We've done our best to make sure that actual bugs that are still happening and still interfering with people's use of the site have all been filed in our new bug tracking system -- you can look over the list, to check whether one that you've reported has been re-opened. If there's a bug you're still experiencing, and it's not on the list, you can report it to Support and they'll doublecheck for you, then file the bug if it needs to be filed. The big one we know about is the problem with icons that weren't given keywords at the time of upload, and were automatically given the system-generated 'pic###' pseudo-keyword before being renamed later, are reverting to the 'pic###' pseudo-keyword in some unknown circumstances. We're still trying to figure that one out.

(User suggestions from

dw_suggestions haven't been moved back into the bug tracker yet, but they will be once I'm able to clean up the suggestions queue a little! No need to re-report those.)

Anyway, moving on to happier news: this time around, we welcome new contributors

darael and

fhocutt. The code tours for the changes that just went live were a little more difficult to pull together, thanks to the problems with the bug tracking database, but we're pretty sure we've found most of it:

24 Nov - 10 Mar
11 Mar - 10 Jun

First steps towards our new responsive design

This code push brought with it a redesign of the community administration pages. This was done for a few reasons, both as a "test run" of some of our new responsive design elements and to improve the workflow for community admins.

I'll talk about the changes in workflow in the next section; this one is about the purposes behind the project, and what kind of feedback we're looking for!

There are several purposes of this redesign. Aside from the boring technical ones (making it easier for us to maintain and troubleshoot the code, reducing browser incompatability, giving us more options in terms of designing, etc), the two big ones are:

* Improve the way the site looks and feels on small screens (such as on tablets and mobile devices) without worsening the experience for people using large screens (desktops and laptops).

* Improve the accessibility of the site: make it easier to access the site with screenreaders, interact with the site using alternate input devices, interact with the site using only the mouse or only the keyboard, etc.

This is a "first draft"-type implementation, and we're sure that people are going to have a lot of feedback! Usually we would have run this through a

dw_beta-style beta test that you could activate for your account, but Boring Technical Reasons means that doing a beta-style test for this would have been difficult if not impossible. So, we've converted the community admin area, including the site skins as displayed on those pages, as a working example. This will let you get a sense of what the site skins will look like, and the sort of "visual vocabulary" we'll be using.

We want to hear two things from you: if there are any display bugs or glitches while viewing these pages in any of your devices (on your computer vs on your phone, for instance), particularly any problems involving your use of assistive technology, and feedback on the aesthetics of both the site-skin elements of the pages and the page-contents elements of the pages.

A note on feedback: "how the site looks" is probably the one thing that people feel most strongly about, and discussions about changes to the layout and appearance of the site can get very passionated and heated, very quickly. We'd like to ask you to make sure your feedback is specific, concrete, and actionable, even if -- especially if -- you dislike it. "It sucks, I hate it" doesn't give us anything to go on, while a list of things you think are in the wrong place/too big/too small/too cramped/too spread out/not enough contrast/too much contrast/etc, and why, is something we can evaluate and work from.

We particularly want to hear about the process of using the redesign in your assistive technology setup, whether good or bad: do you find it easy to navigate with the keyboard only? does your screenreader have trouble identifying the links? That kind of thing would be very helpful.

Likewise, if you really like the design, we'd love to hear that. (And not just because it's nice to hear! When it comes to redesigns, it's pretty common for only the people who hate it to speak up, which makes it hard to evaluate the overall reaction.) But if you have a few moments, we'd especially love to hear the specifics of what you like about it, both so we can keep those elements in second and subsequent drafts and so we can make sure we do more of those kinds of things as we work through other pages on the site.

Finally, this is a first draft, and those of you who have been through this process with us before know that things can change considerably between first draft and final draft. If you're newer to the Dreamwidth way of doing things, rest assured that if you really do hate something, we'll do our best to figure out why and what can be done about it. We can't always please everybody, and there are times when y'all disagree with each other or when we have to do something in a particular way because of external reasoning, but we do our best to find something that works for as many people as possible and to explain why we're doing what we're doing if we can't.

Please do not leave your feedback here -- it'll just get lost among the usual responses to announcements. We want to be able to find all the feedback together at once. Go to the

dw_beta post "New responsive-design site skins and community admin pages" and leave your feedback there, please! We'll take it into account and use it to make improvements.

If you aren't a community admin, don't feel like you have to make a community just to see what it looks like. You can still see the site skin on those pages if you want to get a preview, and we'll be doing another round of feedback when we convert the next set of pages. (We chose the community admin area to start with because it's a smaller subset of people who use it regularly, and it's always helpful to limit the participants in a first-draft beta to just a subset of y'all so the feedback is less overwhelming.)

Community admin redesign: admins take note!

Community admins: if you only read one section of this update, read this one! We've changed a bunch of stuff about how you handle the day-to-day work of adminnning your communities.

We chose the community administration area as the "proving ground" for our new responsive design because it was a self-contained area and a smaller subset of y'all who use it, but while we were in there, we made a number of small workflow changes and one large change to how "posting access" works.

The workflow changes are mostly merging pages and views together so you no longer have to remember where exactly things are hidden. For instance, communities used to have two places to change their settings: the Manage Settings page, and a separate settings page where you'd pick the settings that only applied to communities and didn't apply to personal accounts. These have been merged, and now even the community-specific settings are part of the Manage Settings page. If you pick a community that you're an admin of from the "Work as" dropdown, or follow the 'Settings' link from your Manage Communities list, a new 'Community' tab will show up. That's now where you change membership and posting access settings, set the community to moderated or unmoderated posting, and designate the links to the community rules that will be shown to members when they join.

The "Edit Community Members" workflow and the community member list has had a few new filters added, making it easier to handle communities with a high number of members: you can filter only to people who have unmoderated posting, for instance, or only people who are moderators, or only people who are admins. "Check-all" checkboxes have also been added at the top of the grid to make it easier to perform actions in bulk.

The processes of inviting a new member and the process of viewing/managing outstanding invites have been merged onto a single "Invitations" page. There, you can issue invitations for people to join your community (as well as set permissions for their accounts, so you can invite someone specifically as an admin rather than waiting for them to join and then adding the admin flag to their membership). That's also where you check for outstanding invites, and cancel them if you've changed your mind.

Finally, we've changed the posting-access settings and what they do, to be a bit more intuitive for new members and first-time community admins:

* Previously, if you had your community posting set to 'select members', gave some people posting access but not others, then switched the community posting setting to 'all members', people who joined the community after that point would have posting access but people who'd already been members of the comm, and didn't have posting access before the switch, would not. Now, when you switch from 'select members' to 'all members', all members of the community can post immediately: they don't need to be granted posting access. If you want to selectively grant posting access, choose 'select members'.

* When you choose 'select members' as the posting setting, there's now a new option that will help make letting people join easier: you can choose to allow new members to have posting access by default, or keep new members from having posting access by default. (The default right now is 'don't allow new members to have posting access', since that most closely replicates the previous behavior.)

A few examples of when you might want to use each of these settings:

* A community that's open for discussion to anyone: set posting to 'all members'.

* A community that posts entirely members-locked to keep casual passersby from reading it, but wants to allow anyone who joins the community to post: set posting to 'all members'. (And set minimum entry security for the community to 'Members'.)

* A community that has open membership, but is intended for announcements by a limited set of people (for instance,

dw_news!): Set posting to 'select members', set new-member posting to 'new members cannot post'.

* The scenario that has changed slightly: A community that has open membership and wants to allow most people to post, but has one or two members: people who disrupt the community enough that the admin doesn't want to allow them to create new posts, but not enough to warrant banning them from the community entirely. Previously, you could leave posting set on 'all members', but uncheck that single person's posting ability. Now, you should set posting to 'select members', set new-member posting to 'new members can post', and then uncheck that person's posting ability.

Bugfix: Deeply nested comments

We have good news!

Previously, when a particular comment thread starts getting up into the hundreds of replies (...or sometimes the thousands -- I've definitely seen it) and most or many of those comments are in a single thread replying to each other, the system would start to error out once you got down about two or three hundred replies into the thread. This always used to happen in journal-skinned pages. You used to be able to avoid it by switching to site-skinned pages, but once we switched over everything to use the same backend, that wouldn't work anymore either.

Thanks to some deep dive code guru work by

mark with assists by

fu, that shouldn't oughta happen anymore. (Why we love

mark: he basically looked at the problem and said, right, we've got to add a whole new function to the S2 programming language that runs the customization system in order to fix it right instead of hackishly.)

We tested it pretty thoroughly (and then of course found a bug that snuck through testing as soon as we introduced it to the realities of how y'all use the site) but there's always a chance there might be more issues cropping up in the edge cases. If you run into problems with comments not displaying when they should, let us know on the post code-push announcement and we'll get right on it.

Apologies about thread tracking

Because we've been getting a lot of people asking about this: I wanted to apologize for a bug we accidentally introduced, but not exactly the bug people were reporting.

With our last code push, when we rewrote the subscription/notification system, we accidentally allowed the "track this thread" subscription option to display to everyone when they were viewing a comment thread, even though tracking individual threads (as opposed to tracking the whole post) was, and always has been, a paid user feature. So, free users would see the option to track the thread, select it, and then wonder why they weren't receiving any emails for that subscription: they'd report to us that they were missing emails or that their subscription wasn't working, when the issue was that they shouldn't have had access to that subscription type in the first place. (There are a few subscriptions that are only available to paid users, either as a thank-you for people who support the site or because those subscriptions tend to generate a lot of email and thus we have to limit them because of server load.)

I'm very sorry for the confusion, and I'm very sorry for us having accidentally given the impression that individual thread tracking was an option available to everyone. We've fixed things now so the option to track individual threads no longer shows to account types that don't have the access to use that subscription.

Advance warning: date/time checks on posting going away in future

Okay, hands up, how many people have ever opened a window to start writing a long entry, remembered something else you wanted to post about, opened a second tab and dashed that one off quickly, then gotten an error message when you try to post the second one: "You have an entry which was posted at [time], but you're trying to post an entry before this. Please check the date and time of both entries..." etc, etc.

This check was added way back in the LJ days to prevent a very common problem with people's computer clocks being set wrong. It was a horrible support burden (leading to dozens if not more support requests per day): someone's computer battery would be dying and their clock was set wrong because of it, or their clock would just be set a year or two off. Because entries in personal journals are displayed on the Recent Entries page by the time they're dated, not by the time the server received the post, a post dated, for instance, 1970-01-01 would disappear completely: the person would post it, it would display on their Recent Entries behind every other post they'd ever made, and they wouldn't be able to find it when they loaded their journal to see it so they would assume it hadn't been posted at all.

(This is not a problem in communities: to avoid the problem with having posters from many timezones, communities show all entries ordered by server time, not by user-supplied time.)

So, to solve that problem, somebody -- I think it was far enough back that it was when it was pretty much just Brad working on LJ! -- added the date/time check on post. It definitely fixed "help, I lost this entry", but introduced the opposite problem: in addition to the scenario above, someone who posts once with an accidental date of, say, 2025 then has to keep remembering to post in 2025 or do a lot of mucking around with the "date out of order" flag. (Which is a whole 'nother complex thing to explain.)

We've decided that check was pretty useful in 2002 or whatever when fewer people used network-based time correction and might have very mis-set computer clocks, but it's 2014 now, and that's pretty much not an issue anymore. So, sometime in the near future -- not yet, but soon -- we'll be removing the check and no longer trying to detect very mis-dated posts. We wanted to give you all a heads up well in advance of the change.

This will not change the order that things sort on your Recent Entries page. If you post something dated 1970-01-01, it will still show up all the way at the very very end of your Recent Entries. If you post something dated 2020-01-01, it will still show up at the top of your Recent Entries page. (But if you're posting something dated 2020-01-01 to make it show on the top of your Recent Entries page, you should probably use a sticky entry instead -- sticky entries were designed to work specifically for that purpose.)

(Also: every time this comes up, someone suggests doing away with the date/time box entirely and simply assigning the entry the server time at the point the entry was received by the server, adjusted for the user's time zone, rather than letting users specify their own times or using the date/time that the update page was opened. This is one of the Perpetual Suggestions: every time it comes up, it becomes clear that opinions are split pretty evenly as to which would be better, default-to-time-you-hit-post or default-to-time-you-open-update-window, and that the people who care about it really care about it. When opinions are that split, and people hold their opinions that vehemently, we generally keep things the way they are. You can still check the 'use time when entry is posted' checkbox while posting if you want the timestamp on the entry to be the time you hit 'post' and not the time you opened the tab.)

Embedding sources

A reminder to all that embedding media, with HTML tags such as <object> and <embed>, is restricted for security purposes. (Someone with malicious intent can use those tags to embed malware in their journal and then trick other people into visiting it.)

We have a very wide range of "whitelisted" sites that we allow embedding media from, and we're always willing to add more. If there's a media site you want to embed media from, and it hasn't been whitelisted yet, open a support request with a sample of the code that the site provides you for embedding, and we'll add it to the whitelist.

Spam (spam spam spam)

And another reminder: we hate spammers! We hate them so much we have an entire project team dedicated to getting rid of them. But we're always happy to have more help from our users.

If you receive a spam comment: Delete the comment, choosing the "mark as spam" option. The

dw_antispam team will take it from there.

If you come across a journal (through Latest Things or site search) that looks like it exists only to be spammy -- posting bland 'articles' only for the purpose of linking to other sites and boosting their search engine ranking, for instance: Open a support request in the Anti-Spam category, with a link to the journal. The

dw_antispam team will nuke it from orbit. (It's the only way to be sure.)

*

That's it from us for another update! As always, if you're having problems with Dreamwidth, Support can help you; for notices of site problems and downtime, check the Twitter status page; if you've got an idea to make the site better, you can make a suggestion. (I'm a lot behind on the suggestions queue, though, just as a warning.)

Comment notifications may be delayed for up to an hour or two, due to the high volume of notifications generated after an update is posted to