Weekly Update: February 16th, 2010

[mark]

Time for the weekly update! It's late this week because I was out with some friends yesterday and didn't get home until late, sorry about that. This week we bring you more information on cross-site reading and some updates on what's been going on around here in Dreamwidth land.

1. Cross-Site Reading

I'll cover this first since it seems to be a well received topic when mentioned last week. This is sort of scattershot facts, feedback is welcome.

* The feature is going to be a paid user feature, because of how much data we have to store and the load I expect it to place on our servers.

* We will only download new entries every 60 minutes (this may change). This means that, when you view your reading page on Dreamwidth, we only contact remote sites every ~60 minutes to ask for new entries.

(If you typically view your reading page and refresh it every 30 seconds to see if there are new entries, it won't work that way with cross-site reading. This 60 minutes might change when we see how many people are using the feature, but this is the main way that we are able to control how often we hit remote sites. If we hit them too often, then they'll ban us for being bots/bad/broken/etc. If we wait too long, we could miss entries.)

* Technically, we can only load the most recent 100 entries from the remote site. This is a limitation of the protocol, it only allows us to get recent items. Skipping back indefinitely won't work. However, if there never get to skip=100 on your reading usually, this won't affect you.

* Comment counts will be updated every time we download entries, but it could be up to an hour out of date. The same goes with edited entries; if we download entries, we won't be able to tell that they were edited until the next time we pull (next hour).

* We only download entries while you're browsing the site. In short, whenever you view a page on Dreamwidth (any page), we will update your reading page with remote entries if it's been more than an hour.

In effect, this feature will work best for average users. If you only watch 200-300 accounts and get fewer than 200-300 new entries a day on your remote reading/friends pages, you will find that this feature works pretty well for you. On the other hand, if you've maxed out your subscriptions and the people you read post hundreds of posts every few hours, you will probably not find this feature too useful.

Okay, with the technicals out of the way, let's discuss privacy. Since we're downloading entries for people that don't use Dreamwidth, privacy is a really tough question here and one we finally think we have a good answer for.

Downloaded entries are only viewable to the person who is using the cross-site reading feature.

That's the rule, and it is pretty simple. We will not show entries to anybody except the person who we downloaded them for. Even if it's a public entry; you will never be able to see it on the site unless you are logged in to the account that has cross-site reading turned on and subscribes to that account on the remote site.

Further, we will not allow any interaction with the entry on Dreamwidth. They can be viewed, but that's about it -- you can't add them to memories, bookmark, comment on, edit, or anything. It's a read-only view.

Lastly, entries are only stored for two weeks. At that point we remove them from our servers completely. (Similar to syndicated feeds.)

2. Weekly Community Participation Meme

I've seen a meme floating about Dreamwidth since yesterday that I really like. It's about community participation and was originated by rydra_wong:

http://rydra-wong.dreamwidth.org/197790.html

In short, people who say 'DW is too quiet' can solve the problem by actually making it louder! If we all take a bit of time out of our week to post to a community about something we find interesting, we'll build them up and work towards being the kind of place people want to gather.

For this week, denise recommends poetry for all of your poetical needs. zarhooie further mentioned selfportraitweekly as a new community that is just getting off the ground and could use some participation, too.

If you have your own suggestions, leave them in the comments!

3. Community Imports

A lot of people have asked us about importing communities from remote sites. We've been doing this manually until now, but we are at the point where we want to start opening this up to everybody.

A bug has been filed to make the importer work for communities for everybody. There will be some guidelines (you must be a maintainer, you should get permission of your community first), but we will no longer be manually vetting every community that gets imported.

This is made possible because of all the OpenID changes we made to allow users of other services to actually maintain their entries in communities: delete them, moderate comments, etc. Now that these users have the tools they need to actually maintain their entries, we feel confident opening up the community import tools.

Please keep an eye on the weekly updates for more information about when this comes to fruition.

4. Latest Posts - Olympics!

The 2010 Winter Olympics are going! We've added a feed to the Latest Things tracker that will let you find posts related to the Olympics:

http://www.dreamwidth.org/latest?feed=olympics

Keep in mind this feed only updates when someone makes a post, so it looks a little barren right now (I just made it!) but should fill up as people post things.

5. Suggestions Update

denise wanted me to mention that our suggestions process has reached another milestone. We just passed 400 suggestions that have gone through the process, and the breakdown of results is:

* 86 (21%) suggestions rejected due to impracticality, community disapproval, or not being what we want for Dreamwidth
* 24 (6%) deferred for later review (read, denise and I sitting down and hashing them out)
* 221 (54.7%) in Bugzilla waiting for implementation
* 73 (18%) have been implemented

So, ~40% of suggestions have been resolved one way or another, and ~60% are still waiting for resolution. I feel this is a good ratio! I don't expect us to ever implement every single good idea people have (if only we had the developers to do that!) but to know that we're making progress on them and actually getting things done that users suggest is a good sign.

6. Update Page Redesign

This is a big issue and we want to start getting everybody thinking about it now. denise has spent a lot of time (a LOT of time) on working on the update page. She's been putting together wireframes, researching other blogging tools, showing designs to people, and iterating. She's getting to the point where the wireframes can move to the next step, so let me take a minute to tell you what's going on.

In short, we're going to change the "metaphor" that represents posting to your account. Right now, the post interaction metaphor is akin to a "neverending scroll". You write an entry, post it, and that's that. If you want to edit it, you edit the "last entry" or you have to look around the scroll to find what you want to edit.

Similarly, if you want to create a draft, it only stores what you're writing at the newest end of said scroll. One item, there until you either rip it off or commit it. This metaphor has worked for a while, but it's not going to support the new functionality that we want to add to the system.

The new metaphor denise has been working on is in the wireframe stage right now. What we're going to do next is create a mockup (a small, non-functional preview) that you will be able to interact with. You will be able to get a feel for how it looks, how we want it to work, and then provide feedback.

Once feedback is received, we'll update the mockup and do another round of review on it. "Okay, this is what you told us about the first mockup, how do you like the second?" That process can repeat as many times as necessary, but at some point it will be considered done and we'll move to a full implementation of it.

One thing to note now: we will not be supporting the 'old update page' AND a 'new update page'. Once we've gone through the community review and gotten the mockup to a good place and implemented it, that will be the only update page.

Hopefully by telling you about this in the weekly update and in advance, we can make this process relatively painless. I know it will be a big change for everybody, and there will certainly be some people who prefer the old page, but I hope we can convince you with the great functionality the new metaphor will provide.

Stay tuned!

7. Long Update is Long

This is probably the longest update I've written in quite some time, but we've now reached the end. Yes, I didn't do a development update section this week, focusing instead on bigger items. Next week we'll dive into the development of the past two weeks and see how things are going.

Thanks to all of you for being part of the Dreamwidth community. See you next week!

Comments

[kishenehn]

I know I'm risking incineration by posting this comment here, but I feel very strongly about this and feel the need to say it.

I'm still on LJ, with no imminent plans to move to Dreamwidth ... and to put it bluntly, I don't want my material read over here. I want it read on LJ, where I interact with people and people interact with me, and where the commenting and dialogue takes place. Posting that material over here will be a disincentive to that dialogue, and that interaction is the reason why I blog. I know that you have a rationale that disagrees with that, but my journal is my intellectual property -- and whether or not this is a legal infringement on that intellectual property, in my mind its clearly an ethical infringement.

If your professed devotion to the privacy and personal rights of internet users is anything more than a sham, you'll make this an opt-in feature for the users of other blogging platforms. And if that professed devotion is a just sham, well -- you won't.

[sky]

They've already said up there that there will be no commenting, memorying, etc. on Dreamwidth's end, that people will have to go to the originating site for any of that. I don't think there's any chance of dialogue on your posts being fragmented by people choosing to read from over here, since if they want to talk to you they still have to go to your LJ.

[kishenehn]

But that's exactly the point -- in my mind, a primary reason for a journal is to engender dialogue, and this will be a disincentive to that. Someone reads over here, they're going to be far less likely to go over to LJ to interact with me. In effect, this will work to turn them from a friend (or potential friend) into a voyeur. And that's not why I journal.

[janinedog]

Why do you think that would happen? The entry will still have the normal "read" and "post" comment links, but they will link directly to the entry on LJ (or wherever). So if someone wants to reply to your entry, there isn't any extra clicking to do at all--it's exactly the same action as if they were reading it on LJ. Click "post comment", write comment, click submit. And all comments are in your original LJ entry. If nothing else, I'd think it'd actually encourage people who have already moved mostly or exclusively to DW to comment on their LJ friends' entries more than they do now.

[sky]

I really can't fathom why you would believe that if someone's interested enough in your journal to add you to their friends list and read from an entirely different site, they wouldn't also be interested enough to click "reply" and go over to your LJ to interact with you. That's contrary to the whole purpose of cross-site reading, in my mind. The entire idea is to make it easy for those of us on DW to stay in touch with people on LJ.

[oconel]

But people can read your (public posts) via RSS anyway (or your private ones via iGoogle, for example). Why do you think people won't comment if they read them here?

[yvi]

Do you feel the same way about people reading from clients? Because the protocol DW will use for this is the one that's used for third party clients to display friends pages, from what I understand. DW will function as this client, in a way.

Datapointing, a bit belatedly.

[brooksmoses]

Honestly, for me, the big reason I find this feature important is that it will allow me to read LJ posts here in a way that makes it easy to be part of the conversation.

As I understand this feature, it will tell me "this is how many comments there are on this", and it will have obvious and immediate "reply to this post" and "read comments" links that take me to your page on LJ for that post -- and where I will already be logged in with my LJ account, because I have to have an LJ account in order for any of this to happen. I can't see how that adds any barriers at all, compared to reading it on my LJ friendspage, and I really hope it does not.

If I am wrong and you are right, and this makes it harder to be part of a dialogue than reading on my LJ friendspage does, then I would consider it a failure.

Edit: Which is to say that I really strongly agree with your views on the purpose of posting, and that I very much respect your speaking out for them, even though I disagree with you as to whether this action helps or hinders them.

[anatsuno]

If someone adds your feed to their online feedreader of choice, do you object also? I mean, since the ONLY person who'll be able to cross read your entries is a person who is ALSO subscribed to you on LJ, this is not public per se, it's just a feedreader-type interaction for that reader and not a publishing. There's no Dreamwidth URL for the imported content, no way it can be harvested by a bot, indexed by google, linked to, anything... It's like adding your Lj feed to my Netvibes or my googleReader - do you think these services are unethical also? I am asking honestly and puzzedly, not because I want to flame you.

[stormy]

What the new feature seems like (to me at least), is almost the same as a feed. The only difference would be recognizing that you are logged into the actual site. I could personally understand not wanting your journal to be read elsewhere, but it seems like it will still only be imported and read by those who can actually see it on livejournal - provided you lock your entries. If you do not lock your entries, than it has always been available to be made into a public feed. I'm not sure how you'd be able to keep all the information private if any of it is public (search engines index the information, there are feeds, etc..) but I don't think you'll be forced to have your information ported to somewhere unless you leave it public and allow it.

If you're concerned with your public entries, then I'm not sure.

[torachan]

Um, this is already possible and already happening all the time. Anyone can create a feed of your journal on another journalling site such as IJ or DW, and anyone can add your journal to a feed reader such as Google Reader. DW is making it more convenient, but they are not really doing anything new that is not already going on.

[landshark]

See, that's what I was thinking, too--but then I thought maybe they were talking about something else.

And now I am curious. :)

[flick]

I think they're talking about this with FLocked posts. Which is possible, but fiddly, so most people don't bother.

[matgb]

Posting that material over here will be a disincentive to that dialogue

Corrected for you there.

This will allow those that predominantly or exclusively use DW to read what you've posted on LJ. If they like or want to respond, they're more likely to do so using this feature.

REgardless, your journal (assuming the same username on LJ) has the RSS feed active.

That means anyone can take your content and read it using their feedreader (off LJ), and users of sites or services that allow for authenticated feeds can also read your freinds only content off site.

If you want only people that use their LJ friends page to know you've updated, you can turn off your feed, and remove those that've switched to DW from your access filters.

But your objection is based on a false assumption; this will, genuinely, encourage more interaction with your journal.

In addition, by ensuring only the person subscribed can access the entry, even if you've posted it publicly, there is no infringement. Regardless of which, you release an unlicenced RSS feed, which has far fewer restrictions.

[kishenehn]

Thank you for noting that I'd forgotten to tighten the RSS restrictions on that journal ... I've corrected that now.

And I'll say that I still strongly disagree with your other assertions, but that's what I expected and that's the way it goes. I'll leave it all at that, but I'll reiterate once more: if you honestly respected the users of other blogging platforms, you wouldn't import our material without first obtaining our permission. If its truly as positive a thing as you all claim it to be, then what would be the big deal in asking first? The fact that DW doesn't plan to do that seems like an implication that my assumption is correct, rather than yours.

[ree]

Does Bloglines ask you? Does Google Reader?

How would you expect to be contacted about this feature?

[sophie]

Remember, the only people who would be able to use this cross-site reading functionality will be those who already have an account on the source site, and they'll more than likely already be logged into the source site. In that case, there's absolutely no difference to the commenting process - they click to comment, and on the target site already logged in, and comment.

If they're not logged into the source site, they might need to login to comment, depending on the entry. Most people would log in anyway though, and to be honest it's not much different from reading a friends page while logged out anyway - with the exception that while logged in at DW, you'll be able to see friends-only entries on your list on the other site, but you'll need to actually login on the other site to comment.

[spudtater]

You're misunderstanding a little, I think. The "read comments" / "post comment" links will point to your LiveJournal entry; to reiterate:

Further, we will not allow any interaction with the entry on Dreamwidth. They can be viewed, but that's about it -- you can't add them to memories, bookmark, comment on, edit, or anything. It's a read-only view.

In fact, the only reason that Dreamwidth will store them at all is for cacheing purposes — if they retrieved a fresh copy from LiveJournal every time, then LJ would soon turn round and tell them to desist! And see above, again; this cached copy will only be stored for two weeks, max.

So Dreamwidth users get the convenience of seeing your entry on their DW reading list, but in order to interact with that entry they'll be sent over to your LiveJournal. After the feature is implemented you should expect to see more of your DW friends, not less!

Hope that allays some fears.

[holyschist]

If you don't want to read a long explanation of how RSS feeds currently work and why people may already be reading your journal this way, I suggest you scroll to the bolded bit in the middle, where I link to an explanation of how to change your LJ's RSS feed to show less information.

Do you have a problem with RSS feeds? I realize many people do across the board, but if you don't, this is not any different from an RSS feed except for allowing authenticated posts to be read through an online feed reader. And that's easy to stop: don't allow access to people who are going to read from DW.

It is already possible to read your LJ friendslist via an offline authenticated RSS feed reader: http://www.livejournal.com/support/faqbrowse.bml?faqid=306
That, I don't know of any way to determine whether any of your readers are doing so.

RSS feeds aren't opt-in on LJ. I'm not sure there's a way to disable it at all, or even a way to control what they show (as webcomic artists often do--ensuring that their feeds link to their site instead of hotlinking images).

I understand your concerns about DW--but these technologies already exist and people may already be using them to read your LJ. A more effective solution would be to convince LJ to allow users the ability to disable RSS feeds. You can, however, set your LJ's RSS feed to show only title, or title and a teaser, rather than full-text. It's buried in the FAQ: http://www.livejournal.com/support/faqbrowse.bml?faqid=149 That should solve your voyeur concerns, unless you think people are going to read one paragraph of each of your entries and never click through.

Here's LJ's policy on other sites who don't want their content syndicated on LJ: http://www.livejournal.com/support/faqbrowse.bml?faqid=155
They say you have to block LJ's IP address from viewing your RSS feed. Unfortunately, I doubt you can get LJ to block DW's IP address from viewing your RSS feed specifically, and there are plenty of LJ users who would object to an across the board ban.

Practical: LJ allowing users more control over their own RSS feeds. (Although it looks like they probably can't allow disabling RSS feeds--I can't find anything more recent than this--so syndications levels may be the most control users will get, and since the lowest syndication setting is "title and link only", that's pretty high control, IMO.)

Not practical: Expecting DW and every online and offline feed reader in existence to wait for people to give them permission to pull RSS feeds (which is what opt-in would mean). That would be the death of RSS, because who's going to go to 10s or 100s of places and say "Yeah, you can pull my RSS feed" every time they start a blog? Some people would like the death of RSS, it's true. You may be one of them. But if you use RSS and simply object to this, then I don't think you're aware of what RSS is already capable of.

[holyschist]

Shorter version: if an RSS feed is publicly available, feed readers take that as permission. It would be sort of ridiculous if each feed reader request had to ALSO be manually confirmed by the owner of the feed.

LJ allows you to change RSS feed settings. It does not, and probably never will, allow you to disable RSS feeds entirely.

If you want FULL control over your RSS feed--the ability to disable it, or to specifically block DW from pulling your feed but not block people using downloadable feed readers--no blog/journal website will give it to you: you need your own website, your own blog installation, and possibly some amount of server access.

This lj_dev discussion about RSS feed settings is a bit old, but I couldn't find anything more recent: http://community.livejournal.com/lj_dev/681766.html

[kishenehn]

I'm sorry if this is just a summary response, but I don't have the time to answer every note here ... though I still wanted to reply generally to the RSS thing.

I'm aware of how RSS works from LJ, and (now, at least) have my journal set so the feed will only include the post title. Most LJ-ers probably aren't, and would be terrified at the thought of actually using the console -- but that's LJ's problem, and certainly not DW's.

Now, if all that DW were doing is getting exactly the same information that an RSS feed would provide, then I wouldn't have much of an objection ... but as I read this, it's not the case. Regardless of how my RSS feed is set, as I understand it DW will be using the LJ friends feature to upload the the full content of all my recent entries -- even the friends-only ones, which aren't part of my LJ RSS feed at all. So it seems to me that the comparison isn't really a valid one.

[axiom_of_stripe]

If you don't want people reading your locked posts off of LJ, why don't you just defriend the accounts belonging to people who aren't reading through LJ? Then they won't have access from any other reading platform such as RSS with auth or DW's new reader. You can always give them their access back if they promise to read only through LJ's website again.

[zing_och]

I don't know if you've seen this, but mark said a in a comment below that cut tags will be respected, with the tag link going to LJ. So maybe that might be a way for you to make sure people go over to your LJ?

[ninetydegrees]

How is allowing people to read your material if they're already allowed to do so a violation of your privacy? I'm a little dismayed by the way you want to control the reading and dealing of material you've given Internet users access to. Can't they print it? Quote it? Discuss it in their own journals (wherever those are), privately or publicly? E-mail it? Do you think that by doing so they're infringing on your rights?

How is limiting interaction to the original site a disincentive to the dialog that could take place on the original site? People who don't mind using the two sites will still comment on LJ since they, well, they don't mind. People who want to move to DW and forget about LJ will be able to keep on reading you instead of deciding their flist is not worth the inconvenience of staying on LJ and that may actually encourage them to keep on commenting on your entries.

[finch]

Since the feature will be entirely dependent on your security settings, your best bet is to post locked and not let anyone who uses DW have access to your posts. Then there is no way anyone can read them on their DW reading page, and therefore DW will never be able to access them.