mid-week surprise feature rollout

[mark]

Hey everybody. This week, we decided to bring you a mid-week news post as we have something kinda exciting to announce:

We've implemented text search for paid users (and all users can search paid communities).

I'll let you read that sentence again... it's a neat sentence. I'm pretty proud of it actually, when you get right down to it. But don't take my word for it! You can test out this feature in your own journal:

http://www.dreamwidth.org/search

And if you want to search through dw_news, you can do so here:

http://www.dreamwidth.org/search?search_user=dw_news

This is currently a beta feature. It's a really new system, and we expect there to be some bumps as we figure out how much load it can take, how big of a server we need, and other unexpected things along the way.

...

Now, let me give the boring details of how we've implemented it right now:

* Search is available for paid user accounts as well as paid communities. Free users are allowed to search through paid communities.

* When searching your journal, you can see all of your posts: public, private, protected, etc. But only you can see them (other people cannot search your journal).

* When searching a community, if you are a member of that community, you can search all posts. If you are not a member, you only see public posts.

* The search index is up to 24 hours behind. This means that when you make a post, it will take up to a full day for that post to appear in the search index. The same goes for edits and deletes.

* Some special characters need to be manually added. I've put in a list of most of the major ones, but if you're using Icelandic or some other more interesting languages, the search may not work so well for you.

...

Well, I think that's it. Go take a look, play around, and see if it works for you. Then come on back and let us know what you think. This is very much a "first draft" of the functionality, but we were just so excited we wanted to push it out for all of you.

For those of you who are curious, this came about because of us being at the Open Source Convention this week. I went to a talk on the Sphinx Open Source search engine where the speaker went over some of the features of the software, how to set it up, and gave a bunch of performance information.

I was really blown away by how easy it looked to get Sphinx working, so I spent that afternoon banging on some code. About ten hours later, I had a working prototype of the search system. Our volunteers spent the night messing with it and coming up with some suggestions and today I spent another six hours refining things and getting it ready for this release.

Reading filters are still in progress (the basic version is up for review!). I hope that you will accept this feature as my way of saying "thanks for waiting!"

And of course, thanks again for using Dreamwidth!