Projects - WELL Blogs [josephson.org]

Home » Projects » WELL Blogs

WELL Blogs
An aggregated view of posts to WELL members' weblogs.

» Goto the WELL Blogs page (shows posts chronologically.)

This works by requesting a copy of each blog's RSS feed once an hour, on the hour and adding any new posts to a database. We then build a page showing 7 days worth of posts.

The script uses Mark Pilgrim's Feed Parser to parse the feeds. In order to minimise bandwidth usage we use the ETag and last modified headers to determine if a feed has changed.

This is inspired in part by Phil Gyford's Haddock Blogs which does the same kind of thing for Haddock members and suggestions by Christian Crumlish in the blog conference on the WELL.

Project Log posts from the WELL Blogs category on SDJournal

Tuesday, 15th March 2005

WELL Blogs tweaks and feedparser upgrade

I've updated WELL Blogs to use the the latest version of feedparser. This newer version of the parser provides feed content in Unicode so I've updated the bot to encode things in UTF-8.

I've also made a few other enhancements to deal with variations in the flavours of RSS an Atom such as supporting summaries.
Posted at 11:37PM

Tuesday, 27th January 2004

OPML for WELL Blogs

I'm now also generating a list of the blogs that are included in OPML at wellblogs.josephson.org/opml.xml.

OPML is a file format that can be used (amongst other things) to exchange subscription lists between applications that read RSS so for example if you're using an RSS news read you can use this link to sychronise a list of WELL blogs or to add individual ones without manually copying and pasting the URLs or whatever.

For example, in FeedDemon this is done using the "New Channel Group" or "Synchronize Group with Online Directory" commands.
Posted at 10:35PM

Tuesday, 16th December 2003

Links and permalinks (in RSS 2.0)

Today, Dave Winer linked to a post by Brent Simmons about link elements in RSS 2.0 and whether they should be permalinks or if they should point to an external page.

I think this is relevant to a couple of the wellblogs feeds so probably one of the next updates I'm going to do is to handle this by looking for guid elements that look like URLs and using these as permalinks where available.
Posted at 02:19PM

Thursday, 20th November 2003

Auto catch up for new feeds added to WELL Blogs

I made a change to the script yesterday so that when an RSS feed is requested for the first time we effectively do a catch up on that feed and the items fetched that time are not visible on the page. The idea here is to prevent too many old entries being shown, kind of like how a WELL conference is marked as seen the first time you visit it. Any new posts entered after that first time should appear.
Posted at 11:47PM

Tuesday, 18th November 2003

WELL Blogs launched

It's now up and running at: wellblogs.josephson.org

It shows excerpts from posts sorted chronologically and indicates which blog they are from. I've been testing it for a week or two using 4 or 5 blogs.

There is also an RSS feed for the WELL blogs page itself at: wellblogs.josephson.org/rss.xml
Posted at 06:44PM

Wednesday, 5th November 2003

A WELL bloggers group blog?

I recently remembered about xian's suggestion for creating a web page that combines posts to weblogs by members of the WELL. I've now made a start on this.
Posted at 11:39PM