There are a few new features that I am in the midst of working on for an upcoming release of FeedWordPress, but I have released version 0.993 now in order to resolve the critical compatability issue with WordPress 2.5.1. I am still doing compatibility testing to see whether there are any kinks in compatibility with WordPress 2.5.x, but upgrading to this release should eliminate the fatal error that prevented 2.5.1 users from accessing the Syndication Options and the feed settings pages from within the WordPress pages. There are some small bug fixes and the beginning groundwork for some features that will become more fleshed out in the upcoming, more feature-rich release, which aren’t worth going into in detail; besides those, here is what’s new since FeedWordPress 0.992:

• WORDPRESS 2.5.1 COMPATIBILITY: FeedWordPress should now be compatible with WordPress 2.5.1.

• WORDPRESS 2.5 INTERFACE IMPROVEMENTS: FeedWordPress’s Dashboard interface has undergone several cosmetic changes that should help it integrate better with the WordPress Dashboard interface in WordPress version 2.5.x.

• SYNDICATED POSTS CAN BE MARKED AS “PENDING REVIEW”: WordPress 2.3 users can now take advantage of WordPress’s new “Pending Review” features for incoming syndicated posts. Posts marked as “Pending Review” are not published immediately, but are marked as ready to be reviewed by an Administrator or Editor, who can then choose to publish the post or hold it back. If you want to review syndicated posts from a particular feed, or from all feeds, before they are posted, then use Syndication –> Syndicated Sites –> Edit or Syndication –> Options to change the settings for handling new posts.

• AWARE OF NEW URI FOR del.icio.us FEEDS: Previous releases of FeedWordPress already automatically split del.icio.us tags up appropriately appropriately when generating categories. (del.icio.us feeds smoosh all the tags into a single <dc:subject> element, separated by spaces; FeedWordPress un-smooshes them into multiple categories by separating them at whitespace.) Unfortunately, del.icio.us recently broke the existing behavior by changing host names for their feeds from del.icio.us to feeds.delicious.com. Version 0.993 accounts for the new host name and un-breaks the tag splitting.

If you have put off upgrading to WordPress 2.5.1 due to this bug, and plan to upgrade after installing FeedWordPress 0.993, please remember that after you upgrade WordPress, you will need to reinstall the FeedWordPress MagpieRSS upgrades in order to keep your feed parsing from getting broken.

Enjoy! As I mentioned, I’m actively working on a release, probably due sometime before the end of the month, including bug fixes and a few significant new features, so let me know about any ongoing issues that you may still have.

This problem should, hopefully, be something that it won’t be too hard to fix once I am able to sit down and work on it. Unfortunately, the release came after I had left to visit family in Alabama, so I can’t get to making the fix until I return home early in the upcoming week. Once I do, I’ll release an immediate compatability fix, and then return to work on a more thorough update, which I hope to release sometime during the month of May.

I’ll check back in with y’all once I’ve gotten home.

That said, one of the most common sources of error reports when new versions of WordPress are released come not from a real compatability issue, but rather from the fact that, if you’re not careful, upgrading your copy of WordPress will downgrade your copy of MagpieRSS from the newer version shipped with FWP to the very old and busted version that WordPress continues, for whatever reason, to ship with new releases of WordPress.

Diagnosis

Here are the most common symptoms of this problem:

• Some feeds (notably, feeds produced by Blogger and other Atom 1.0 feeds) stop syndicating post contents. You get the headline of the post and nothing else.
• Some feeds (notably, those produced by blogs hosted at WordPress.com (!)) start appearing with just the capital letter A as the content of the post.
• Categories stop being properly syndicated. Everything is placed in Uncategorized or in bizarre, mashed-up categories (only one per post) that seem to contain several category names.
• Podcast attachments are no longer syndicated.

And so on, and so forth. If you notice these problems with your feeds just after you’ve upgraded your copy of WordPress, it’s probably because you need to re-install the MagpieRSS upgrade.

Cure

Here’s how you do that. In the FeedWordPress plugin directory (wp-content/plugins/feedwordpress/, relative to your WordPress installation), there is a directory called MagpieRSS-upgrade, which contains, or at least at one point contained, two files, rss.php and rss-functions.php. If you still have these files, you need to copy them to your WordPress wp-includes directory, where they will overwrite the older version of MagpieRSS that ships with WordPress. If you do not (because, for example, you moved them rather than copying them when you first installed FeedWordPress), then you can get new copies of these files by downloading the latest version of FeedWordPress, and extracting these two files from the archive.

Etiology and pognosis for the patient

The reason that this happens is that every installation of WordPress includes a very old version of MagpieRSS, the library that FeedWordPress uses to parse the feeds that it syndicates. As of the 2.5 release, WordPress still ships with a package derived from MagpieRSS 0.51, which is the same version it shipped with when I started work on FeedWordPress three years ago. This version of MagpieRSS is adequate for what WordPress needs it to do (basically, fetch headlines for the Dashboard from a select few feeds), but it was already outdated three years ago, and it is especially outdated now–it could not handle multiple categories; it could not handle enclosures, it could not translate feeds in alternate encodings; and, importantly, it cannot correctly handle Atom 1.0 feeds (now the default for Blogger feeds) or feeds with MediaRSS extensions (now the default for WordPress.com feeds). Unfortunately, since there is a version of MagpieRSS, which is loaded every time you load WordPress, it is hard to drop in a newer version which can do these things without causing errors from the collision in function and class names.

The solution I settled on was the bundled MagpieRSS upgrades, which in the past I sometimes described as optional, but which now really are mandatory if you hope to do any serious syndication in the modern environment. Users can avoid collisions by copying the upgrade so that it just overwrites the older version in wp-includes. Problem solved for the time being.

But the downside of this solution is that every time an upgrade of WordPress comes out, it comes out with older versions of the MagpieRSS package included, and when you overwrite all the files in wp-includes with the newer files from the WordPress release, one of the things you overwrite is your upgraded copy of rss.php. Meaning that, unless you remember to re-upgrade MagpieRSS every time you upgrade WordPress (something which is easy enough to forget), it breaks your syndication until you remember, or I remind you, to re-do the upgrade.

I frankly consider this a design flaw in FeedWordPress, but it’s not a flaw that is easy for me to fix. I am considering different ways of getting around it, and honestly the most likely solution at this point is probably simply to abandon MagpieRSS and package another feed parsing package (such as SimplePie) in the feedwordpress plugin directory, where upgrades to the WordPress core code cannot interfere with it. But doing that will involve pretty dramatically refactoring some of FeedWordPress’s internal workings, and that may take a while. In the meantime, if you have a working aggregator, you should probably apply this quick fix and see how many of your problems it solves.

This release fixes a bug in the initial release, version 2007.10.08, which could cause the plugin to insert inappropriate HTML code into your blog’s feeds (thus making them invalid Atom or RSS) under certain conditions.

It is also the first public release of Humanized History to be listed in the WordPress Plugins Directory and hosted in the WordPress Plugins Repository. This has required a bit of re-arranging in plugin files; if you notice anything that seems amiss, let me know.

Incidentally, the current URI for my plugin, http://wordpress.org/extend/plugins/humanized-history-for-feedwordpress/, is the result of making a typo in the process of applying for the hosting. I have written in to the Repository administrators to get it fixed to what it should be, http://wordpress.org/extend/plugins/humanized-history-for-wordpress/. I don’t know how long it will take to get it fixed. But in any case, in the meantime, I’ll just say here that, in spite of the URI, my plugin has nothing in particular to do with FeedWordPress, except in that they are both written by the same author. You can happily use Humanized History without FeedWordPress, or FeedWordPress without Humanized History, or both together at the same time, as you prefer.

Since I’ve had to spend time either traveling or working on other projects, it’s been longer than I would have liked since the last update of FeedWordPress. This release is a rather limited one: it fixes one outstanding bug and adds one major new feature:

1. BUG RELATED TO URIS CONTAINING AMPERSAND CHARACTERS FIXED: A bug in WordPress 2.x’s handling of URIs in Blogroll links created problems for updating any feeds whose URIs included an ampersand character, such as Google News RSS feeds and other feeds that have multiple parameters passed through HTTP GET. If you experienced this bug, the most likely effect was that FeedWordPress simply would not import new posts from a feed when instructred to do so, returning a “0 new posts” response. In other cases, it might lead to unpredictable results from feed updates, such as importing posts which were not contained in the feed being syndicated, but which did appear elsewhere on the same website. This bug has, hopefully, been resolved, by correcting for the bug in WordPress.

2. NEW FEATURE - AUTHOR RE-MAPPING: I’ve been promising this one to my e-mail correspondents for a couple of months now; and I’m happy to announce that I’ve been able to polish up the preliminary implementation that I was working on late last year and make it ready for general consumption. FeedWordPress now offers a new feature in the site-wide Syndication Options (Syndication –> Options), and in the settings for each syndicated feed (Syndication –> Syndicated Sites –> Edit). You can now create re-mapping rules to determine how author names in a feed are translated into usernames within the WordPress database.

   Traditionally, what FeedWordPress has done, when adding a post, is to take the author information from the feed and search the WordPress database to determine whether there is a username with the same name or e-mail address. If it found a match, then the new post would be assigned to that user. If there was no match, then FeedWordPress would consult the settings for the feed, or the global default settings, for what to do with an unfamiliar author. These could be set by the administrator from within the WordPress Dashbard, with three options: (1) create a new user account with the same name as the author, and assign the new post to that new user; (2) filter out the post rather than adding it to the database; or (3) assign the new post to a default user (user #1 in the WordPress database, i.e. the site administrator).

   With the new re-mapping feature, you have considerably more control over what FeedWordPress does with post authorship data from the feed. In the settings for each syndicated feed (Syndication –> Syndicated Sites –> Edit), you can now define rules that tell FeedWordPress what to do with each particular author name on that feed — to filter out all posts by that author, or to assign posts by that author to any username that you like. You can also tell FeedWordPress what to do with usernames that aren’t listed in your rules, and which don’t match any of the users already in the WordPress database — to filter out the posts, or to assign them to any username you like, or to create a new username to match the new author, or to follow the default setting for new author names on all feeds. The site-wide default setting can be changed using Options –> Syndication.

   So, for example, one of my friends runs a FeedWordPress aggregator site that syndicates http://praxeology.net/blog/feed/. The problem is that, like an increasing number of WordPress blogs out there, all the posts on this blog are attributed to Administrator. For those visiting the blog directly, it’s clear that Administrator is the blogger; but when someone aggregates the blog on another website, the naming now mistakenly implies that the Administrator of the aggregator website is the author of the post. (It also means that if two or more blogs both attribute their posts to Administrator, the aggregator site will mistakenly treat all of the posts from all of those blogs as being by the same author.) With the new re-mapping feature, you could now define a rule that would attribute posts by Administrator from the praxeology.net feed to a different username — in this case, Roderick Long. Problem solved. Huzzah!

One more note before I go. I regret that I haven’t been able to develop FeedWordPress more actively than I currently am developing it, or to spend as much time handling and responding to bug reports as I would like. I originally created FeedWordPress for my own use, and made it available to others in the hope that it would be useful, so the best guarantee of getting a feature added or a bug identified and quickly fixed has always been whether it’s one that I personally encountered, or one of my friends encountered, in the course of using it. But I’m really very pleased with how much uptake and interest there has been for FeedWordPress. Ideally I would like to devote much more time to FeedWordPress development and support than I currently do. But I earn my living freelance, by the hour or by the project, and I do have to pay my rent every month, so the only way that I can keep up with this on more than a limited and casual basis is if the donations from FeedWordPress users allow me to free up the time needed to work on FeedWordPress, rather than spending the same time looking for paying gigs.

So, if you would like to see more regular upgrades and bugfixes, and more rapid replies to your support questions, I’d urge you to consider how much ongoing development and support for FWP is worth to you, and consider making a contribution through the project donation jar at http://projects.radgeek.com/feedwordpress/. It’s in your hands, and anything you can offer will help. (If I had $5 for everyone who ever sent me a FeedWordPress tech support question, I’d be flush for the next several years….)

Thanks,
-C

Version 0.991 fixes some bugs that were present in version 0.99, adds a couple of minor features, reorganizes the file structure of the archive, and adds one very significant change. Let’s start with the significant change, which I am very happy to announce:

• WORDPRESS MU COMPATABILITY: FeedWordPress should now be compatible with recent releases of WordPress MU. Once FeedWordPress is made available as a plugin, each individual blog can choose to activate FeedWordPress and syndicate content from its own set of contributors.

And now on to the bug fixes:

• BUG RELATED TO INTERNATIONAL CHARACTERS IN AUTHOR NAMES FIXED: Due to a subtle incompatability between the way that FeedWordPress generated new user information, and the way that WordPress 2.0 and later added new authors to the database, FeedWordPress might end up creating duplicate authors, or throwing a critical error message, when it encountered authors whose names included international characters. This incompatability has now been fixed; hopefully, authors with international characters in their names should now be handled properly.

• media:content BUG IN MAGPIERSS FIXED: A bug in MagpieRSS’s handling of namespaced elements has been fixed. Among other things, this bug caused items containing a Yahoo MediaRSS <media:content> element (such as many of the feeds produced by wordpress.com) to be represented incorrectly, with only a capital “A” where the content of the post should have been. Feeds containing <media:content> elements should now be syndicated correctly.

• MAGPIE WARNINGS NO LONGER DISPLAYED BY DEFAULT: A number of MagpieRSS warnings or error messages that were displayed when performing an automatic update are no longer displayed, unless debugging parameters have been explicitly enabled.

Some minor bugs in the administrative interface have also been fixed.

I have received some feature requests concerning the new updating system that I introduced in FeedWordPress 0.99. Under the old system, there was a (needlessly complicated) system for having FeedWordPress poll its feeds for updates, which could be activated with a cron script on a regular schedule, thus keeping FeedWordPress up to date. In version 0.99, I replaced this with a much simpler system, in which FeedWordPress would automatically poll its feeds, according to a certain schedule, when users visited your website. (Of course, a cron script could still be used to make sure that this was activated on a regular schedule.) However, some users wrote me that they preferred that visitors not have to wait for the polling to finish before the page loads. So, I’ve added an HTTP parameter which gives you the option. If you prefer not to use WordPress’s cron-less automatic updates, you can now use the update_feedwordpress parameter instead. Here’s how it works:

• Leave Automatic Updates turned off. This will ensure that FeedWordPress polls for updates only when you tell it to do so.

• Add a line something like the following to your crontab:

  */15 * * * * curl --silent http://www.zyx.com/blog/?update_feedwordpress=1
  

  … replacing http://www.zyx.com/blog/ with the address of the blog where FeedWordPress is installed. This line will instruct FeedWordPress to poll for updates once every 15 minutes. If you have left Automatic Updates off, it will only check for updates on the 15 minute interval.

If you prefer the cron-less update method, then that’s still available; just go to Options –> Syndication and turn Automatic Updates on.

Finally, a word about the re-organization of the archive. FeedWordPress is now listed in, and hosted by, the kind folks at the WordPress Plugin Directory. In line with their style guide about how to organize archives, I have moved the main plugin files — feedwordpress.php and syndication-options.php — to the root directory of the archive, and I have relocated the MagpieRSS upgrade to a subdirectory called, creatively enough, MagpieRSS-upgrade. However, please remember that in order to take advantage of the MagpieRSS upgrade — which you will definitely want to take advantage of, unless you don’t care about Atom 1.0 feeds, proper handling of multiple categories, or several bug fixes — you must still copy the contents of the MagpieRSS-upgrade directory to the wp-includes folder of your WordPress installation.

Enjoy, and have a wonderful Thanksgiving, y’all.

The plugin was inspired by the points that Aza Raskin made at Humanized 2006-04-25: No More Pages?

Of course, this page-chunking phenomenon isn’t limited to search sites. It’s used everywhere from blogs to forums, from e-commerce sites to e-mail programs. And it’s surprising how often one finds oneself just giving up and going somewhere else when one has reached the end of a page.

The problem is that every time a user is required to click to the next page, they are pulled from the world of content to the world of navigation: they are no longer thinking about what they are reading, but about about how to get more to read. Because it breaks their train of thought and forces them to stop reading, it gives them the opportunity to leave the site. And a lot of the time, they do.

The take away? Don’t force the user to ask for more content: just give it to them.

The Humanized History for WordPress plugin allows you to just give it to your readers. The trick was to develop a way for the plugin to make those posts — potentially hundreds of posts stretching over several years — all available to readers without making them wait for everything to download. (Not only would that make everyone wait for those posts to download; it would also make the website completely inaccessible for mobile devices and other alternative web platforms.) The solution to the problem was some PHP on the back end, and a little bit of unobtrusive JavaScript on the front end, inspired by Humanized’s implementation, which work together to automatically display the older posts as you scroll down towards the bottom. That way, for users with conventional web browsers there’s always more waiting for you to read as you scroll down (at least, until you reach the earliest post). But mobile users and other people who have JavaScript turned off can still access the website the same way they already were, with no substantial change.

I implemented a version of this trick on a couple of other websites that I run (1, 2) using a bit of ugly spit-and-bailing-wire kludgery in the WordPress templates before I bothered to sit down and package it up as a plugin. But I think it’s a neat feature, and I’ve gotten a couple of requests for help in implementing it on other WordPress weblogs. Hopefully this release should make that easier to do.

The plugin has a couple of weak or kludgy aspects, which are probably inevitable due to certain limitations of WordPress. Be sure that you read the installation instructions and the section on templates and styling carefully. To make good use of this plugin you will, unfortunately, probably need at least a passing familiarity with editing WordPress templates.

Let me know what you think — and whether you notice anything weird or broken. Enjoy, and scroll on!

To explain what I mean, I’ll have to back up a bit, first.

There are a couple things that FeedWordPress uses date and time information for:

1. Dating new posts: When a new post comes in over a feed, FeedWordPress uses the date and time information reported by feeds to date posts appropriately in the WordPress database. This means parsing date and time information and putting the resulting timestamps into the database.

2. Checking for updates to existing posts: After a post has been imported into the WordPress database, FeedWordPress will keep checking to see whether the feed reports any updates to that post. This means comparing the last-updated timestamp on the feed to the last-updated timestamp in the database to see which version is newer.

The problem that I noticed came about because FeedWordPress was trying to do something sensible and easy to handle time zones when it was doing all this. Date/time handling in WordPress is fairly easy, once you understand what to do, but it is certainly anything but sensible, and therein lies the problem.

Part of the problem is, as usual, the stack of programs on top of which FeedWordPress has to sit. In order to get dates from the feed, FeedWordPress has to process two different human-readable date formats. RSS feeds use RFC 822 (or something like it), which FeedWordPress can convert to a Unix timestamp using the PHP strtotime() function. Atom feeds use the W3C DateTime Format, which FeedWordPress can convert to a Unix timestamp using a custom function provided by MagpieRSS, called parse_w3cdtf(). Unix timestamps supposedly do not vary by time zone: a Unix timestamp for a particular time is defined as the number of seconds between that time and 12:00 midnight on January 1, 1970 Greenwich Mean Time. So both strtotime() and parse_w3cdtf() make use of the time zone data provided by the format that they handle, and use it to convert the date-time information to a timestamp based on GMT.

So far so good. Now, when converting a syndicated item into a post for the WordPress database, FeedWordPress has to generate four different timestamps: post_date, which gives the date and time that the post was first published in the local time zone; post_date_gmt, which gives the date and time that the post was first published in Greenwich Mean Time; post_modified, which gives the local date and time that the post was last modified; and post_modified_gmt, which gives the GMT date and time that the post was last modified. These fields are stored in the WordPress database as MySQL DATETIME objects. WordPress later converts them back into Unix timestamps whenever it is necessary for formatting purposes. Here is how I did this in version 0.981:

$post['post_date'] = date('Y-m-d H:i:s',
    (!is_null($post['epoch']['issued'])
    ? $post['epoch']['issued']
    : $post['epoch']['modified']));
$post['post_modified'] = date('Y-m-d H:i:s',
    $post['epoch']['modified']);
$post['post_date_gmt'] = gmdate('Y-m-d H:i:s',
    (!is_null($post['epoch']['issued'])
    ? $post['epoch']['issued']
    : $post['epoch']['modified']));
$post['post_modified_gmt'] = gmdate('Y-m-d H:i:s',
    $post['epoch']['modified']);

The $post['epoch'] array contains the Unix timestamps that FeedWordPress got from the feed. The PHP date() function formats a Unix timestamp relative to the local time zone. The gmdate() function formats it relative to Greenwich Mean Time. Given that I need local and Greenwich representations of the date and time that the post was first published, and of the date and time that the post was last updated, this seems like the sensible thing to do. But in spite of (or because of) its seeming so sensible, this way of generating post timestamps introduces a subtle bug. More on that later.

When determining whether or not a post has been updated, FeedWordPress uses two items of information: the last-updated date and time provided by the feed, and the last-updated date and time stored in the database. In order to get these two dates into a common format so that they can be compared, FeedWordPress uses either strtotime() or parse_w3cdtf() to convert the feed’s last-updated date and time to a Unix timestamp, and it uses the MySQL function UNIX_TIMESTAMP() to get a Unix timestamp from the last-updated date and time in the database. So here is how I did this in version 0.981:

    $guid = $post['guid'];
    $result = $wpdb->get_row("
    SELECT id, guid, UNIX_TIMESTAMP(post_modified) AS modified
    FROM $wpdb->posts WHERE guid='$guid'
    ");

    if (!$result) :
        $freshness = 2; // New content
    elseif ($post['epoch']['modified'] > $result->modified) :
        $freshness = 1; // Updated content
    else :
        $freshness = 0;
    endif;

strtotime() and parse_w3cdtf() take account of time zone data; UNIX_TIMESTAMP() presumes that the time is in the local time zone. So, I just need to feed it the DATETIME object that’s in the local time zone — post_modified instead of post_modified_gmt, right? Wrong. Again, in spite of (or because of) its seeming so sensible, this way of comparing timestamps introduces a subtle bug.

Here’s what was wrong with both of these sensible steps: in any given WordPress installation, there are three potentially distinct local time zones that you have to consider:

1. The local time zone of the web server on which WordPress is running (usually set by the web server’s administrator);

2. The local time zone of the MySQL server that provides the WordPress database (usually set by the MySQL server’s administrator);

3. The local time zone set by the user in WordPress’s General Options page (set by the blog owner);

If any of these differ from each other, then the mismatch could cause problems for the way that older versions of FeedWordPress handled dates.

When the PHP date and time functions convert back and forth between human-readable formats and Unix time stamps, they do so relative to the web server’s local time zone. When MySQL converts a DATETIME object to a Unix timestamp, it does so relative to the MySQL server’s local time zone. When WordPress prepares dates and times for storage in the database, or processes them for sorting and displaying posts, it does so relative to WordPress’s local time zone, as set under General Options.

So, when FeedWordPress used the PHP date and time functions to generate post_date and post_modified, it used the wrong time zone — WordPress expects these to be local times in the time zone set under General Options, but the PHP functions use the web server’s local time zone. If the user has set a different time zone from the web server’s default time zone, then this date and time information will be incorrect. In order to get the correct time, we need to get the time zone information from the WordPress database, and then manually apply that offset, rather than leaning on PHP’s date and time functions. So here is how we do this task in FeedWordPress 0.99:

// Dealing with timestamps in WordPress is so fucking fucked.
$offset = (int) get_option('gmt_offset') * 60 * 60;
$this->post['post_date'] =
    gmdate('Y-m-d H:i:s', $this->published() + $offset);
$this->post['post_modified'] =
    gmdate('Y-m-d H:i:s', $this->updated() + $offset);
$this->post['post_date_gmt'] =
    gmdate('Y-m-d H:i:s', $this->published());
$this->post['post_modified_gmt'] =
    gmdate('Y-m-d H:i:s', $this->updated());

Note that we use gmdate() in all cases, because we are doing the time zone offset manually rather than letting PHP do it for us.

On the other hand, when FeedWordPress used the MySQL date and time functions to convert MySQL DATETIME objects to Unix timestamps, it used the wrong time zone again — since it was using post_modified, and in the older versions of FeedWordPress post_modified was generated using PHP date and time functions, the time was a local time relative to the web server’s time zone. But UNIX_TIMESTAMP() presumes that the time it is given is a local time relative to the MySQL server’s time zone. Usually this difference should cause no problem, since the web server and the MySQL server are the same machine, or if not the same machine, at least machines that are located in the same place as one another. But if that assumption ever fails, UNIX_TIMESTAMP() will return the wrong time–a time that is either a few hours before, or a few hours after, the real last-updated time. Which will mean that posts either get updated when there’s nothing new to update, or don’t get updated even when there is something new to include.

So we need to convert the MySQL DATETIME value to a Unix timestamp in a context where we know, and can set, the right time zone for the conversion. In this case, the best thing to do is to get the GMT date and time of the last update (in order to avoid issues that might arise from changes in the local timezone setting in WordPress), and then manually convert that into a Unix timestamp using a function that works relative to GMT. So here’s how FeedWordPress 0.99 checks the update times against each other:

$guid = $wpdb->escape($this->guid());

$result = $wpdb->get_row("
SELECT id, guid, post_modified_gmt
FROM $wpdb->posts WHERE guid='$guid'
");

preg_match('/([0-9]+)-([0-9]+)-([0-9]+) ([0-9]+):([0-9]+):([0-9]+)/',
    $result->post_modified_gmt, $backref);
$updated = gmmktime($backref[4], $backref[5], $backref[6],
    $backref[2], $backref[3], $backref[1]);
if (!$result) :
    $this->_freshness = 2; // New content
elseif ($this->updated() > $updated) :
    $this->_freshness = 1; // Updated content
    $this->_wp_id = $result->id;
else :
    $this->_freshness = 0; // Same old, same old
    $this->_wp_id = $result->id;
endif;

post_modified_gmt always returns the string representation of a MySQL DATETIME object; the regular expression breaks that representation down into its component parts; and the PHP function gmmktime() reassembles those parts into a Unix timestamp. (You might worry that using the PHP date and time functions might reintroduce the first problem, since it doesn’t account for the local time zone set in WordPress. But since everything is guaranteed to be in GMT, this problem doesn’t arise.)

Strictly speaking, it would probably be better to use MySQL functions, instead of a regular expression, to extract the parts of the MySQL DATETIME object, since ostensibly MySQL knows more about its internal formats than PHP does. In practice this is not likely to make a difference, but it’s likely that in future releases of FeedWordPress I’ll change the section to something more like this:

$guid = $wpdb->escape($this->guid());

$result = $wpdb->get_row("
SELECT
    id, guid,
    YEAR(post_modified_gmt) AS year,
    MONTH(post_modified_gmt) AS month,
    DAYOFMONTH(post_modified_gmt) AS day, 
    HOUR(post_modified_gmt) AS hour,
    MINUTE(post_modified_gmt) AS minute,
    SECOND(post_modified_gmt) AS second
FROM $wpdb->posts WHERE guid='$guid'
");

if (!$result) :
    $this->_freshness = 2; // New content
else:
    $updated = gmmktime(
        $result->hour, $result->minute, $result->second,
        $result->month, $result->day, $result->year
    );
    if ($this->updated() > $updated) :
        $this->_freshness = 1; // Updated content
        $this->_wp_id = $result->id;
    else :
        $this->_freshness = 0; // Same old, same old
        $this->_wp_id = $result->id;
    endif;
endif;

Oh, and in case you were wondering, the reason that moving across the country helped me find this out is that I moved from Eastern Time to Pacific Time, and I changed the default time zone on one of my web servers before I changed the default time zone on my MySQL server. The mismatch exposed this bug while I was doing testing for the most recent release of FeedWordPress. Not particularly interesting, but it did expose a bug to fix, and I hope the guide to time zone issues that resulted may be of some interest.

Update 2007-11-21: FeedWordPress 0.99 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

The public (non-beta) release of FeedWordPress version 0.99 is now available for download.

There have been changes to the way that FeedWordPress’s code is organized since version 0.98. If you successfully installed either of the beta releases, you don’t need to do anything special to install the current release. However, if you are upgrading from version 0.98 or before, be sure to see the installation instructions below.

Changes since version 0.98

This release provides compatibility with WordPress 2.2 and 2.3. It has been extensively tested against WordPress version 2.2.3 and the version 2.3 release candidate. I think that all the compatibility issues have been hammered out; of course, if you notice any problems, please let me know and I’ll get on a bugfix as soon as possible.

Version 0.99 also includes an overhaul to the user interface, some significant new features, and a number of bug fixes:

• AUTOMATIC UPDATES WITHOUT CRON: FeedWordPress now allows you to automatically schedule checks for new posts without using external task scheduling tools such as cron. In order to enable automatic updates, go to Syndication –> Options and set “Check for new posts” to “automatically.” When this option is turned on, FeedWordPress will check for new posts automatically (1) when someone views your page, (2) if it has been ten minutes (or whatever interval you set) since the last time someone viewed your page. This offers a way to keep FeedWordPress up-to-date without having to schedule a cron script. It also simplifies the process of updating if you do choose to use a cron script — just have curl fetch your home page on a fixed schedule (so, for example, I would execute curl http://feministblogs.org/ every 15 minutes to keep Feminist Blogs up-to-date). Note that this is not the same thing as precisely scheduled updates — at a minimum, FeedWordPress will not check for new posts unless and until the next time somebody views your page. But for practical purposes it does allow you to keep your aggregator updated without having to run cron, and it is as close to precisely scheduled updates as you can get without using real scheduling tools such as cron.

  An important side-effect of the changes to the update system is that if you were previously using the cron job and the update-feeds.php script to schedule updates, you need to change your cron set-up. The old update-feeds.php script no longer exists. Instead, if you wish to use a cron job to guarantee updates on a particular schedule, you should have the cron job fetch the front page of your blog (for example, by using curl http://www.zyx.com/blog/ > /dev/null) instead of activating the update-feeds.php script. If automatic updates have been enabled, fetching the front page will automatically trigger the update process.

• INTERFACE REORGANIZATION: All FeedWordPress functions are now located under a top-level “Syndication” menu in the WordPress Dashboard. To manage the list of syndicated sites, manually check for new posts on one or more feeds, or syndicate a new site, you should use the main page under Syndication. To change global settings for FeedWordPress, you should use Syndication –> Options.

• FILE STRUCTURE REORGANIZATION: Due to a combination of changing styles for FeedWordPress plugins and lingering bugs in the FeedWordPress admin menu code, the code for FeedWordPress is now contained in two different PHP files, which should be installed together in a subdirectory of your plugins directory named feedwordpress. (See README.text for installation and upgrade instructions relating to the change.)

• MULTIPLE CATEGORIES SETTING: Some feeds use non-standard methods to indicate multiple categories within a single category element. (The most popular site to do this is del.icio.us, which separates tags with a space.) FeedWordPress now allows you to set an optional setting, for any feed which does this, indicating the character or characters used to divide multiple categories, using a Perl-compatible regular expression. (In the case of del.icio.us feeds, FeedWordPress will automatically use \s for the pattern without your having to do any further configuration.) To turn this setting on, simply use the “Edit” link for the feed that you want to turn it on for.

• REGULAR EXPRESSION BUG FIXED: Eliminated a minor bug in the regular expressions for e-mail addresses (used in parsing RSS author elements), which could produce unsightly error messages for some users parsing RSS 2.0 feeds.

• DATE / UPDATE BUG FIXED: A bug in date handling was eliminated that may have caused problems if any of (1) WordPress, or (2) PHP, or (3) your web server, or (4) your MySQL server, has been set to use a different time zone from the one that any of the others is set to use. If FeedWordPress has not been properly updating updated posts, or has been updating posts when there shouldn’t be any changes for the update, this release may solve that problem.

• GOOGLE READER BUGS FIXED: A couple of bugs that made it difficult for FeedWordPress to interact with Google Reader public feeds have been fixed. Firstly, if you encountered an error message reading “There was a problem adding the newsfeed. [SQL: ]” when you tried to add the feed, the cause of this error has been fixed. Secondly, if you succeeded in getting FeedWordPress to check a Google Reader feed, only to find that the title of posts had junk squashed on to the end of them, that bug has been fixed too. To fix this bug, you must install the newest version of the optional MagpieRSS upgrade.

• FILTER PARAMETERS: Due to an old, old bug in WordPress 1.5.0 (which was what was available back when I first wrote the filter interface), FeedWordPress has traditionally only passed one parameter to syndicateditem and syndicatedpost filters functions — an array containing either the Magpie representation of a syndicated item from the feed, or the database representation of a post about to be inserted into the WordPress database. If you needed information about the feed that the item came from, this was accessible only through a pair of global variables, $fwpchannel and $fwpfeedmeta.

  Since it’s been a pretty long time since WordPress 1.5.0 was in widespread usage, I have gone ahead and added an optional second parameter to the invocation of the syndicateditem and syndicatedpost filters. If you have written a filter for FeedWordPress that uses either of these hooks, you can now register that filter to accept 2 parameters. If you do so, the second parameter will be a SyndicatedPost object, which, among other things, allows you to access information about the feed from which an item is syndicated using the $post->feed and the $post->feedmeta elements (where $post is the name of the second parameter).

  NOTE THAT THE OLD GLOBAL VARIABLES ARE STILL AVAILABLE, for the time being at least, so existing filters will not break with the upgrade. They should be considered deprecated, however, and may be eliminated in the future.

• FILTER CHANGE / BUGFIX: the array that is passed as the first argument syndicatedpost filters no longer is no longer backslash-escaped for MySQL when filters are called. This was originally a bug, or an oversight; the contents of the array should only be escaped for the database after they have gone through all filters. IF YOU HAVE WRITTEN ANY syndicatedpost FILTERS THAT PRESUME THE OLD BEHAVIOR OF PASSING IN STRINGS THAT ARE ALREADY BACKSLASH-ESCAPED, UPDATE YOUR FILTERS ACCORDINGLY.

• OTHER MINOR BUGFIXES AND INTERNAL CHANGES: The internal architecture of FeedWordPress has been significantly changed to make the code more modular and clean; hopefully this should help reduce the number of compatibility updates that are needed, and make them easier and quicker when they are needed.

Installation instructions

To upgrade an existing installation of FeedWordPress to version 0.99:

1. Download the FeedWordPress archive in zip or gzipped tar format and extract the files on your computer.

2. If you are upgrading from version 0.98 or earlier, then you need to create a new directory named feedwordpress in the wp-content/plugins directory of your WordPress installation, and you also need to delete your existing wp-content/update-feeds.php and wp-content/plugins/feedwordpress.php files. The file structure for FeedWordPress has changed and the files from your old version will not be overwritten, which could cause conflicts if you leave them in place.

3. Upload the new PHP files to wp-content/plugins/feedwordpress, overwriting any existing FeedWordPress files that are there. Also be sure to upgrade wp-includes/rss.php and wp-includes/rss-functions.php if you use the optional MagpieRSS upgrade, or don’t use it yet but do want to syndicate Atom 1.0 feeds.

4. If you are upgrading from version 0.96 or earlier, immediately log in to the WordPress Dashboard, and go to Options –> Syndicated. Follow the directions to launch the database upgrade procedure. The new versions of FeedWordPress incorporate some long-needed improvements, but old meta-data needs to be updated to prevent duplicate posts and other possible maladies. If you’re upgrading an existing installation, updates and FeedWordPress template functions will not work until you’ve done the upgrade. Then take a coffee break while the upgrade runs. It should, hopefully, finish within a few minutes even on relatively large databases.

5. If you are upgrading from version 0.98 or earlier, note that the old update-feeds.php has been eliminated in favor of a (hopefully) more humane method for automatic updating. If you used a cron job for scheduled updates, it will not work anymore, but there is another, simpler method which will. See Setting Up Feed Updates to get scheduled updates back on track.

6. Enjoy your new installation of FeedWordPress.