Hello everyone,

As mentioned in RSS bandwidth rules, the issues with bandwidth have to do with either server-side implementations (when the entire feed data is served regardless of whether there is a date component in the request) or client-side implementations (asking for an RSS update every two minutes). Further discussion here.

In order to alleviate server load, Charles Miller blogged on HTTP Conditional Get for RSS Hackers, which basically allows clients to only download a feed when there's new stuff in it, and servers to reply with a polite "no new stuff" answer when they have nothing new to tell.

I was wondering whether anybody had looked into implementing this for Xoops, and whether it's relevant at all to try to do so in PHP vs. Apache (there's a start of a discussion and PHP implementation of Conditional Get here).

Comments also appreciated.

Eric

Xoops RSS feeds are cached, so the server side is fine (I believe )

The whole thing about RSS feeds is the client side - IMHO, many popular websites will start pulling out RSS feeds due to enormous pressure on their servers. Offering RSS feeds to individual users sooner or later will come back to bite you.

Imagine 10,000 users checking your news feeds every two minutes - even the server returns 0 bytes, it still consumes processing power. You would intentionally create DOS attack on yourself.

Doesn't the caching feature only imply that feed content is reconstructed regularly to limit database access? Correct me if I'm wrong: the entire (cached) feed is still sent for every request that comes in to backend.php, right?

Conditional GET would send back a near-empty response to a feed request, something like:



According to RSS specifications, aggregators should send requests including two variables (namely HTTP_IF_MODIFIED_SINCE and HTTP_IF_NONE_MATCH). Checking the variables against actual RSS modification time would enable sending back a server-304 response when the feed doesn't contain any update.

While not all aggregators implement RSS specs as they should (yet), sending back 304 headers to responsible aggregators would be less bandwidth-expensive than sending back the whole feed with the last 10 news stories.

Consider Thunderbird's aggregator integration. Provided Thunderbird follows the specs (and I'm pretty sure it does), and even if it implements a minimal time between updates (doubtful), I'd much rather send a Bird user a 200-byte header than a 30Kb file.

As for processing power, I honestly think RSS is there to stay, even for individual users (Thunderbird is further proof that it's been widely accepted). RSS proxy networks might come in handy at some stage, providing relays to obtain feeds from. But that's outside of the scope of Xoops...

Eric

Conditional GET only solves the bandwidth issue, just one of two burdens on the server.

The server responds to every request. For 10,000 requests, the server will have to respond 10,000 times, does not matter if the server returns anything. 100,000 requests will have 100,000 reponses from the server. Doing a simple math, you get the conclusion. Imagine the server is reponding to 1,000 requests at the exact same time.

Proxy is merely a way of shifting burdens.

I guess what I'm driving at is ways to alleviate bandwidth for a XOOPS site, and whether implementing Conditional GET in backend.php would help at all.

Obviously that would do nothing for the number of requests coming for the backend feed. But it would do something for the number of bytes sent back in response to those requests.

Eric