RSS feed for daily MR news

[ResearchTalk]

I wanted to track MR news in my blog reader but couldn't because the key publications didn't seem to support RSS.

So I've created a feed that you're welcome to use: http://feeds.feedburner.com/newsMR.

If you find it useful / want other sources included then do say so, hope you enjoy!

[Page Schorer]

Thank you very much.

[Ray Poynter]

Great work!

[Mario Menti]

That's useful, thanks! Am I right in thinking that at the moment this is just news from research-live.com? If yes, any chance of adding other MR news channels like mrnews and similar?

[ResearchTalk]

I'm having a prob w/ mrnews - it doesn't seem to be using a utf-8 chr set and so when I suck it in and try to sanitize, it still occasionally breaks the feed with invalid characters. I've tried googling for a way round this...not sure if you're able to help?

[Mario Menti]

What tool/technology do you use to scrape the content?

[ResearchTalk]

php (file_get_contents)

[Sam Winstanley]

do you have an example of a story thats knackered? The site is definately sending headers that suggest that its UTF-8 and its being produced from ASP.Net which means that somebody needs to go a bit out of their way to not send UTF-8.

very useful tool btw, currently I don't see any broken stories but I could be being unobservant. I prefer using curl e.g.:

$c = curl_init();
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_URL, $URL);
$contents = curl_exec($c);
curl_close($c);

this is partly because I used to scrape some content from a site that jammed from time-to-time and this caused me some stabilty problems with file_get_contents becuause it didnt' handle the timeout so nicely.

[ResearchTalk]

Cool. I'll run it daily and send you something as soon as I get a feed error (don't worry I won't mix it with current feed which will stay res live only until I get this issue resolved).

Can I run that curl within php without change/any version of php?

Glad you like it.

[Sam Winstanley]

I'd do a check that your server has the curl library (php4-curl or php5-curl) depending which you have set up, its likely you do have it though.

[Mario Menti]

As I've been playing around with the Google App Engine, I created a little app that scrapes http://mrweb.com/drno and returns a RSS 2.0 feed of the news items, complete with links and all that jazz - just go to http://mario-test.appspot.com (sorry for the lame URL, but during beta Google limit the amount of apps you can run on its framework) - this URL returns the DRNO feed you can directly subscribe to.

Not tested very much, and mainly a toy to let me experiment with the google app engine, so please let me know if there are any issues.

It's actually quite telling that neither of the main MR news sources have felt the need (or persumably had the demand) for providing RSS feeds. As an industry, we still have some way to go...

[ResearchTalk]

Wow, that was fast. Yeh, my feed validates fine so far: http://feeds.feedburner.com/temp001

Bummer I can't do on app engine - goog haven't bothered to sort out that error.

It's strange, many planners use blog readers!

[Simon Kendrick]

Thanks for that Mario - I've put it into Google Reader and it has picked out all of the news on the page at http://www.mrweb.com/news/

As this is only today's news, will these be wiped from the feed once the page is updated?