Collaboration on Feed Radar

RTM is building systems that operate on data input from normal sources for RSS feeds. As an alternative to constructing our own filter for observing aggregators and a list of well-know feeds, recognizing that other startups have addressed and solved pretty much the same issue, we could pursue data-sharing or ping-partner of some kind.

What i propose is that in response for echoing your feeds by pinging us as they pass any first stage filter, is that we offer to inject the event-driven low granularity pings from the current radar into a middlelayer distribution built for PUB/SUB and for hi-volume document exchange where anyone interested in receiving joe's blue-flavored feeds can do so by calling an API and establishing a subscription.

Ping Process for echo service

  • any new feed passing your filter, captured by your Radar has Properties to include in echo Ping
  • Properties of each NewFeed: title, url
  • echo properties from each feed by connection to URI below:
  • http://rtmsrv.com/echo/feeds?title=%26url=
  • my servlet handles the above posts adding properties to buffered list
  • buffer spills to Filesystem as Document messages, bundled 1000 per msg
  • messages are the objects in hi-volume Doc exchange
  • Documents published on any Pub/sub topic and may route to any Subscribers, that have been granted priviledges by data filter owner

There is an assumption here that may not be valid if every developer with intentions to observe large chunks of feeds can simple go to weblogs and ingest their data. I think that you would agree that its not a trivial process to build observational radar for rss. Ive looked at some of the issues of bootstrapping a weblogs schema or a technorati schema and it is my feeling that its worth some due diligence and some "build or buy" evaluation before we go and try to do it all ourselves.