News-Aggregators

[rzach41]

Hey gang,
Curious to know if any of you have experience with news aggregators. I am starting on a side project where we will be using one to pull free content from other sites onto ours (If that wasn't already obvious enough). Anyways right now I am going through a long list trying to research some reputable ones. If any of you have a good service to recommend I would appreciate it!


List found here: NewsOnFeeds.com - List of News Aggregators

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[czach41]

Just to help clarify, I think the commonly used term for what he is referring to is a Web Scraper.

 

[LightHouse]

Hey gang,
Curious to know if any of you have experience with news aggregators. I am starting on a side project where we will be using one to pull free content from other sites onto ours (If that wasn't already obvious enough). Anyways right now I am going through a long list trying to research some reputable ones. If any of you have a good service to recommend I would appreciate it!


List found here: NewsOnFeeds.com - List of News Aggregators

why arregate all that? that is going to be quite a bit of server load, why not narrow it down to what your target market wants?

Just to help clarify, I think the commonly used term for what he is referring to is a Web Scraper.

that is not necessarily correct. a lot of arregators pull seprate RSS or ATOM feeds and mix them together. Most news sites have or have had RSS data feeds to pull from, the issue always being you do not get the full text.

 

[rzach41]

why arregate all that? that is going to be quite a bit of server load, why not narrow it down to what your target market wants?.

Hey Lighthouse,

Our site is going to have a constant stream of live, up to date information on a certain niche being added to our site every minute. Or at least that is the goal. The Niche will be narrowed down to what the target market wants. Right now I am in research stage and figuring out which CMS etc is appropriate to use on a site like this.

Once again in research mode, definitely haven't used anything like this in the past.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[rlucas]

It really depends on what you are trying to do. CSS and ATOM feeds are good if all you want is news data. If this is the case simply just subscribe to these feeds.

If you want to actually parse an entire page of data and store it in your database, than a web scraper is what you want. You can either build your own HTML parser or get one from Velocityscape.

You would have a task that lives on your web server that runs x amount of times a day (or every minute). This task goes out and parses your desired pages and stores the scraped data on your MySQL table. This is a custom table that you design to house all the data points that you want to record on each 'scrape'.

Try a download at : http://www.velocityscape.com/Products/WebScraperLite.aspx

 

[rzach41]

It really depends on what you are trying to do. CSS and ATOM feeds are good if all you want is news data. If this is the case simply just subscribe to these feeds.

If you want to actually parse an entire page of data and store it in your database, than a web scraper is what you want. You can either build your own HTML parser or get one from Velocityscape.

You would have a task that lives on your web server that runs x amount of times a day (or every minute). This task goes out and parses your desired pages and stores the scraped data on your MySQL table. This is a custom table that you design to house all the data points that you want to record on each 'scrape'.

Try a download at : Web Scraper Lite - Put the Web in a Spreadsheet

Thanks Ryan! Looking into this today. Rep ++

I'm also looking into XML/RSS/AJAX as well. All this stuff is a foreign language to me, much rather hire it out to web developers


On a side note, how are ya bud? I heard you're mixing nowadays, I'm jealous.