corsasport.co.uk
 

Corsa Sport » Message Board » Off Day » Geek Day » [PHP] screen scraping


New Topic

New Poll
  Subscribe | Add to Favourites

You are not logged in and may not post or reply to messages. Please log in or create a new account or mail us about fixing an existing one - register@corsasport.co.uk

There are also many more features available when you are logged in such as private messages, buddy list, location services, post search and more.


Author [PHP] screen scraping
Sam
Moderator
Premium Member


Registered: 24th Dec 99
Location: West Midlands
User status: Offline
5th Nov 11 at 11:17   View User's Profile U2U Member Reply With Quote

What are the legalities of doing this?

I need to get pricing information off a particular website on a daily basis but they don't offer XML feeds or anything like that so at the moment I am having to manually copy and paste the prices from their website.

I've done a fair bit of PHP programming back in my web design/development days so I know how to do it, but my question is, is it OK to do this?

Guessing the answer is probably "no"
Ian
Site Administrator

Avatar

Registered: 28th Aug 99
Location: Liverpool
User status: Offline
5th Nov 11 at 12:33   View Garage View User's Profile U2U Member Reply With Quote

Depends what the information is and what you're going to do with it, it's probably protected as intellectual property but it's also in the public domain and your business case wouldn't necessarily harm the business of the place you get the data, which weakens their case.

Ryanair did have a case a while ago which I think is still ongoing.

I think in the first instance you would be told to stop rather than anything more serious. Speaking positively it could even lead to a proper relationship and proper data if you're actually driving business their way etc. in the case of price comparison or similar.
Sam
Moderator
Premium Member


Registered: 24th Dec 99
Location: West Midlands
User status: Offline
5th Nov 11 at 12:54   View User's Profile U2U Member Reply With Quote

Well I use this website's pricing to check against my own prices, and also to check whether they sell anything cheaper than my usual trade suppliers.

I also resell their products sometimes when my usual sources are out of stock, so I need to know when to adjust my prices as I don't want to be making a loss by selling lower than my cost price.
Ian
Site Administrator

Avatar

Registered: 28th Aug 99
Location: Liverpool
User status: Offline
5th Nov 11 at 13:10   View Garage View User's Profile U2U Member Reply With Quote

All of which is for your own purposes so it may not even be apparent that you're automating things unless they keep an eye on the access logs. Even then there's absolutely no issue with IP, it's just public domain price information which you are using to conduct your business.

I would think you'll be fine. The supermarkets do this type of thing and make TV adverts about it
Sam
Moderator
Premium Member


Registered: 24th Dec 99
Location: West Midlands
User status: Offline
5th Nov 11 at 14:22   View User's Profile U2U Member Reply With Quote

Nice one, cheers mate!
pow
Premium Member

Avatar

Registered: 11th Sep 06
Location: Hazlemere, Buckinghamshire
User status: Offline
5th Nov 11 at 17:52   View Garage View User's Profile U2U Member Reply With Quote

Just randomise the time a little
noshua
Member

Registered: 19th Nov 08
User status: Offline
7th Nov 11 at 22:08   View User's Profile U2U Member Reply With Quote

Did this for a couple website a couple month ago, I found this the easiest way;

http://php.net/manual/en/domxpath.query.php

ed
Member

Registered: 10th Sep 03
User status: Offline
8th Nov 11 at 08:20   View User's Profile U2U Member Reply With Quote

This is useful too: http://simplehtmldom.sourceforge.net/

 
New Topic

New Poll

  Related Threads Author Forum Replies Views Last Post
Screen help... Trucido Help Zone, Modification and ICE Advice 7 264
19th Feb 04 at 21:29
by Richie
 
dvd and screen corsa_griff Help Zone, Modification and ICE Advice 1 163
14th Sep 05 at 12:58
by djderek
 
Screen Scraping Legal Issues James Geek Day 7 290
6th Nov 07 at 16:14
by James
 
Someone posted a screen scraping proxy the other day Steve Geek Day 0 161
13th Nov 07 at 10:11
by Steve
 
Corsa B Lowered 60mm - Front Mudflaps Catching moka Help Zone, Modification and ICE Advice 11 364
11th Jun 09 at 15:20
by gregwalters
 

Corsa Sport » Message Board » Off Day » Geek Day » [PHP] screen scraping 28 database queries in 0.0107329 seconds