Legal matters aside, if you scrape data using your account, they will probably catch and ban you
Join over 90,000 entrepreneurs who have rejected the paradigm of mediocrity and said "NO!" to underpaid jobs, ascetic frugality, and suffocating savings rituals— learn how to build a Fastlane business that pays both freedom and lifestyle affluence.
Free registration at the forum removes this block.To address your last point, I would also be hesitant if someone were to scrape my data, but on the other hand, as long as I am not trying to directly compete with them, I do not see a big problem with it.
Why not tell them something generic? That you are looking to market to xy population for project z. It would be more honest and if you are not exactly the competition, then they will tell you their price.God bless you. Thank you for the info. If I buy data from them, they would need to know what I am doing with their data for ethical purposes. What do I tell them if I do not want to tell them my business idea?
Why not tell them something generic? That you are looking to market to xy population for project z. It would be more honest and if you are not exactly the competition, then they will tell you their price.
On the other hand, if you try stealing their data it may give you a lot of stress and may provide problems. I have had people asking for information and then putting it directly on their website. That doesn‘t make me want to work with them in the future.
If I go down the route of trying to partner with them. That is what I would tell them. However, to touch on your second point, I would be taking data from their website, visualizing it through easy to understand graphs/charts, and the putting it onto my website, all the while targeting a different market than the website I pull data from. I do not know if they would be interested in partnering with me if I am doing that.Why not tell them something generic? That you are looking to market to xy population for project z. It would be more honest and if you are not exactly the competition, then they will tell you their price.
On the other hand, if you try stealing their data it may give you a lot of stress and may provide problems. I have had people asking for information and then putting it directly on their website. That doesn‘t make me want to work with them in the future.
I think we are back at square one.Thanks for the advice. I was thinking the same thing. Just buy the monthly subscription to access the data, extract it using a data scraper, use the data to do what i am planning to do with it and then make my website/business around it. They will not know/care where I got my data from until I am big enough to compete with them, but by that time, I will be able to source my own data. Does that sound ethical?
I appreciate the insight. I would need to extract a lot of data to start out, but then once my website is up and running, I can look into partnerships once I have value/leverage. By using scraping and proxies, can I get a lot of data within a few weeks? I am trying to not get sued down the road if the website knows I got my data from them without asking. Of course, as stated, none of this would be personal information, or could hint at any personal information. Thanks again, God Bless.Scraping + proxies
It's not illegal, but most sites disallow in TOS.
Depends if this is a one-time data extraction, or something you'll need to do repeatedly.
If the latter,
Thanks for the information. If I did that, I would have to get them to sign NDA's.There's a lot of scraping software out there, otherwise you can hire people on Upwork to do it for you. It's not illegal.
The data is behind a paywall. I would have to subscribe to the website for a fixed amount a year to gain the data. Along with that, I would use the data by analyzing it and creating key trends and putting it out on my own website. Is this something that sounds like it can be illegal? I do have good intentions in helping people.It's how you use the data that you scraped that might be illegal rather than the scraping itself. If the data is publicly available to anyone it should'nt cause issues. If it's behind a paywall or login or something it's a more delicate question.
You can also check what the site allows by adding "robots.txt" after the url. This is more from an ethical point of view than a legal one.
"www.examplesite.whatever/robots.txt"
God bless you. Thank you for the info. If I buy data from them, they would need to know what I am doing with their data for ethical purposes. What do I tell them if I do not want to tell them my business idea?You would still need their consent in case you want a non-profit or charity. And if they consent they would probably have a more efficient way to provide data for you.
In the other case, try to find out if they act as a data provider, so you can buy data from them.
If no, tryto figure out where they get the data. Do they generate it themself? Or simply collect it? Or do they have a data provider? In the latter case you can buy the data from their provider.
Thanks for the advice. I was thinking the same thing. Just buy the monthly subscription to access the data, extract it using a data scraper, use the data to do what i am planning to do with it and then make my website/business around it. They will not know/care where I got my data from until I am big enough to compete with them, but by that time, I will be able to source my own data. Does that sound ethical?If they are into selling data to you, they will not care until you aren't a direct competitor.
Thanks for the further clarification. I will do some research into this today. I do not know whether they buy their data or if they source it themselves.I think we are back at square one.
In my experience, if you are using their data to make your own product, you cannot aproach it this way. You have to buy the data from them (accessing their platform is not equal to buying their data and then using it for you own profit)
You'll have to find out if they are willing to sell data trough an API or flat files to you.
Do you know if they source the data themselfs or use a data provider?
It might be worth doing a web search to see if there is a data provider in that field, whose business model is to provide the data to companies that need it rather than operating a platform.
For example for financial data Refinitiv is a data provider
Appreciate the comment. Agreed. I do not want to do anything unethical or borderline illegal. I am going to meet with a corporate attorney soon to discuss how I can go about tackling this problemLegal matters aside, if you scrape data using your account, they will probably catch and ban you
Thanks for the comment Mike. I did take a look at their ToS and EULA, but neither of them state if I can extract their data or not. From my understanding, if it goes unsaid, I should go about it like it is prohibited.I expect that the terms and conditions for signing up for the paid account will confirm what you can do with their data. "Illegal" is a broad term, and will probably vary depending on which country/countries you and the web site owner are in, but even if it's not an actual crime, I'd expect them to take some action if they find out you're doing it. The fact that it's behind a paywall suggests that there is some value to the data, so they're unlikely to take your theft of it without any action.
Once your site is done, how will you feel if someone scrapes all your data and uses it for something else?
It is not personal data. None of the data I need to collect would directly or indirectly link to anyone’s personal data. I just have to find the right angle to leverage a mutually beneficial partnership while I do not have much to offer while they have all the power.I guess you also have to consider what it costs them to gather and maintain that data set - whether they gather it themselves or license it from a data provider, it will be costing them. I don't know what the data is made up of - if it's personal details, for example, they may have agreed with each individual that they will not share the data with a third party.
Join Fastlane Insiders.