The Entrepreneur Forum | Startups | Entrepreneurship | Starting a Business | Motivation | Success

Link scanning for content sites

Accelerate wealth. Build a business that pays freedom. Join more than 70,000 entrepreneurs and register for the Fastlane Entrepreneur forum. Remove ads? Join the INSIDERS.

Rabby

Platinum Contributor
FASTLANE INSIDER
Read Millionaire Fastlane
I've Read UNSCRIPTED
Summit Attendee
Speedway Pass
Aug 26, 2018
1,534
4,512
1,133
Florida
After some pathetic troglodyte swiped a "suicide help" post from reddit and posted it in this forum, I'm kind of fuming. I admit they got me... I tried to post something helpful, but I was writing to a link-building flunky, probably offshore in a cubicle-center, or maybe a post bot. The WHOLE POINT was to sneak in later and add a link inside the original post, to make their shitty scam site more SEO -ified. Or maybe to feed traffic to a botnet installer or some other seedy garbage venture. It had an affiliate tag plastered to the end! Come on!

I hate to see people who behave that way make any profit. So for those who have content-based sites... how do you usually find insincere links like this? I imagine some platforms have it built in, but maybe it can get overwhelming on a site where a lot of things are posted. Maybe links that don't deserve to exist get missed?

If I built a tool that scanned your site from the outside, listed all the links and their locations, and ran some logic to help find the low quality / suspicious ones, would you use it? Would you need it or is it a solved problem?

What about duplicates? It wouldn't be too hard to detect duped, copied, plagiarized posts either. Would you use it?

I don't even have to build it, you can build it. I'll walk people through how to do it if I don't build the thing myself. I just hate to see these dirtbags make anything by wasting everyone's time and energy, stealing content, and preying on people's goodwill.
 

alexkuzmov

Silver Contributor
Read Millionaire Fastlane
I've Read UNSCRIPTED
Speedway Pass
Sep 20, 2019
735
922
335
Bulgaria
After some pathetic troglodyte swiped a "suicide help" post from reddit and posted it in this forum, I'm kind of fuming. I admit they got me... I tried to post something helpful, but I was writing to a link-building flunky, probably offshore in a cubicle-center, or maybe a post bot. The WHOLE POINT was to sneak in later and add a link inside the original post, to make their shitty scam site more SEO -ified. Or maybe to feed traffic to a botnet installer or some other seedy garbage venture. It had an affiliate tag plastered to the end! Come on!

I hate to see people who behave that way make any profit. So for those who have content-based sites... how do you usually find insincere links like this? I imagine some platforms have it built in, but maybe it can get overwhelming on a site where a lot of things are posted. Maybe links that don't deserve to exist get missed?

If I built a tool that scanned your site from the outside, listed all the links and their locations, and ran some logic to help find the low quality / suspicious ones, would you use it? Would you need it or is it a solved problem?

What about duplicates? It wouldn't be too hard to detect duped, copied, plagiarized posts either. Would you use it?

I don't even have to build it, you can build it. I'll walk people through how to do it if I don't build the thing myself. I just hate to see these dirtbags make anything by wasting everyone's time and energy, stealing content, and preying on people's goodwill.
There are services like this available, the one which comes to mind is ahrefs (which is pretty expensive, but worth the money if SEO is your focus).
This is the option they provide: How to see inbound and outbound links for your site
This is kind of blanket solution though and you learn about the bad links after the fact.
Its always better to have a built in option which when users add links adds the rel="nofollow" attribute by default. Ideally you would be able to configure which users have this enadbled on their accounts.
My personal favorite is to hide outbound links with elements.
You put the link inside a <label> for example, and you add a data-src attribute which is then made active by JS code.
This way you are 100% sure that no search engine would find the outbound link.

Google used to provide outgoing/outbound link data, but they have removed that option. (the current Search Console only provides backlink data, internal and external)
Not sure why since its very important when determening which links to leave and which links to remove.

To answer your question, yea, lots of people use tools to scan and keep their links in order, its a valuable service.
 

alexkuzmov

Silver Contributor
Read Millionaire Fastlane
I've Read UNSCRIPTED
Speedway Pass
Sep 20, 2019
735
922
335
Bulgaria
What about duplicates? It wouldn't be too hard to detect duped, copied, plagiarized posts either.
P.S.
This is extremly hard to do.
Finding duplicates of data across the internet is a gargantuan task.
Unless I didnt understand you correclty, you want a service which would pre-check entered data to make sure its not duplicated?
 

Rabby

Platinum Contributor
FASTLANE INSIDER
Read Millionaire Fastlane
I've Read UNSCRIPTED
Summit Attendee
Speedway Pass
Aug 26, 2018
1,534
4,512
1,133
Florida
P.S.
This is extremly hard to do.
Finding duplicates of data across the internet is a gargantuan task.
Unless I didnt understand you correclty, you want a service which would pre-check entered data to make sure its not duplicated?
I just happen to have recently worked on web crawlers for building private search repositories, and scanning sites for security vulnerabilities, bad content (like disclosing private information on public sites, etc), hijacked sites with affiliate links injected into them, and that sort of thing. Yes, it's harder than building a website, but it's doable (at least, if you do this sort of thing). Anti-plagiarism plugins are similar to the duplicate detection.

a service which would pre-check entered data
Not necessarily pre-checking. Just checking 'often enough', then flagging or alerting someone so that people can't sneak bad things into your site over time. Think of it this way... either crawling or reading from the API of your own site, analyzing new links and accounts for spamminess, maybe running a duplicate check on the most recent content.

Regarding ahrefs... no, not the nofollow tags. I'm thinking more about catching scam links that people put in profiles, forum posts, etc., especially when the user isn't a 'real' contributor. OR if their behavior changes for example, because their account is stolen and they suddenly start adding things to their old posts (maybe there's a time limit for editing on most platforms though). I don't know how often that sort of thing actually comes up though, so maybe nobody needs it as a discreet piece of software. Still might be something I would add to a generalized site scanner.

Thanks for the comments :)
 

alexkuzmov

Silver Contributor
Read Millionaire Fastlane
I've Read UNSCRIPTED
Speedway Pass
Sep 20, 2019
735
922
335
Bulgaria
Regarding ahrefs... no, not the nofollow tags. I'm thinking more about catching scam links that people put in profiles, forum posts, etc., especially when the user isn't a 'real' contributor.
Thats how you use it, yes.
Do you mean you want to bundle both the site scanner and the link checker service?
That I dont think exists as a bundle outside of some desktop apps like Screaming Frog (and even there I dont know if you could detect scam links automatically).
 

Rabby

Platinum Contributor
FASTLANE INSIDER
Read Millionaire Fastlane
I've Read UNSCRIPTED
Summit Attendee
Speedway Pass
Aug 26, 2018
1,534
4,512
1,133
Florida
Thats how you use it, yes.
Do you mean you want to bundle both the site scanner and the link checker service?
That I dont think exists as a bundle outside of some desktop apps like Screaming Frog (and even there I dont know if you could detect scam links automatically).
Yep, that's the idea. I'm not sure it's big enough to be a product all on its own but it seems at least potentially useful.
 

Sponsored Offers

MARKETPLACE Fox Web School "Legend" Group Coaching Program 2021
Hi - I sent out some emails with the updates but I'll PM you now also, thanks. The issue is...
  • Sticky
MARKETPLACE NEW: The Best School for Going Fastlane (Now open for summer enrollment)
Read the free book, some excellent insights. I also did the website quiz and I did a visual on a...
MARKETPLACE Not sure how to start? This free book will teach you how to build a successful web design business
Hi Fox. Starting the book and got through the introduction. Had a conversation with Andy Black...
  • Sticky
MARKETPLACE You Are One Call Away From Living Your Dream Life - LightHouse’s Accountability Program ⚡
Chris is super sharp and is aware of many facets of entrepreneurship and can help get your...
Introducing MJ's Personal Unscripted Network, Join Now for FREE!
Any chance to make it available outside of US? It has been available outside of the US on...

Forum Sponsor

Learn Fastlane Business Skills & Get Profitable Within 30 Days...

Get Started Now

New Topics

Fastlane Insiders

View the forum AD FREE.
Private, unindexed content
Detailed process/execution threads
Ideas needing execution, more!

Join Fastlane Insiders.

Top Bottom