Massive Scale Cross-Border Amazon Arbitrage

[GoldFibre]

​

Weekly Goal​

Make my compliance bullet-proof

Daily Goals​

Deactivate account (holiday mode)
Nuke catalog
Start uploading smaller, safer catalog

Deactivating Account​

I woke up this morning to yet another order on an out of stock item that I had to cancel. I now have this beautiful mark on my account health page:


After that I put my entire account on 'holiday mode' which makes all of my offers inactive. Leaving my account up and hoping for orders is only making things worse.

Nuking Catalog​

I've got a bunch of problems right now compounding on each other:
1. I have no BuyBox winners, so I don't get sales on any 'good' products even when I'm competitive
2. My account health is suffering because I didn't have good enough filtering in place and I keep getting warnings for prohibited products
3. I have too big of a catalog, so I can't update on time and list many out of stock products, and I haven't applied my blacklists to most of my catalog yet
4. Since I have out of stock products listed where other sellers do not, this is where my sales are coming from and it's driving my cancelation rate up
5. The high cancelation rate also hurts my account health and takes me further from getting the BuyBox eligibility back

Yesterday I was considering nuking my account - deleting every single offer - and then restarting with a much safer list. I'm now sure this is the way to go, because my current catalog is a cancer that needs to be removed before I get perma-banned. I also don't have the ability to keep a catalog of that size up to date.

I will start with a smaller catalog of clean products that I can update several times per day and live with whatever sales I do or do not get. Since I still have no BuyBox eligibility, I can't expect much. In the meantime I will work on more speed optimizations and I will only expand my catalog if I can maintain my offer refresh rate.

It is also possible that my account is beyond saving at this point due to all the warnings. I don't really know. If that's the case, I still have my software and all my learnings in-hand and can potentially start a new account with another legal entity.

I understood that there are two ways to do this:
1. Delete 250 items at a time using the web interface. It takes about 30 seconds to do each page and I have about 2,400 pages
2. Upload templates - probably can delete everything using 4-5 spreadsheets

But then I found the nuke button:


So I uploaded a blank inventory template and nuked 600k offers that I have built up over the past two months or so.

Rebuilding Catalog​

I decided I want to cap the number of offers that I list today at 1,000. And I will apply significantly stricter restrictions on them.

Here is the new logic:
1. Don't list anything that also has an offer from Amazon US or Amazon UK
2. Only list if the product has an offer from two of the top competitors AND I can beat their prices
3. Deactivate listing if one of the top competitors also deactivates their offer (a mimic strategy)
4. Deactivate listing if top competitor ever prices below my minimum

So I know my top competitor pretty well and piggy-backing on their listings will improve my compliance. There is still some risk of selling out-of-stock products because I know they source from more than just Amazon US. This means that it is possible for them to list a product as in stock, but in Amazon US it is out of stock. Maybe they are tracking the same thing on walmart.com or another site, which I don't have the tech to do right now.

BUT if I do get a sale on something like that, I can still fulfill it (at a loss), because I could order it from my competitor and then ship it locally to my customer once I receive it. So I'm hoping this mimic strategy solves my order cancelation problem as well as my compliance problem.

The offer cap of 1k should also solve my refresh rate problem, because I will have a much smaller catalog, focused only on items that I have a good chance to win, especially if I ever get my BuyBox eligibility restored. The offers will get updated several times per hour.

All this will have been done at the cost of a huge percentage of sales. But 10% of something is better than 100% of nothing - which is where I was headed with my account at risk of perma-ban.

So I let the thing run for about an hour and I definitely met the bar of 'bullet-proof'. In fact I would probably consider it air-tight. I only had 7 offers after processing thousands of ASINs.

Loosening the Screws​

I'm happy with the outcome of very few products listed, because it's better to start too strict (with no risk) than to start too loose (and risk my account getting shut down).

Now I want to analyze and see what is the best way to slowly loosen the screws.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

I have a couple of questions and thoughts that come from a lack of understanding of what you are doing

The obvious thing to do would be do what everyone else does, pick a product that’s not available or maybe isn’t good in your market and build a brand

I assume you have decided not to do this for a reason, not sure what that reason is but whatever

You mention the possibility of excluding brands and trademarks

If you aren’t sourcing branded products

Why source them from Amazon?

Why not source them from aliexpress for example

Also I wonder if you could take all these listings and have your bot automatically create Shopify products and have them rank organically on google shopping

Could you share an example of one of these products you’ve sold? Or even just something similar

The reason I chose this model is that it is highly scalable and highly automated with minimal capital requirements.

A lot of your ideas are things that I think are possible down the road.

I both source from and sell on Amazon because the products are already cross-listed and share a data model through Amazon's identifier, ASIN. Listing from any other source would eliminate that identifier and I would have to fully populate the listings myself - upload images, descriptions and so on.

With my current logistics setup I fulfill in about a week. Aliexpress ships from China, is more expensive and takes longer.

I might also list on a Shopify store. I already have a bot from a previous iteration of this business that automatically uploads to Shopify. I found out while building it that Shopify has a daily limit on the number of new variations you can create.

I've sold mostly no-brand clothes. Everything else is miscellaneous with no real pattern: a wallet, a suitcase, parts for an RC car, etc.

 

[GoldFibre]

​

Weekly Goal​

Make my compliance bullet-proof

Daily Goals​

Analyze data from current compliance setup
Split name blacklist into name-only and name or description blacklists
Switch string search to regex word tests

Loosening the Screws​

I made some changes to the filtering based on Amazon's own rules and the severity of the possible violation. More risky words/phrases are searched for in both the name and the description, while the less severe sales jargon that Amazon bans from names is only searched for in names.

I will let this run overnight and then decide if/how to loosen the screws further.

 

[GoldFibre]

​

Weekly Goal​

Make my compliance bullet-proof

Daily Goals​

Add offer cap
Remove restriction of having 2 top sellers on the listing
Measure update frequency

Loosening the Screws​

I coded an offer cap mechanism and set the cap to 1,000 to start with. Then I removed my restriction for having two top sellers on the listing.

I'm also checking the update frequency. At the beginning of the day I had about 120 listings and they were updated every minute.

For now I want to be checking at least once per hour, so I won't release the offer cap unless I'm confident I can stay in that range.

I'm looking through all the listings and the stats from my bot to see where else I can tweak and keep my compliance high while also listing more products.

There have been no marks against my account yet since I nuked the catalog.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

​

Weekly Goal​

Make my compliance bullet-proof

Daily Goals​

Analyze data and improve filtering
Fill out selling applications
Only sell when no other offers are present

Raising Offer Cap​

At an offer cap of 1,000 my offers were getting updated every 8 minutes, so I increased the cap to 5,000. Now each offer is getting updated about once per hour. I have some ideas to dramatically improve the speed though. I will work on these over the weekend.

Selling Applications​

Over half of the products I've put offers on need a selling application. Right now the only way I have to do this is to manually go through each one. It only takes about 10-20 seconds, but there are thousands of these to do. I did a bunch of these - maybe 100 or so. Some of them are for many products at a time, so the effect is larger than 100 offers.

Offer Criteria​

To improve my account health I need orders, but I can't get orders if someone else wins the BuyBox, so I switched my listing criteria to only products on which there are no other offers. With this strategy nobody will be in the BuyBox, but my offer will be available if the user clicks 'see all buying options'.

This keeps my bot from wasting time updating offers that are never going to get me any sales. Once my BuyBox eligibility comes back I will definitely change this though.

Professional Advice​

I chatted with a law firm in the US that specializes in legal support for Amazon sellers. They said there is no way to know the reason for the restriction, but I need to focus on my account health and warnings. In the worst case it will take 180 days to get things back to normal - this would be if the restriction is due to that sex toy listing. It could come back earlier but there isn't a way to know.

 

[GoldFibre]

​

Saturday Goal​

Optimize offer creation/update flows for throughput

Daily Goal​

Refactor offer update flow
Store all needed values in DB
Split offer update into three optimized workflows

Account Status​

We back, boys!!

The account got unblocked in the afternoon and I almost choked on my protein shake. I thought there was a bug in my code, because in my DB it said I had BuyBox winners. But then I checked the seller portal and all my offers were featured, plus I already had one pending order.

Those two weeks of account closure felt like an eternity. I'm not going to take it for granted again!

Setting Weekend Goal​

I feel like I already accomplished my weekly goal during the work week, so I'm setting separate Saturday and Sunday goals. I get a lot more time do work on things during the weekend so I should be able to make major changes to my code.

Optimization​

I tried to do this a few weeks back but didn't fully implement it. I have one API call that is the bottleneck due to the rate limit and I can restructure things to call it less frequently.

This is the target:
1. Call it two times for new listings (source and destination)
2. Call it one time for cost/availability checks using polling (source)
3. Call it zero times for competitor changes on push notification (destination)

And once I'm at my offer cap, 1 won't be running.

So this would give me a theoretical maximum of:

        20   ASINS / call
  x      0.1 Call / second
  x      2   Accounts
  x  3,600   Seconds / hour
----------------------------
  = 14,400   Updates / hour

Currently all three workflows are calling the bottleneck API endpoint twice. First I got 3 to go from two calls down to one because it didn't require any DB modification. I'm not sure why I didn't do this earlier when I implemented the notifications API.

To get 2 down to one call and 3 down to zero calls I needed to make sure I store all the relevant information from each call in the DB, so it can be reused later in the pricing calculation. This is just adding a cache around the API.

First I got 3 down to zero calls by storing/updating/deleting the source offers whenever I fetch them from the API.

Getting 2 down to one call was similar, but with a bit more complicated object mapping.

After all these changes I got my oldest offer update on my 5,000 offers from 1 hour down to 20 minutes. So I upped the offer limit to 15,000, which is actually higher than my theoretical maximum.

While trying to get my offer count up to 15k I also found a number of things I want to filter out as well as filters to loosen up, so I did that on the fly while monitoring the bot.

Although I had 600k offers before I nuked my account, only a fraction of those were actually BuyBox winners. I'm currently only listing on products where my price is definitely the lowest, so my BuyBox rate should be much higher than before, which creates additional efficiency.

Other Optimization​

I was thinking of other ways to optimize besides just making more API calls or using fewer API calls to update.

It dawned on me that the reason that the top competitors never seem to bid on clothing is that clothing tends to have a lot of variations. Each variation has to separately be updated with availability and price based on the source, so it multiplies the cost to your API bandwidth.

Some clothing items have more than 100 variations. Would you rather have those 100 sizes/colors of a single shirt up to date, or would you rather have 100 distinct products up to date? I think it's obvious that the latter is better. In order for the former to be better you have to believe that the shirt gets purchased 100x as often as the average of the 100 unique products.

The other thing I was thinking is that if I can find a relatively good way to determine products unlikely to go out of stock, then I can slow down the refresh rate on those, but keep it high on products more likely to go out of stock.

I have a registered entity in the US. I could register as an Amazon seller in the US and then add the API as a worker on my bot, increasing throughput by another 50%.

I also researched residential proxy networks and scraping APIs to see what the cost would be. A fully managed solution will be too expensive, so this route would involve some development in fleet management and html parsing.

 

[GoldFibre]

​

Sunday Goal​

Optimize new offer listing flow

Daily Goals​

Fix whatever is wrong with new offer queue
Add earlier blacklisting when profiling new ASINs
Figure out how to automate selling applications

New Offer Queue​

I ran this overnight but it got stuck pretty quickly and stopped processing new offers. So this morning I worked on debugging it. I thought it was an issue with the SQL library but instead it was a bug with my rate limiter code.

Once that was fixed I let it run and the offer count went up steadily. It took pretty much the entire day to get to 15k.

Earlier Blacklisting​

Right now I'm blacklisting on offer creation and offer updates, but I have two earlier bots that collect some of the information used in the blacklisting process. So if I blacklist earlier, I will actually improve the efficiency of my new offer queue by excluding many ASINs that will never get an offer anyway.

Automating Selling Applications​

Roughly 50% of my offers are inactive because they require a selling application to be submitted manually through the seller portal. This takes about 20 seconds per application and is not the greatest use of my time. So I want to get this done more efficiently.

Here are my ideas:
1. Use browser automation w/ puppeteer - is there any risk of getting blocked for 'scraping'?
2. Use some python UI automation to click on the screen and check the text to pick the correct flow - harder to code, but safer
3. Mechanical turk - see if someone is willing to do a bunch of applications for a low cost

I started with approach 1 and after a few attempts I was able to submit the entire first page of ASINs in the list with a puppeteer script. Then when I made the page-switching code, Amazon decided to stop sending me OTPs to log in. So then I thought maybe I can switch my 2FA method to an authenticator app, but to even make that change I need to receive the SMS OTP.

Then I added a second of my email addresses as a user on my account and set that one up with the authenticator app 2FA. This let me log in as much as I want and run the script continuously.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[circleme]

That's basically Algo Trading but instead of trading stocks, you are trading physical products. That's really cool.

So maybe I need to resort to web scraping, which with Amazon is notoriously hard.

It's the same story with trying to get the available quantity. With the SP API you only know if something is in stock or not, but you don't know how many are available.

I've written a SaaS project a few year ago where I've compared the prices between two different marketplaces based on the ISBN, etc. It was a project during my studies, so it's been a while and I don't know how much has changed since then, but I had quite some success with faking headers as well as simulating the 999 trick. I've done the scraping part with Python & Beautiful Soup and the stock-tracking part with Selenium (like test-ordering 999 quantity of the given item in order to know the exact number of items in stock or if there are more then 999 items in stock) You will not know about the exact if it is above 999, but you will if it is beneath.

It was all quite challenging, but the selenium way worked at the end of the day.

Anyways, really nice progress thread.

Btw: Have you thought about selling the data as a SaaS solution instead of doing retail arbitrage. It kind of sounds like an "selling shovels in a gold rush" - opportunity the me.

 

[msufan]

Btw: Have you thought about selling the data as a SaaS solution instead of doing retail arbitrage. It kind of sounds like an "selling shovels in a gold rush" - opportunity the me.

THIS IS IT.

You sell the scraper tool and let others blacklist what they want, sort how they want, and do what they want with it.

 

[GoldFibre]

Btw: Have you thought about selling the data as a SaaS solution instead of doing retail arbitrage. It kind of sounds like an "selling shovels in a gold rush" - opportunity the me.

Yes, I've thought about it a little.

An Amazon employee told me there are about 20 players in the market right now doing the same model. The top competitor has custom software that is much larger in scope, so I couldn't sell a SaaS solution to them. The others are buying some software through a consultant that Amazon recommends. I got a demo of it from a non-technical person. My guess is that it runs on an EC2 instance and so is not SaaS and you need to pay a developer if it ever needs changes/upgrades. They pay one time to get the software set up, and then they have the source code and pay monthly for the infrastructure. I was quoted about 3k USD.

So I think it's a relatively small market here, but it's possible that there are more people who would get into it if the software was simpler to set up and I could sell it in any combination of markets.

But then I need to make a lot of changes to the software and also market it myself.

Right now I think the SaaS idea is premature - I'm still learning the requirements of the software through doing the business myself and I think I can make really good money like this. If down the line I feel like I've milked this business model enough I will look at other revenue streams; either SaaS or expanding beyond Amazon to Shopify or other marketplaces or expanding regions.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Today's Stats​

Revenue              0 USD
Profit               0 USD
Offers          16,015
BuyBox Winners   4,597

Workweek Goal​

Optimize product selection

Daily Goals​

Raise offer cap to 30k
Start new blacklist for low value keywords
Store number of siblings against ASIN

Low Value Keywords​

By 'low value', I mean low value to me: stuff that is unlikely to sell, and so I don't want to waste my API throughput on it.

I got rid of a lot of US-centric stuff and holiday themed stuff where the transit time would scare off customers.

I was up to about 20k offers, but this change caused the bot to remove a bunch of offers from junk I don't think I was going to sell, so now it's dropping again.

Siblings​

My theory is that the more variations a parent ASIN has the less value there is in this business model. Many ASINs have no variations, and some have more than 200. So I want to start storing this number somewhere and the do a bit of analysis before deciding where to draw the line on when not to add an offer.

After this change the very first set of siblings I found numbered 698 on some Columbia branded shirt. What a waste!

Time Crunch​

I had to work quite late at my full time job and so I only had about an hour for this project today. I think it made me work more efficiently.

 

[circleme]

Can u tell me/us a little bit more about your background and your status quo? It seems to me that you are working on something really promising with a lot of different opportunities but you don't have the time to push the project as fast as you might want to (just an assumption though). Are you currently working full time as an SWE?

If you don't mind, I would be curious.

 

[GoldFibre]

Can u tell me/us a little bit more about your background and your status quo? It seems to me that you are working on something really promising with a lot of different opportunities but you don't have the time to push the project as fast as you might want to (just an assumption though). Are you currently working full time as an SWE?

If you don't mind, I would be curious.

I think it's less about time investment and more about building the right thing.

Even after 100's of hours working on this code base it really isn't so complicated. I'm just learning what I need based on the feedback I get from my system and implementing whatever ideas come to mind.

I work in logistics technology, so this business is a pretty good match for my background. Most of my competitors are logistics companies that don't know much about software. I think my advantage over them is that I have zero cash outlay for development, while they have to pay to outsource.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Today's Stats​

Revenue              0 USD
Profit               0 USD
Offers          23,964
BuyBox Winners   6,224

Workweek Goal​

Optimize product selection

Daily Goals​

Run selling applications on more than one device
Combine offer processor queues
Examine desync issues

Selling Applications​

I have my puppeteer script that automatically submits the selling applications for all the brands, but even though I run it continuously it couldn't keep up with all the new products. I actually have two computers though, so I set it up on both to run at the same time.

Combine Offer Processor Queues​

Earlier my big idea for optimization was to have separate queues for high-priority products. But this doesn't really work well because they are ultimately sharing the same API rate limit. So I removed that code to focus on a single queue. I can add prioritization in other ways with a single queue, if I feel I need it later.

Desync Issues​

The count of offers to update in my queue is different from the number of offers in my cache is different from the offers in the seller portal. I'm not sure the root cause, but since the seller portal has the highest number, the risk is that some items don't get updated and I run into fulfillment issues.

So I downloaded a report from Amazon with all of my 'inventory' and uploaded the missing ASINs to my queue.

Fulfillment​

I had an order sitting in the warehouse since yesterday, but the warehouse guy didn't inform the delivery company even though we are all in the same chat. I told them about it yesterday but I guess I can't rely anybody else yet - I have to push and make sure it happens every delivery.

 

[GoldFibre]

Today's Stats​

Active Offers   14,971 / 30k goal

Revenue              0 USD
Profit               0 USD

Workweek Goal​

30k active offers

Daily Goals​

Blacklist brands that I cannot sell
Figure out how to scale up selling application submissions
Restart scraping keywords

Changing Workweek Goal​

I've only had one sale since getting my featured offer eligibility back, and the cause is that my catalog is so small after nuking it. I'm comfortable with the compliance I've put in place so the next thing is to ramp up my offers to start getting sales again.

I switched my workweek goal to getting to 30k active offers as reported by the Amazon seller portal, not how many I think I have based on my DB tables. This morning I was at 10,587 active offers, out of 33,822 total offers.

Most of the ones that are inactive are due to selling applications, which I'm currently using my script to automate. Some of these products I cannot sell at all and should be removed from the catalog because they are big name brands like Nike or Oakley.

So I think the surest path to getting to the 30k is to figure out a way to scale up my selling applications and to prune those brands which shouldn't be there.

Pruning Restricted Brands​

There is a list in the seller portal with all the open applications. These are essentially the ones my script failed to submit because they require proof of distribution rights. So I copied over a bunch of those brand names into my blacklist and restarted the bot.

Scaling Up Selling Applications​

Because I have to log in and input an OTP to start the script, I can't run it in headless mode, so I was assuming I can't run it two or three times on the same machine at once. But I didn't actually try it, so first I tried it out. It works under certain conditions, so for most of the day I was working on one machine, but running four copies of the script on the other.

While I sleep tonight I will try running four on each machine at the same time.

 

[maverick]

Just skimread this - sounds interesting.

But I see that you're making 0 revenue with 15k listings? I can do that with just 1 listing.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Just skimread this - sounds interesting.

But I see that you're making 0 revenue with 15k listings? I can do that with just 1 listing.

 

[GoldFibre]

Today's Stats​

Active Offers   22,175 / 30k goal

Revenue              0 USD
Profit               0 USD

Workweek Goal​

30k active offers

Daily Goals​

Debug workflow
Handle pricing compliance issues

Debug Workflow​

I noticed that my number of offers was going down when I thought that the only particular process that could do that was disabled, so I looked into it and found a pretty bodacious bug in my SQS handler that caused my bot to not only remove many offers but also to do incorrect pricing.

I fixed the bug and then reviewed other parts of my workflow, doing some refactoring along the way. I set up some test flows, because right now it can be hard to understand which paths an ASIN will take through the code.

In the end by fixing the bug and turning back on all my automation my active offer count went up pretty quickly. If the rate continues tomorrow I should hit 30k active offers.

Account Status​

It looks like at some point yesterday my featured offer eligibility was removed. Not much has happened in my account except for pricing issues caused by the bug above. So I'm hoping that fixing it an repricing correctly will be enough to bring back the eligibility.

Since this has already happened to me once, I'm not too worried this time around. I will just build up my catalog until it comes back and then it should come back together with a spike in sales.

 

[GoldFibre]

Today's Stats​

Active Offers   37,282 / 30k goal

Revenue              0 USD
Profit               0 USD

Workweek Goal​

30k active offers

Daily Goals​

Hit 30k offers
Make sure DB offer list aligns with Amazon offer list

Workweek Goal​

After running the automation overnight I was already over 30k active offers, and then after running it while I was at work all day put my over 35k. So my workweek goal is achieved.

While I'm ramping up active listings I'm not running the updater, so when I got home I turned the updater back on instead to make sure the pricing and the availability are up to date.

Aligning DB List w/ Amazon​

I made a script that will take an inventory report from Amazon and add the missing ASINs to my own processor queue. This runs in just a few seconds. I can probably do this once every few days to stay on top of it. There might even be a reports API that could automate it. But automating something that takes five minutes per week isn't top priority.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Today's Stats​

Active Offers      44,882

Revenue            107.12 USD
Profit (est.)       17.99 USD

Saturday Goal​

Fix the loose ends in my workflow

Daily Goals​

Fix destination offer cache
Find optimizations for notification processing
Start calculating profit each day

Fixing Destination Offer Cache​

I should have offers cached in my DB for everything on Amazon, but I can see in my logs that sometimes there is a cache miss. So I debugged this and fixed the de-sync as far as I can tell.

Now the total number I see in the seller portal is very close to what I have in the DB.

Optimizing Notification Processing​

I get notifications when there is any change to an offer, so they get processed with a pull mechanism. I'm nearing the point where I have so many offers that my processor can't keep up:


That chart shows that even with my processor running overnight it still hasn't cleared the queue from a few hours of downtime earlier yesterday.

So I added some conditions to recalculating the offer and it now appears to skip the majority of the notifications that it was previously processing for no reason.

Offer Cap​

I'm comfortable now with how all the systems are running, and so I upped the offer cap to 75k and turned the new offer processor back on.

Account Status​

Fourth day of featured offer ineligibility.

 

[GoldFibre]

Today's Stats​

Active Offers      58,447

Revenue                 0 USD
Profit (est.)           0 USD

Sunday Goal​

Build out tools for common tasks

Daily Goals​

Make tool for bulk price updates
Make tool or automation for order imports
Make tool for ASIN workflow assessment
Automatically poll for orders and add them to fulfillment spreadsheet

Bulk Price Updates​

I made a script that will update the offers on an arbitrary list of ASINs, which is something I do pretty frequently for a variety of reasons. I already did this through a generic test file, but I had to comment/uncomment code and format the list each time. Now I can just put the ASINs in a CSV and run the script.

Order Imports​

I made an importer that can take the orders report from Amazon and then imports them all.

I ran into the issue that some of my orders were from before I nuked my catalog, so my price and cost information was missing from the normal DB table. Luckily I had moved all those ASINs to a different table and used that to populate the details.

After importing everything I adjusted my profit calculations for the canceled orders I got and then calculated the gross profit. It looks like it is about $130 total across all orders. This is affected greatly by one cancelation on a very expensive item, which I now have with me and could potentially sell to regain some of that loss.

I'm not concerned at all about the low performance because this is an automated machine. Once it is tuned properly it should just be printing money with minimal effort: Controllable Unlimited Leverage

ASIN Workflow Assessment​

I have a bunch of SQL queries that I run to see what is going on with a given ASIN:
- Which queue(s) is it in?
- When was it last updated?
- Does it have an offer in the DB?
- Are the source and destination offers cached?
- Is it blacklisted and for what reason?

I spend a lot of time scrolling back and forth and running these queries. But they can be done all at once with a couple of joins. So first I just combined the queries into one.

Anyway it took me longer to write this description than to write the query, but I think it will be a big time save compared to how I was doing things before.

Next Steps​

I managed to do all this before 10am, and I had kind of run out of ideas on what to work on next. So I took some time to write down my ideas while I went on an ice cream walk.

I had my ice cream and then ended up walking about two miles. Here are the notes I took:

Next Steps:
1. Further optimize updater by delaying calls to getItemOffersBatch until other checks are cleared
2. Automatically monitor DB performance
3. Automate parts of fulfillment process
4. Also list on Shopify
5. Set up accounting integration and record all company costs
6. Analyze stats on number of siblings and find exclusion criteria
7. Expand to next marketplace
8. Analyze lowest prices and competitors on listings
9. Analyze sales ranks
10. Explore ads API and design some ad strategies
11. Source some working capital financing
12. Build a course on using the SP API
13. Build a course on Amazon drop shipping or cross border (this could lead to building a market for a SaaS solution based on my software)
14. Partner with my friend to do wholesale/FBA opportunities
15. Hire and train someone for fulfillment
16. Re-import offers from old ASIN list
17. Get my US seller account working
18. Exclude product types that are restricted
19. Move blacklists into DB
20. Refactor code into proper concerns
21. Add logging flags to classes
22. Migrate tech to cloud
23. Create some strategy to force competitor prices lower
24. Register another company
25. Start own brand to sell on Amazon and own store

In general I want to think about scaling and bottlenecks. Currently my bottleneck is the offer eligibility restriction, but this is outside my control and I just have to wait it out.

After that my bottleneck is how many offers I can keep reasonably up to date. And at some level of offers my orders will grow so much that my bottleneck will be managing the fulfillment.

If all of that is taken care of then my bottleneck will be the number of marketplaces I'm serving, so it will be a good time to expand to the next marketplace.

If I run out of marketplaces to serve then my bottleneck will be the channels I have, which right now is only one: Amazon

So right now I'm addressing the correct bottleneck by scaling up my offers. And it goes really fast. However this is fully automated now, and I don't have any ideas to speed it up, so I suppose I should work on my next bottleneck in the meantime: fulfillment

Fulfillment Improvements​

First I made a polling mechanism to pull orders from the SP API into my DB. Then I looked up how to manipulate Google Sheets over an API.

After a bit of wrestling with the permissions I got the spreadsheet to automatically populate, and then I added in whichever fields are available at the time of initial order.

I switched on the polling mechanism together with the Google Sheets integration and hoped that I would get an order today

Account Status​

Fifth day of featured offer ineligibility.

 

[GoldFibre]

Today's Stats​

Active Offers   81,599

Revenue              0 USD
Profit               0 USD

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API

Workweek Goal​

There are several opportunities to automate parts of the fulfillment process, so I'm going to start knocking those off one by one. This work will get reused once I eventually replace my fulfillment Google Sheet with an OMS.

Create AWBs via API​

The API docs were pretty straightforward. First I stubbed out all the calls I think I will use. Then I made the access token refresh and then worked on the create AWB API. The only real business logic is mapping the city/state values between Amazon's format and the courier's format.

And then I realized where I got stuck earlier trying to import orders - address and contact information is PII and there is some sort of authorization delegation involved that I couldn't sort out before. So I went back to the documentation to try to figure it out.

It turns out there was an option in the app settings to apply for access to PII and you have to fill out a lengthy security questionnaire. So I did that and now it is going through a manual review. Until that completes I won't be able to finish this feature.

Offer Ramp-Up​

I let the bot ramp up offers while I was at the office all day. The current cap is 100k offers.

Account Health​

Sixth day of featured offer ineligibility.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[biophase]

Today's Stats​

Active Offers   81,599

Revenue              0 USD
Profit               0 USD

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API

Workweek Goal​

There are several opportunities to automate parts of the fulfillment process, so I'm going to start knocking those off one by one. This work will get reused once I eventually replace my fulfillment Google Sheet with an OMS.

Create AWBs via API​

The API docs were pretty straightforward. First I stubbed out all the calls I think I will use. Then I made the access token refresh and then worked on the create AWB API. The only real business logic is mapping the city/state values between Amazon's format and the courier's format.

And then I realized where I got stuck earlier trying to import orders - address and contact information is PII and there is some sort of authorization delegation involved that I couldn't sort out before. So I went back to the documentation to try to figure it out.

It turns out there was an option in the app settings to apply for access to PII and you have to fill out a lengthy security questionnaire. So I did that and now it is going through a manual review. Until that completes I won't be able to finish this feature.

Offer Ramp-Up​

I let the bot ramp up offers while I was at the office all day. The current cap is 100k offers.

Account Health​

Sixth day of featured offer ineligibility.

Congrats on sticking to this by posting everyday and analyzing and solving your problems.

One day you’ll realize that you are way too smart and dedicated to be doing what you’re currently doing.

 

[GoldFibre]

Today's Stats​

Active Offers   85,782

Revenue            293 USD
Profit              38 USD
Gross Margin        13 %

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API
Confirm Amazon orders via API

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

Confirm Amazon Orders via API​

Once I get the AWB number I can confirm the order with Amazon. Normally this is copy-paste from my delivery partner's web portal into the Amazon web portal. But I always triple-check that I'm putting the correct number against the correct order, because you cannot change it once submitted.

Even though I haven't made an error yet, eventually it will happen if a human is performing the task. So automating this will eliminate one possible chance of error.

I don't have any sandbox environment set up for my app, so I can't really test this call without risking an unreversible error. But I understand how the SP API works and we are really just passing a single value plus the order info. So I added a command line prompt that displays the order number and the AWB number and asks me to confirm. That way I still have the human triple-checking in place until I feel confident enough to let it run on it's own.

Account Health​

Around noon today I got my featured offer eligibility back. It was gone for 5-6 days.

Just to be extra safe, once the restriction was lifted I stopped adding new offers and switched to updating existing offers.

 

[GoldFibre]

Today's Stats​

Total Offers    95,350
Active Offers   85,683
Revenue            417 USD
Profit              35 USD
Gross Margin         8 %

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API
Block items that take US voltage
Cap sibling count on offers

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

US Voltage​

I made a nice sale on an expensive item, but it is corded and runs only on US voltage. I'm worried the customer is not aware and will return it, handing me a $300 loss.

I messaged the guy to confirm he understands the power requirements. If he doesn't reply back to me I may just cancel the order to avoid the risk. I can see there are four local reviews on the product and all of them are one star claiming the product doesn't work (due to voltage).

There are voltage and power source attributes on the product API response, so I added these to the blacklist configuration to prevent the same issue from happening again.

Siblings Cap​

I added a cap of 500 siblings on my offers. I'm starting really high and then will slowly lower it so that I can keep my offer count high. If I start too low, then I will delist a bunch of my offers at once and it will take a long time to build them back up.

I also plotted a histogram to start the analysis:

Account Health​

Still have featured offer eligibility.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Today's Stats​

Total Offers   104,583
Active Offers   95,451
Revenue            399 USD
Profit              56 USD
Gross Margin        14 %

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API
Investigate scraping solution

Offer Cap​

I bumped the offer cap from 100k to 105k in the morning. I'll keep bumping about 5k each day until I feel like I'm getting close to a risky refresh rate on my offers.

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

Scraping Solution​

I didn't have a lot of time today, but I tried to cobble together a PoC using a proxy network. I think that ultimately I have to use a solution like this to scale.

Account Health​

Still have featured offer eligibility.

 

[GoldFibre]

Today's Stats​

Total Offers   108,774
Active Offers   99,891
Revenue            192 USD
Profit              20 USD
Gross Margin        10 %

Workweek Goal​

Automate more of the fulfillment process

Daily Goals​

Create AWBs via API
Set up minimum margin
Fix handling days edge case

Settings​

• Bumped offer cap from 105k to 110k
• Lowered max siblings from 400 to 300
• Set a new minimum margin of $5

Minimum Margin​

Now that I'm analyzing profit I notice that I have some sales with only $1-2 in gross profit. These are cheap items where there is competition, so I only take my minimum of about 6%. These aren't worth the API bandwidth, so I set a minimum margin of $5 on any product, even if it pushes the margin percent above my normal max of 25%.

Handling Days Edge Case​

I sold one product today where my offer had a really short handling time that shouldn't be possible. It turned out to be an edge case with product availability dates being set in the past, and it was an easy fix. Luckily I was able to pay for expedited shipping to get the product a bit faster and avoid Amazon slapping me for a late delivery.

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

Another Account​

An associate of mine is starting a separate Amazon business and asked me for some help. I'm going to offer to trade him my help in exchange for being able to use his API key to increase my bandwidth by 50%.

Account Health​

Still have featured offer eligibility.

New orders were paused on my account today after I got a couple of orders. Once I confirmed some of the orders the block was released. I have no idea if there is some ramp up period after the pause before it's fully back to normal or not.

 

[GoldFibre]

Today's Stats​

Total Offers   126,985
Active Offers  113,653
Revenue             61 USD
Profit               7 USD
Gross Margin        12 %

​

Daily Goals​

Investigate scraping solution

Scraping Solution​

I looked into various infrastructure setups to minimize cost while also having scalability as well as the security of the residential proxy - though I'm not 100% sure that the proxy is necessary.

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

​

Account Health​

Still have featured offer eligibility.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[GoldFibre]

Today's Stats​

Total Offers   146,989
Active Offers  130,837
Revenue             65 USD
Profit              14 USD
Gross Margin        21 %

​

Daily Goals​

Take real action on scraping solution

Scraping Solution​

I decided that I'm overthinking all this and was getting bogged down with trying to think of the perfect solution. I should just start with the simplest solution that does not use my home IP address and see if it works.

So once I stopped over-thinking and action-faking, I just made a Google Cloud Function directly in the cloud console that does some basic task and deployed it in less than an hour. But after a handful of attempts with just using HTTP GET, Amazon blocked the cloud function and it would only return error messages for any Amazon page.

The second attempt was to use Puppeteer and I tried from another Google Cloud region, but was quickly blocked again. I think this is likely due to the IP address, so will try next with a proxy. There is an API with the service I'm looking at that will let me pass in a URL along with UI actions to take and it will execute from the residential proxy.

PII Access Application​

My application is still under review by Amazon, so I can't finish the AWB creation yet.

​

Account Health​

Still have featured offer eligibility.

 

[GoldFibre]

Last Week Summary​

Revenue                       1,494 USD
Profit                          182 USD
Margin                           12 %

Avg Offers                  100,000
Revenue / 100k Offers         1,494 USD
Profit  / 100k Offers           182 USD

Orders                           19
Revenue / Order                  78 USD
Profit  / Order                  10 USD

I'll start posting weekly summaries to measure the output of my efforts.

This week I'm hoping to at least double the profit.

The key things I need to work on are increasing the maximum number of offers I can support as well as improving the average 'yield' on my offers by optimizing the criteria for making an offer. This is essentially the quantity and quality of my catalog.