| ASIN | Product | Price | Sellers | Buy Box | Category | Sales/Mo | Score |
|---|---|---|---|---|---|---|---|
| Category | Parent | Discovered | Checked | Eligible | Hit Rate | Pages | Scanned At | ||
|---|---|---|---|---|---|---|---|---|---|
No categories scanned yet. | |||||||||
New here? Take the interactive tour
A step-by-step walkthrough of every section — takes about 2 minutes.
Getting Started
ADAW Scanner is an Amazon FBA product intelligence tool that automatically discovers, analyzes, and scores products across Amazon's catalog to find profitable resale opportunities.
How It Works
- Phase 1 — Category Discovery: The scanner builds a tree of Amazon's browse node categories to identify leaf categories for scanning.
- Phase 2 — ASIN Discovery: For each leaf category, the scanner paginates through Amazon's catalog to discover product ASINs.
- Phase 3 — Eligibility Check: Each discovered ASIN is checked against Amazon's SP-API for pricing, fees, seller data, gating status, and sales rank. Products that pass all filters are marked "eligible."
Starting a Scan
- Click Start Scan in the top-right corner to begin scanning.
- The scanner will resume from where it left off — it remembers its progress across restarts.
- Click Pause Scan to stop at any time. Your data is saved automatically.
Dashboard Sections
Overview
Shows real-time scan progress: categories scanned, ASINs discovered, eligibility checked, and the eligible/ineligible split. Click "Overview" to collapse or expand this section.
Product Intelligence
Appears once you have eligible products. Shows aggregate analytics:
- Est. Monthly Profit — Per-seller profit after Amazon fees, summed across Green-rated products: (sale price − Amazon fees) × monthly sales ÷ seller count.
- Avg Fee Margin — Average percentage of sale price remaining after Amazon fees.
- Top Category — The category with the most eligible products.
- Avg Competition — Average seller count across eligible products.
- Quality Distribution — Breakdown of products by Green / Yellow / Red light score.
Eligible Products Table
A sortable, searchable table of all eligible products. Click any row to open the product detail modal.
- Use the search bar to filter by name, ASIN, or category.
- Toggle Green Light Only to see only top-rated products.
- Toggle Hide Amazon Buy Box to exclude products where Amazon holds the Buy Box.
- Click ★ Watchlist to show only starred products.
- Click the star icon (☆) next to any ASIN to add it to your watchlist.
- Use Compare checkboxes to select 2-3 products for side-by-side comparison.
- Click any column header to sort ascending/descending.
Keyboard Shortcuts
- / — Focus the search bar
- j / k — Navigate product rows up/down
- Enter — Open product modal for the focused row
- s — Toggle watchlist star on focused row
- ? — Show keyboard shortcuts help dialog
- Esc — Close modals and overlays
Recent Categories
Shows all scanned categories with their discovery count, checked count, and eligible count. Useful for understanding which categories yield the most opportunities.
Product Detail Modal
Click any product row to open the detail modal. It shows:
Keepa Historical Data
Keepa provides 90-day historical pricing, sales rank trends, and competition data. This data helps you understand whether a product's current metrics are typical or unusual.
Keepa Metrics Explained
How Keepa Enrichment Works
Keepa data is fetched automatically when products pass the eligibility check during scanning. Each eligible product is enriched with Keepa pricing, sales rank trends, and monthly sales estimates in real-time as the scan processes it. No manual enrichment is needed.
Keepa API — Important Notice
- Do not change the Keepa API key unless you know exactly what you are doing. The key is pre-configured and linked to a paid subscription plan.
- Do not modify Keepa settings in the code or environment variables. The token budget (1 token per product) and rate limits (20 tokens/minute) are calibrated for the current subscription tier.
- Token usage is automatic. The scanner manages its own rate limiting and will pause when tokens run low. You do not need to manually manage tokens.
- "Enrich All" button: This will consume tokens for every un-enriched eligible product. For ~300 products, this takes about 15 minutes and uses ~300 tokens. Only click it once — there is no need to click repeatedly.
- Cost awareness: Keepa API access is a paid service. Excessive manual enrichment requests or code modifications could increase token consumption beyond the monthly budget.
- If enrichment fails: Check the Keepa status bar for token count. If tokens show 0, wait a few minutes for the automatic refill (20 tokens/minute). Do not restart the app or modify settings.
Best Practice
Let the scanner enrich products automatically during scans. Use the "Enrich All" button only once after your first scan to backfill existing products. After that, new products are enriched automatically as they are discovered.
Green Light Score Guide
The Green Light Score uses a weighted 0–10 scale across five components:
- Profit Quality (up to 2.5): Margin percentage and net profit per unit — higher margins score higher on a continuous curve
- Sales Velocity (up to 2.0): Estimated monthly sales — uses Keepa's monthly sold when available, falls back to BSR estimate. Confidence penalty applied: HIGH (Keepa badge) = full weight, MED (BSR drops) = 0.7x, LOW (BSR formula) = 0.4x
- Competition (up to 2.0): Amazon Buy Box absence (+1.0) and FBA seller count — fewer FBA sellers score higher
- Price Point (up to 1.5): Sale price on a sliding scale — products $30+ score highest, under $8 score zero
- Keepa Trends (up to 2.0): Price stability, sales rank trend, and competition trend from Keepa historical data
ROI Bonus (up to +0.5): Products with high Max Buy Cost (30% ROI) receive a small bonus.
BSR Safety Penalty: Products with BSR > 500,000 receive a penalty (very slow-moving items).
Private Label Risk: Products with fewer than 2 historic sellers (90-day Keepa avg) receive a -0.5 penalty for possible IP/PL risk.
Amazon Buy Box Override: If Amazon holds the Buy Box, the product is automatically scored Red (0) regardless of other criteria.
Score Interpretation
- Green (7.0+): Strong opportunity. High margins, low competition, good demand.
- Yellow (4.5–6.9): Moderate opportunity. One or two concerns — investigate further.
- Red (0–4.4): Weak opportunity. Multiple red flags — likely not worth pursuing.
Data Export
Click the Export CSV button (top of the Eligible Products table) to download all eligible products as a spreadsheet. The export includes all columns visible in the table plus the Green Light score.
Tips & FAQ
- Why are some products marked ineligible? Products are filtered out if they are gated (restricted), have very low margins, are in excluded categories (Books, Kindle), or have other disqualifying traits.
- How accurate are sales estimates? The BSR-to-sales formula provides reasonable estimates but is not exact. Use it directionally — a product with 500 est. sales/mo is selling much better than one with 10/mo.
- Why does a product show "Error loading product data"? This can happen if the product was delisted or if some SP-API data fields are missing. The product data in the table is still accurate.
- Can I run multiple scans? Only one scan runs at a time. Pause the current scan before starting a new one.
- Does the scanner auto-save? Yes. All data is saved to the database in real-time. You can close your browser and come back — nothing is lost.
- What does "Retailers" mean? This column shows if Amazon is selling on the listing. Products with Amazon as a seller are harder to compete with.
Who We Are
ADAW LLC is an e-commerce distribution and fulfillment company headquartered in the San Francisco Bay Area, California. We specialize in Amazon FBA wholesale distribution and online arbitrage, leveraging proprietary technology and data-driven strategies to identify and capitalize on profitable product opportunities across Amazon's marketplace.
Our Mission
To democratize Amazon product research through automation and data intelligence. Our proprietary platform, DistroTrack (the scanner you are using right now), transforms what was once hours of manual product research into an automated, comprehensive analysis pipeline that scans Amazon's entire catalog.
What We Do
Wholesale Distribution
We source products from authorized distributors and brands at wholesale prices, selling through Amazon's FBA program for margin-positive returns.
Product Intelligence
Our DistroTrack platform scans Amazon's top-selling products to identify ungated, profitable, low-competition opportunities automatically.
Multi-Channel Fulfillment
We leverage Amazon's FBA infrastructure for storage, shipping, and customer service, enabling scalable operations without physical warehouse overhead.
Compliance & Licensing
Fully licensed for distribution across all 50 states with comprehensive insurance coverage and zero compliance violations on record.
Company Details
| Company Name | ADAW LLC |
| Headquarters | San Francisco Bay Area, California |
| Industry | E-Commerce Distribution & Fulfillment |
| Specialization | Amazon FBA Wholesale & Online Arbitrage |
| Technology Platform | DistroTrack (Proprietary Product Intelligence) |
| Brand Partnerships | 32+ Authorized Brands |
| Licensing | All 50 U.S. States |
Organizational Structure
Wholesale Distribution
Direct sourcing from authorized distributors and brand partners. We maintain relationships with 32+ brands across multiple product categories, purchasing at wholesale pricing for resale on Amazon. Our distribution network covers health & household, beauty & personal care, grocery, home & kitchen, and more.
Amazon FBA Fulfillment
End-to-end fulfillment through Amazon's FBA program — from product listing and prep to shipping, customer service, and returns handling. Leveraging Amazon's logistics network for 1-2 day delivery nationwide with Prime eligibility.
DistroTrack Intelligence Platform
Our proprietary product intelligence tool that scans Amazon's top-selling products via Keepa. Features include automated product discovery, eligibility checking, Green Light scoring, Keepa historical analysis, tiered enrichment, and comprehensive competitive analytics.
Compliance Management
Full regulatory compliance across all 50 states. We maintain distribution licenses, tax registrations, W-9 documentation, and comprehensive business insurance. Our compliance-first approach ensures zero violations and sustainable operations.
Market Intelligence & Analytics
Data-driven category analysis, pricing strategy, and competition monitoring. Integrated Keepa historical data provides 90-day price trends, sales rank trajectories, and competition changes. Smart Scan technology uses machine learning to prioritize the highest-value categories.
Multi-Channel Strategy
While Amazon FBA is our primary channel, our data infrastructure and product intelligence capabilities extend across e-commerce platforms. Category-level insights and brand relationships position ADAW for expansion into additional marketplaces.
Authorized Brands
ADAW LLC maintains authorized distribution agreements with 32+ brands across multiple product categories including health & household, beauty & personal care, grocery, and home & kitchen.
Brand Relationship Overview
Authorized Distribution
Direct wholesale agreements with brand owners or their authorized distributors, ensuring authentic products and competitive pricing at scale.
Ungated Access
Our Amazon seller account is approved (ungated) to sell in restricted brand categories, giving us access to products most sellers cannot list.
Category Diversity
Portfolio spans multiple Amazon categories — reducing risk through diversification and enabling us to capitalize on seasonal trends across niches.
Continuous Expansion
Our business development team actively pursues new brand partnerships, with a focus on high-margin, low-competition product lines identified by DistroTrack.
How Brand Data Integrates with DistroTrack
When the scanner discovers eligible products, it automatically detects and tracks the brand associated with each listing. This data feeds into the Brands tab, which provides:
- Brand-level analytics: Total products, average margins, and revenue estimates per brand
- Ungated highlighting: Brands we are authorized to sell are visually highlighted for quick identification
- Product drill-down: Click any brand to see all eligible products from that brand with full competitive analysis
- Export capabilities: Export brand-level data for sourcing and purchasing decisions
Compliance Overview
ADAW LLC maintains comprehensive compliance across all operational jurisdictions, ensuring legal and regulatory adherence for all distribution and sales activities.
| Credential | Status | Details |
|---|---|---|
| State Distribution Licenses | Active — All 50 States | Licensed for wholesale distribution in every U.S. state |
| Compliance Record | Zero Violations | Clean compliance record with no violations, citations, or penalties |
| W-9 Tax Documentation | On File | Current W-9 form on file with all distribution partners and platforms |
| Business Insurance | Active | Comprehensive general liability and product liability coverage |
| Amazon Seller Account | Professional — Good Standing | Active Professional seller account with full FBA enrollment |
| Business Entity | Active LLC | Registered Limited Liability Company in the state of California |
| Resale Certificates | Active | Valid resale certificates for tax-exempt wholesale purchasing |
Compliance Practices
Regulatory Monitoring
We actively monitor regulatory changes across all 50 states to ensure continued compliance with distribution and sales requirements.
Documentation Management
All licenses, permits, and compliance documents are tracked with renewal dates and maintained in a centralized system.
Amazon Policy Compliance
Full adherence to Amazon's seller policies, restricted product guidelines, and intellectual property requirements.
Brand Authorization
All products sold are sourced through authorized channels with proper documentation to prevent counterfeit or unauthorized listings.
What Is ADAW Scanner?
ADAW Scanner is a fully automated Amazon FBA product research tool built for wholesale and online arbitrage sellers. It scans Amazon's entire product catalog across thousands of categories, identifies products that are profitable, ungated, and low-competition, and presents them in a data-rich dashboard with actionable insights.
Unlike manual product research (which involves clicking through Amazon listings one by one), ADAW Scanner automates the entire pipeline:
Crawl Amazon's browse tree to find every leaf category
Search each category for products, collecting ASINs
Verify eligibility, pricing, fees, competition for each ASIN
Rate each product on a Green/Yellow/Red scale
The end result is a curated list of eligible products with detailed competitive analysis, fee breakdowns, sales estimates, and investment recommendations.
Alerts & Notifications: The scanner supports Discord Webhook alerts for high-quality Green Light products (score 7.0+ with 100+ monthly sales) and Browser Notifications for scan completion and Green Light batch discoveries.
How Scanning Works
The scan operates in three sequential phases, each building on the previous one:
Phase 1: Top Sellers Discovery
The scanner uses Keepa's Best Sellers API to fetch top-selling ASINs across all Amazon root categories. This costs approximately 36 Keepa tokens and returns 500K+ products ranked by sales demand.
Phase 2: Eligibility Pipeline
Discovered ASINs enter a multi-step eligibility pipeline. Key parameters:
- Batch processing: ASINs are checked in parallel batches
- Rate limits: Respects SP-API rate limits (1 request every 2 seconds for fees)
- Tiered enrichment: Green-scored products are enriched with Keepa data first
Phase 3: Eligibility Checking
Each discovered ASIN is evaluated through a multi-step pipeline. The scanner uses parallel API calls (threading) to maximize throughput:
Early Rejection
To save API calls, products are rejected early when they fail basic thresholds. However, offer details are still fetched even for rejected products (to detect Amazon on listing for scoring):
| Trigger | Threshold | What Happens |
|---|---|---|
| Low price | Sale price < $12.00 | Skips fee estimate (Wave 2 fees), still fetches offers for Amazon detection |
| Too many sellers | Seller count > 15 | Skips fee estimate, still fetches offers. Oversaturated listing. |
Early-rejected products still get basic data stored: estimated_monthly_sales, amazon_on_listing, fba_seller_count, buy_box_price, buy_box_seller. The scanner processes approximately 30-55 ASINs per minute depending on the eligibility rate.
Sales Estimation
Monthly sales estimates use verified Keepa data only. The Amazon "bought in past month" badge is never used as a primary signal — it counts unique customers (not units), shows only rounded ranges (50+, 100+, 200+), and is frequently stale.
| Priority | Source | Confidence | Description |
|---|---|---|---|
| 1 | BSR Drops (Keepa) | High/Medium | Each BSR rank drop ≈ 1 sale. Most reliable real-time signal from Keepa enrichment service. |
| 2 | BSR Formula | Medium | Category-specific power curves calibrated against SellerAmp data. Uses root category BSR from Keepa. |
| 3 | Pending | None | Product awaiting Keepa enrichment. No guessing — shows "Pending" honestly. |
Confidence multipliers affect the Sales Velocity score component: High = 1.0×, Medium = 0.7×, Low = 0.2×, None = 0.0×.
BSR Formula
The estimate_monthly_sales(bsr, category) function uses a power-curve formula calibrated against SellerAmp SAS ground truth (11 products cross-referenced, 73% error reduction vs prior calibration):
Category-specific (factor, exponent) pairs are defined for 27 categories (e.g., Home & Kitchen: 114069 × BSR^-0.77, Electronics: 237841 × BSR^-0.75). The formula requires the root category BSR from Keepa — subcategory BSRs from SP-API produce wildly inaccurate results.
Scoring & Green Light System
Every eligible product is scored using the Green Light Score, a composite rating based on real-world arbitrage viability criteria derived from extensive SellerAmp SAS analysis.
Hard Rejects (Instant Red)
This condition immediately disqualifies a product with a score of 0 (Red), regardless of all other criteria:
| Condition | Why |
|---|---|
| Amazon Holds Buy Box | FBA sellers cannot compete with Amazon on the Buy Box. Amazon will always win the sale. Products where Amazon is the Buy Box holder receive an automatic score of 0 and grade F (Avoid). |
Weighted Components (0-10 scale)
The total theoretical maximum is ~10.5 points, clamped to 10.0. Products without Keepa data are rescaled from 0-9.0 to 0-10.0 so they aren't unfairly capped below the Green threshold.
| Component | Max Points | What It Measures |
|---|---|---|
| Profit Quality | 2.5 | Sliding scale based on fee margin (15%-60%+), gated by net profit ($1-$3+). Uses PREP_COST ($0.20) constant. Margin 60%+ = 2.5, 50% = 2.0, 40% = 1.5, 30% = 1.0, 15% = 0.5. |
| ROI Signal | 2.0 | Based on max_buy_cost_30_roi (maximum you can pay and still hit 30% ROI). >$20 = 2.0, >$15 = 1.5, >$10 = 1.0, >$7 = 0.7, >$5 = 0.5, >$3 = 0.3. This is the key profitability anchor. |
| Sales Velocity | 2.0 | Continuous curve from 10 to 500+ monthly sales. Uses verified Keepa data only (BSR drops or BSR formula). Confidence-adjusted: high = 1.0×, medium = 0.7×, low = 0.2×, none = 0.0×. ≥500/mo = 2.0, ≥200 = 1.4, ≥100 = 1.0, ≥50 = 0.8, ≥20 = 0.5, ≥10 = 0.3. |
| Competition | 2.0 | Amazon absent (+1.0). FBA sellers: 0-2 (+0.7), 3-4 (+0.5), 5-7 (+0.3), 8+ (0). Falls back to total seller_count if FBA count unavailable: ≤5 (+0.5), ≤10 (+0.3). Single seller (≤1) gets -0.5 penalty. |
| Keepa Trends | 1.5 | Price stability CV <0.10 (+0.6), <0.15 (+0.4). Rank trend improving (+0.55), stable (+0.25). Competition decreasing (+0.35), stable (+0.15). Only available with Keepa enrichment. |
| Price Point | 0.5 | Light bonus: $50+ (0.5), $30+ (0.35), $20+ (0.2), $12+ (0.1). Higher prices absorb Amazon fees better. |
Penalties
| Modifier | Points | What It Measures |
|---|---|---|
| Single-Seller Penalty | -0.5 | Only 1 seller on listing = monopoly risk, no buy-box rotation. Not a hard reject but significantly lowers score. |
| BSR Safety Penalty | -0.5 to -1.0 | BSR > 500K (-0.5) or > 1M (-1.0). Extremely slow-moving products get penalized regardless of other signals. |
| Private Label Risk | -0.5 | Triggered when Keepa 90-day avg seller count < 2.0. Products with historically few sellers may be private label — risky for resale due to potential IP complaints. |
Score Labels
Top Opportunities Quality Gates
Products must pass ALL of these filters to appear in Top Opportunities:
| Gate | Condition | Why |
|---|---|---|
| No Amazon on listing | amazon_on_listing = false | Amazon as seller kills buy-box share for 3P sellers |
| No Amazon Buy Box | buy_box_seller ≠ "Amazon" | Even if Amazon isn't "on listing," they can hold Buy Box |
| Healthy competition | seller_count ≥ 5 | 5+ sellers = IP-safe distributed listing, healthy 3P presence |
| FBA seller present | fba_seller_count ≥ 1 | At least 1 FBA seller confirms the product is suitable for FBA arbitrage |
| Not Red score | score_label ≠ "Red" | Only Yellow and Green products pass (some viable criteria met) |
| Minimum profit | profit_per_unit ≥ $3.00 | Below $3 profit, FBA prep costs and shipping eat into margins |
Opportunity Score Formula
Products that pass all quality gates are ranked by a composite opportunity score:
Keepa Multipliers
If Keepa data is available, the base score is multiplied by trend factors:
| Factor | Condition | Multiplier |
|---|---|---|
| Sales rank trend | Improving (growing demand) | × 1.2 |
| Sales rank trend | Declining (shrinking demand) | × 0.8 |
| Price stability | CV < 0.15 (stable pricing) | × 1.1 |
| Price stability | CV > 0.25 (volatile pricing) | × 0.9 |
| Competition trend | Increasing (more sellers entering) | × 0.9 |
Letter Grades
Opportunity scores are converted to percentile-based letter grades among all candidates:
| Grade | Percentile |
|---|---|
| A+ | ≥ 85th percentile |
| A | ≥ 70th percentile |
| B+ | ≥ 50th percentile |
| B | ≥ 30th percentile |
| C | < 30th percentile |
The top 10 products by opportunity score are displayed on the Overview tab.
Category Intelligence
After running a Top Sellers scan, the Insights tab shows category-level analytics including hit rates, average margins, competition levels, and price sweet spots. Use this data to understand which product categories offer the best opportunities.
Prerequisites
| Requirement | Version | Why |
|---|---|---|
| Python | 3.11+ | Runtime for the backend (Flask, scanner engine, Keepa client) |
| Git | Any | Clone the repository, push deploys |
| GitHub Account | — | Host the repository, trigger Railway auto-deploys |
| Amazon Professional Seller Account | — | Required for SP-API access (Individual plan won't work) |
| Amazon SP-API Developer Registration | — | API credentials for product data, eligibility, fees |
Step 1: Amazon SP-API Setup
The scanner uses Amazon's Selling Partner API for all product data. You need 6 credentials:
- Register as SP-API Developer: Go to
sellercentral.amazon.com→ Apps & Services → Develop Apps → Register as developer. Choose "Self-authorization" (you're accessing your own seller account). - Create an IAM User: In the AWS Console, create an IAM user with
sts:AssumeRolepermission. Save the Access Key ID and Secret Access Key. - Create an IAM Role: Create an IAM role with a policy that allows
execute-api:Invokeonarn:aws:execute-api:*:*:*. Set the trust policy to allow your IAM user to assume the role. Copy the Role ARN. - Create an LWA App: In Seller Central → Develop Apps, create a new app. After approval, note the LWA Client ID and LWA Client Secret.
- Self-Authorize: Authorize the app on your own seller account. This generates a Refresh Token.
- Get your Seller ID: In Seller Central → Account Info → Your Merchant Token (also called Seller ID).
You now have these 6 values:
Also set your marketplace and seller:
Step 2: Supabase Auth Setup
Supabase provides user authentication (login/logout, JWT tokens, password reset). It's free for small projects.
- Go to
supabase.comand create a free account. - Create a new project. Choose a name and set a database password (you won't use the DB directly).
- Go to Settings → API and copy:
Project URL→ this is yourSUPABASE_URLanon / public key→ this is yourSUPABASE_ANON_KEYJWT Secret(under "JWT Settings") → this is yourSUPABASE_JWT_SECRET
- Go to Authentication → Providers and ensure Email is enabled.
- Go to Authentication → Users and create a user with email + password. This will be your login credential.
SUPABASE_JWT_SECRET empty, all authentication is skipped. Useful for local testing without a Supabase account.Step 3: Keepa API Setup (Optional)
Keepa provides 90-day historical data (price trends, sales rank, competition changes). Without it, the scanner still works but scores max at 8/10 instead of 10/10 (the Keepa Trends component adds up to 2.0 points).
- Go to
keepa.comand create an account. - Subscribe to the API Starter Plan (€49/mo). This gives you 20 tokens/minute.
- Go to your API settings and copy your API key.
Token budget: Each product query costs 1 token. At 20 tokens/min you can enrich ~1,200 products/hour.
Step 4: Clone & Install
The requirements.txt includes: flask, python-amazon-sp-api, gunicorn, PyJWT, keepa, numpy, python-dotenv, psutil.
Step 5: Configure Environment Variables
Create a .env file in the project root with all your credentials:
Step 6: Run Locally
Open http://localhost:5000 in your browser. If SUPABASE_JWT_SECRET is empty, you'll be logged in automatically.
The SQLite database (checked_asins.db) is auto-created on first run with all required tables and schema migrations.
Step 7: Deploy to Railway
- Push your code to a GitHub repository.
- Go to
railway.appand create a new project. Choose "Deploy from GitHub repo" and select your repository. - Add a persistent volume: In your Railway service, go to Settings → Volumes → Add Volume. Mount it at a path like
/data. SetDB_PATH=/data/scanner.dbso the database survives redeploys. - Set environment variables: In Settings → Variables, add all the env vars from Step 5 (with real values this time, including
SUPABASE_JWT_SECRET). - Add a custom domain (optional): In Settings → Networking → Custom Domain, point your domain to Railway.
- Railway auto-deploys on every push to
main. TheProcfiletells Railway how to start:web: gunicorn dashboard:app --bind 0.0.0.0:$PORT --workers 2 --timeout 120
Step 8: Verify Everything Works
| Check | How | Expected |
|---|---|---|
| Dashboard loads | Visit your URL | Login screen or dashboard appears |
| Login works | Enter Supabase user credentials | Dashboard loads with "Start Scan" button |
| Scan starts | Click "Start Scan" | Status shows "RUNNING", categories begin appearing |
| Products appear | Wait 5-10 min, check Products tab | Eligible products populate with prices, scores |
| Keepa enrichment | Click a product → "Refresh Live" | Keepa Insights section fills with trend data |
| Database persists | Redeploy (push a commit) | Products tab still shows previous scan data |
Complete Environment Variables Reference
| Variable | Required | Description |
|---|---|---|
SP_API_REFRESH_TOKEN | Yes | OAuth refresh token from self-authorization in Seller Central |
SP_API_LWA_APP_ID | Yes | Login With Amazon app client ID |
SP_API_LWA_CLIENT_SECRET | Yes | Login With Amazon app client secret |
SP_API_AWS_ACCESS_KEY | Yes | IAM user access key with sts:AssumeRole |
SP_API_AWS_SECRET_KEY | Yes | IAM user secret key |
SP_API_ROLE_ARN | Yes | IAM role ARN with execute-api:Invoke permission |
SELLER_ID | Yes | Your Amazon Merchant Token / Seller ID |
MARKETPLACE_ID | No | Amazon marketplace (default: ATVPDKIKX0DER = US) |
SUPABASE_URL | For auth | Supabase project URL |
SUPABASE_ANON_KEY | For auth | Supabase public/anon key (safe to expose) |
SUPABASE_JWT_SECRET | For auth | JWT signing secret. Leave empty to disable auth (local dev) |
ALLOWED_EMAILS | No | Comma-separated email allowlist. Empty = all authenticated users allowed |
KEEPA_API_KEY | No | Keepa API key. Without it, historical data unavailable but scanner still works |
DB_PATH | No | SQLite database file path. Default: checked_asins.db in script directory |
DISCORD_WEBHOOK_URL | No | Discord webhook URL for Green Light product alerts (score 7.0+ with 100+ sales) |
ENRICHMENT_MODE | No | Set to 1 on the enrichment worker to route Procfile to python enrichment.py |
ENRICHMENT_INTERVAL_HOURS | No | Hours between enrichment cycles (default: 4). Controls how often the enrichment service refreshes stale catalog entries. |
ENRICHMENT_DAILY_BUDGET | No | Maximum Keepa tokens the enrichment service can spend per day |
Project File Structure
How It Works
The Top Sellers scan uses Keepa's Best Sellers API to fetch the top-selling ASINs across all Amazon root categories. It directly pulls pre-ranked bestseller lists — getting you to the highest-demand products fast.
Process Flow
- Fetch Root Categories — Gets Amazon's ~37 root categories from Keepa (1 token)
- Pull Bestsellers — For each root category, fetches up to 100K top-selling ASINs (~1 token each, ~36 tokens total)
- Deduplicate — Merges results and removes ASINs already in your database
- Eligibility Check — Runs the standard SP-API eligibility pipeline on all new ASINs
- Enrichment — Keepa enrichment worker auto-enriches eligible products with sales data
Token Cost
| Step | Tokens | Notes |
|---|---|---|
| Root category lookup | 1 | One-time lookup |
| Bestseller queries (per category) | ~36 | 1 token per root category |
| Total | ~37 | Plus eligibility uses SP-API (separate budget) |
When to Use
- First-time setup — Great starting point. Gets you 500K+ high-demand ASINs instantly.
- Regular rescans — Only costs 37 tokens per scan. Run periodically to discover new top sellers.
- Quick discovery — Directly pulls pre-ranked bestseller lists from Keepa.
Safeguards
- Budget check — Won't start if you don't have enough Keepa tokens (need ~37).
- Stacking prevention — Won't start if >100K unchecked ASINs are already queued.
- Deduplication — Uses INSERT OR IGNORE to avoid duplicate ASINs in the queue.
- Budget exhaustion — If tokens run out mid-scan, it stops gracefully and proceeds with ASINs collected so far.
Technology Stack
ADAW Scanner is built with a modern, lightweight stack designed for reliability and speed:
| Layer | Technology | Purpose |
|---|---|---|
| Backend | Python 3.12 + Flask | Web server, API endpoints, scan orchestration |
| WSGI Server | Gunicorn (2 workers, 120s timeout) | Production HTTP server with max-requests recycling (50 + jitter 10) |
| Database | PostgreSQL (Railway) / SQLite (local dev) | Multi-tenant persistent storage. db.py auto-translates SQL between both dialects. |
| Amazon API | SP-API (python-amazon-sp-api) | Official Amazon Selling Partner API for eligibility, pricing, fees, offers, catalog |
| Historical Data | Keepa API | 90-day price/BSR trends, sales rank drops, monthly sold badge, competition tracking |
| Keepa Cache | keepa_cache table (global) | Persistent tenant-independent Keepa data cache. Enrichment worker processes stale entries via queue. |
| Authentication | Supabase Auth (JWT) | User login, token management, session security |
| Payments | Stripe | Scout ($49), Pro ($99), Enterprise ($179) subscriptions with webhooks |
| Frontend | Vanilla HTML/CSS/JS + Chart.js | Single-page dashboard with no build step. 8 tabs, product modal, real-time polling. |
| Hosting | Railway | Cloud deployment with auto-deploy on git push to main |
| Version Control | Git + GitHub | Source code management, CI/CD trigger for Railway auto-deploy |
dashboard.py) as inline HTML/CSS/JS. This means zero build tools, zero npm dependencies, and instant deploys. The dashboard is served as a single HTML page by Flask. This makes the system extremely portable and easy to maintain.
Architecture Overview
The system consists of several core files:
| File | Lines | Responsibility |
|---|---|---|
dashboard.py | ~9,000 | Flask web server: API endpoints, embedded HTML/CSS/JS frontend, Supabase auth, sales intelligence, scan control, scoring, all dashboard UI (8 tabs + help docs) |
finder.py | ~4,200 | Scanner engine: SP-API integration, browse tree discovery, ASIN scanning, eligibility checking, fee calculations, BSR-to-sales estimation, DB schema & migrations, ThreadPoolExecutor concurrency |
keepa_client.py | ~655 | Keepa API wrapper: historical pricing, sales rank trends, competition tracking, monthly sold extraction, token budget system, bug #221 monkey-patch |
db.py | ~430 | Database abstraction: auto-translates SQLite ↔ PostgreSQL (INSERT OR REPLACE, LIKE/ILIKE, bool casting). Multi-tenant support with tenant_id columns. |
stripe_config.py | ~160 | Stripe subscription plans (Scout/Pro/Enterprise), price IDs, plan limits, feature gates |
Data Flow
When a scan starts, dashboard.py spawns finder.py as a subprocess. The scanner writes results to the database as it runs. The dashboard polls the DB every 1 second to show live progress. All scan ↔ dashboard communication happens through the shared database.
Key Processes
| Process | How It Runs | Purpose |
|---|---|---|
| Top Sellers Scan | Subprocess: python finder.py bestsellers | Fetches 500K+ best-selling ASINs from Keepa (~36 tokens) then runs eligibility |
| Enrichment Worker | Subprocess: python enrichment.py | Queue-based Keepa enrichment — tiered priority (Green first) |
| Dashboard | Gunicorn: dashboard:app | Flask web server — serves UI, APIs, manages scan lifecycle |
Shared Catalog & Enrichment Service
A background enrichment service (enrichment.py) runs as a separate Railway worker, maintaining a global shared_catalog table with Keepa sales data for 550K+ ASINs. All tenants share this data — ADAW absorbs the Keepa API cost. The enrichment service runs on a 4-hour cycle with 60 tokens/min Keepa plan.
- Web service —
dashboard.pyvia Gunicorn (serves UI + API) - Enrichment service —
enrichment.py(background Keepa enrichment loop) - PostgreSQL — shared database between both services
Frontend Build Pipeline
Production serves 2 minified bundles built by node build.js: app.min.js (172KB) and app.min.css (141KB). Development mode serves 19 separate files (14 JS modules + 5 CSS files). The dashboard auto-detects which to serve based on static/dist/ existence.
System Diagram
Three Railway services share one PostgreSQL database. The web service runs the dashboard and scanner. The enrichment service runs Keepa data collection independently.
dashboard.py Flask API + embedded SPAfinder.py scan engine (subprocess)50+ API endpoints, 8 tabs, real-time polling
Gunicorn · 2 workers · 120s timeout
checked_asins productsshared_catalog Keepa datascan_categories browse treeMulti-tenant via
tenant_id
enrichment.py background workerKeepa API · 60 tokens/min
4-hour cycles · 20 ASINs/batch
Drops, trends, prices, competition
Scan Pipeline
When you click Start Scan, the system progresses through these phases:
Data Enrichment Cycle
The enrichment service runs independently on a 4-hour cycle, fetching fresh Keepa data for products in the shared catalog:
Sales Estimation Hierarchy
When displaying estimated monthly sales, the system uses verified Keepa data in priority order:
External Integrations
API Rate Limits & Performance
Amazon's SP-API enforces strict rate limits per endpoint. The scanner is designed to operate at maximum efficiency within these constraints:
| API Endpoint | Rate Limit | Used For |
|---|---|---|
| Catalog Search | 2 req/s | Discovering ASINs in categories (Phase 2) |
| Product Eligibility | 5 req/s | Checking if you can sell a product |
| Competitive Pricing | 2 req/s | Getting current listing price |
| Catalog Items | 2 req/s | Product details (name, BSR, brand) |
| Product Fees | 0.5 req/s | Amazon referral + FBA fees (slowest endpoint) |
| Item Offers | 0.5 req/s | Buy Box holder, offer count |
Throughput Estimates
- Category discovery: ~4 pages/second (limited by Catalog Search rate)
- Eligibility checking: ~30-55 ASINs/minute (depends on how many pass the $12 threshold)
- Top Sellers scan: ~2 minutes for discovery (500K+ ASINs), then eligibility pipeline runs
- Keepa enrichment: ~1,200 products/hour (1 token/product, 20 tokens/min on Starter plan)
Dashboard API Endpoints
The Flask backend exposes 50+ authenticated API endpoints. All require a valid JWT token via the Authorization: Bearer header (except the public waitlist POST and /health endpoint).
Stats & Analysis
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/stats | Aggregate scan statistics (products, categories, score breakdown) |
| GET | /api/stats/data-quality | Diagnostic: count products with missing fields |
| GET | /api/analysis | Product intelligence for dashboard charts and Top Opportunities |
Products & Details
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/products | Paginated list of eligible products (with search, filters, sorting) |
| GET | /api/product/<asin> | Single product detail for the product modal |
| POST | /api/product/<asin>/refresh-keepa | On-demand Keepa refresh for a single product |
| GET | /api/opportunities | Paginated top opportunities (sorted by composite score) |
Insights & Analytics
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/insights/categories | Per-browse-tree category intelligence (hit rates, scores, averages) |
| GET | /api/insights/price-analysis | Price sweet spots — product count & avg margin per price band |
| GET | /api/insights/keepa-trends | Keepa trend analysis for enriched eligible products |
Scan Control
| Method | Endpoint | Purpose |
|---|---|---|
| POST | /api/scan/start | Start Top Sellers scan (also aliased as /api/scan/start-bestsellers) |
| POST | /api/scan/stop | Stop the active scan |
| GET | /api/scan/log | Last 200 lines of scan.log |
| GET | /api/categories/list | Distinct category names from eligible products |
Scan History & Sessions
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/scan/sessions | All scan sessions, newest first |
| GET | /api/scan/sessions/<id>/products | Products from a specific scan session |
| POST | /api/scan/session/finalize | Finalize current scan session |
Keepa & Enrichment
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/keepa/status | Keepa API token balance & enriched product count |
| GET | /api/keepa/estimate | Token cost estimates: unenriched/stale/fresh counts & estimated time |
| POST | /api/keepa/reenrich | Smart re-enrichment with modes: stale_only, unenriched, green_first |
| GET | /api/keepa/reenrich/progress | Re-enrichment progress polling (status, done/total, percent) |
Export & Brands
| Method | Endpoint | Purpose |
|---|---|---|
| GET | /api/export/sas | ASINs as newline-separated text (for SAS batch lookup) |
| GET | /api/export/sas-csv | Full CSV with scores, pricing, competition data |
| GET | /api/brands | Auto-detected brand data with stats from scanned products |
| GET | /api/export/brands-csv | Export all brand listings as CSV |
Waitlist (Public)
| Method | Endpoint | Purpose |
|---|---|---|
| POST | /api/waitlist | Public: collect email for SaaS waitlist (no auth required) |
| GET | /health | Health check: returns {"status":"ok","db":"connected"} (no auth required) |
| GET | /api/diagnostic/shared-catalog | Shared catalog health: total ASINs, enrichment coverage %, freshness |
| GET | /api/watchlist | List user's watchlisted ASINs |
| POST | /api/watchlist/<asin> | Add ASIN to watchlist |
| DELETE | /api/watchlist/<asin> | Remove ASIN from watchlist |
| GET | /api/waitlist | Admin: view all waitlist signups |
Database & Data Model
Production uses PostgreSQL on Railway. Local development uses SQLite with WAL mode. The db.py abstraction layer auto-translates SQL between both dialects (INSERT OR REPLACE → ON CONFLICT, LIKE → ILIKE, boolean casting, etc). All tables include a tenant_id column for multi-tenant isolation. The keepa_cache table is the exception — it is global (shared across all tenants) to avoid re-fetching the same ASIN data per user. Schema migrations run automatically on startup.
Table: checked_asins (Products)
The main products table. Every ASIN that has been evaluated is stored here, regardless of eligibility.
| Column | Type | Description |
|---|---|---|
asin | TEXT PK | Amazon Standard Identification Number |
product_name | TEXT | Product title from catalog |
is_eligible | BOOLEAN | Whether seller can list this ASIN |
restriction_reason | TEXT | Why ineligible (NOT_ELIGIBLE, APPROVAL_REQUIRED, ASIN_NOT_FOUND) |
sale_price | REAL | Lowest New listing price from CompetitivePricing API |
total_amazon_fees | REAL | Referral + FBA fulfillment fee estimate at sale_price |
max_buy_cost_30_roi | REAL | Max buy cost for 30% ROI: (price - fees - $0.20) / 1.30 |
max_buy_cost_50_roi | REAL | Max buy cost for 50% ROI: (price - fees - $0.20) / 1.50 |
seller_count | INTEGER | Number of New condition offers on listing |
category | TEXT | Amazon browse-tree category display name |
bsr | INTEGER | Best Seller Rank (top-level category rank) |
estimated_monthly_sales | REAL | Monthly sales estimate (from 3-tier hierarchy) |
sales_data_source | TEXT | Which method estimated sales: "Amazon Badge", "BSR Drops", or "BSR Estimate" |
amazon_on_listing | BOOLEAN | Amazon is a seller on this listing (True/False/NULL) |
fba_seller_count | INTEGER | Count of FBA-fulfilled sellers specifically |
buy_box_price | REAL | Current Buy Box winner price |
buy_box_seller | TEXT | "Amazon" if Amazon holds Buy Box, else raw seller ID |
bsr_formula_version | INTEGER | BSR estimation model version (2 = current Books-calibrated) |
scan_session_id | INTEGER | FK to scan_sessions.session_id |
checked_at | TIMESTAMP | When this ASIN was last evaluated |
| Keepa Enrichment Columns (populated by Keepa API) | ||
keepa_avg_price_30 | REAL | 30-day average price (dollars) |
keepa_avg_price_90 | REAL | 90-day average price (dollars) |
keepa_price_stability | REAL | Price coefficient of variation (0-1, lower = more stable) |
keepa_sales_rank_avg_30 | REAL | 30-day average BSR |
keepa_sales_rank_avg_90 | REAL | 90-day average BSR |
keepa_sales_rank_trend | TEXT | "improving", "stable", or "declining" (recent vs historical BSR) |
keepa_offer_count_avg | REAL | 90-day average seller count |
keepa_competition_trend | TEXT | "increasing", "stable", or "decreasing" (seller count trend) |
keepa_buy_box_price | REAL | Latest Buy Box price from Keepa stats |
keepa_monthly_sold | INTEGER | Amazon "bought in past month" badge value |
keepa_sales_rank_drops_30 | INTEGER | BSR drops in last 30 days (~1 sale per drop) |
keepa_sales_rank_drops_90 | INTEGER | BSR drops in last 90 days |
keepa_sales_rank_drops_180 | INTEGER | BSR drops in last 180 days |
keepa_last_updated | TEXT | When Keepa data was last refreshed (ISO timestamp) |
Table: scan_categories (Browse Tree)
| Column | Type | Description |
|---|---|---|
node_id | TEXT PK | Amazon browse node ID |
node_name | TEXT | Category display name |
parent_node_id | TEXT | Parent category node ID (for tree hierarchy) |
is_leaf | BOOLEAN | True if searchable leaf category (vs folder node) |
scan_status | TEXT | pending, in_progress, done, or error |
asins_found | INTEGER | Count of ASINs discovered in this category |
pages_scanned | INTEGER | Search result pages scanned so far |
last_page_token | TEXT | SP-API pagination token for resume |
error_message | TEXT | Error details if scan_status is "error" |
created_at | TIMESTAMP | When category was added to the tree |
updated_at | TIMESTAMP | Last status change time |
Table: scan_asins (Discovery Queue)
| Column | Type | Description |
|---|---|---|
asin | TEXT PK | Discovered ASIN |
source_node_id | TEXT | Which browse node discovered it |
discovered_at | TIMESTAMP | When discovered |
eligibility_checked | BOOLEAN | False until processed by eligibility checker |
Table: scan_sessions (History)
| Column | Type | Description |
|---|---|---|
session_id | INTEGER PK | Auto-increment session ID |
started_at | TIMESTAMP | When scan started |
stopped_at | TIMESTAMP | When scan stopped (NULL if still running) |
status | TEXT | "running" or "stopped" |
scan_type | TEXT | "full", "targeted", or "smart" |
target_categories | TEXT | Comma-separated node IDs (for targeted scans) |
categories_scanned | INTEGER | Categories discovered this session |
asins_discovered | INTEGER | ASINs found this session |
asins_checked | INTEGER | ASINs evaluated for eligibility |
eligible_found | INTEGER | Products that passed eligibility |
not_eligible | INTEGER | Products that failed eligibility |
green_count | INTEGER | Score ≥ 7.0 |
yellow_count | INTEGER | Score 4.5–6.9 |
red_count | INTEGER | Score < 4.5 |
Table: scan_meta (Key-Value Store)
| Column | Type | Description |
|---|---|---|
key | TEXT PK | Metadata key (e.g., "scan_phase", "current_session_id", "daemon_status") |
value | TEXT | Metadata value |
Table: waitlist (SaaS Signups)
| Column | Type | Description |
|---|---|---|
id | INTEGER PK | Auto-increment ID |
email | TEXT UNIQUE | Signup email address |
created_at | TIMESTAMP | Signup time (default: now) |
Table: shared_catalog (Global Sales Data)
| Column | Type | Description |
|---|---|---|
asin | TEXT PK | Amazon product identifier |
monthly_sold | INTEGER | Amazon "bought in past month" badge value |
estimated_sales | INTEGER | Computed sales estimate after stale detection |
sales_source | TEXT | "Badge", "BSR Drops", or "BSR Formula" |
sales_confidence | TEXT | "high", "medium", "low" |
enriched_at | TIMESTAMP | When Keepa data was last fetched |
enrichment_count | INTEGER | Number of times enriched |
Table: watchlist (User Favorites)
| Column | Type | Description |
|---|---|---|
tenant_id | UUID FK | References tenants(id) |
asin | TEXT | Watchlisted product ASIN |
added_at | TIMESTAMP | When the product was starred |
notes | TEXT | Optional user notes |
Indexes
idx_sessiononchecked_asins(scan_session_id)— fast session-based product filteringidx_scan_categories_statusonscan_categories(is_leaf, scan_status)— fast pending-category lookupsidx_scan_asins_uncheckedonscan_asins(eligibility_checked)— fast unchecked-ASIN batch fetching
Keepa Integration
Keepa provides 90-day historical data for every product, enriching the scanner's real-time data with trends and context:
- Price history: Track price stability via coefficient of variation (CV). CV < 0.15 = stable pricing, CV > 0.25 = volatile
- Sales rank trends: Identify products with improving (growing demand) or declining (shrinking demand) sales rank over 90 days
- Competition changes: Monitor seller count fluctuations — increasing seller count triggers a -1 scoring penalty
- Monthly sold estimate: Keepa's proprietary "bought in past month" badge data, often more accurate than BSR-based calculations
- Sales rank drops: Count of 30-day rank drops (BSR dips that indicate sales events)
How It Works
Keepa data is fetched via the Shared Catalog enrichment service, which runs as a background worker refreshing sales data weekly. Each product lookup costs approximately 3 API tokens (1 base + 2 offers). ADAW absorbs the Keepa API cost — users don't need their own Keepa subscription. The Data Enrichment section on the Overview tab shows coverage and freshness.
Keepa data is now maintained by the Shared Catalog enrichment service (enrichment.py), which runs as a background Railway worker. The dashboard reads from the shared_catalog table first (Priority 0), falling back to per-tenant Keepa columns if unavailable.
Token Budget & Rate Limits
| Parameter | Value |
|---|---|
| Plan | Keepa Starter (€49/mo) |
| Rate limit | ~20 tokens/minute |
| Tokens per product | ~3 (1 base + 2 for buybox) |
| Throughput | ~6-7 products/minute, ~400 products/hour |
| Token wait logic | Sleeps 10-120 seconds when tokens insufficient, retries up to 3 times |
Monthly Sold: 3-Tier Priority
The scanner extracts monthly sales data from Keepa using a 3-tier priority to get the freshest value:
| Priority | Source | Description |
|---|---|---|
| Tier 1 | monthlySoldHistory | Array of timestamped snapshots — takes the last value (newest). This is the freshest source. |
| Tier 2 | product.monthlySold | Static field on the product object. May lag behind history data. |
| Tier 3 | stats.monthlySold | Stats-level aggregate. Last resort if both above are missing. |
Each tier is validated: value must be non-null and > 0 before use. If all three are missing, the scanner falls back to BSR-based estimation.
Live Refresh (update=1)
When the product modal requests live Keepa data, the enrichment worker processes the request via the enrichment queue. The Keepa API default is update=1, which refreshes data if it's older than 1 hour — matching SellerAmp SAS freshness.
Monkey-Patch for Keepa Library Bug #221
The official keepa Python library has a known bug (#221) where update_status() overwrites self.status from a proper Status dataclass to a raw Python dict, causing 'dict' object has no attribute 'refillRate' errors.
The scanner patches this in keepa_client.py with a replacement _safe_update_status() that detects dict corruption and reconstructs a proper Status dataclass with all attributes.
Numpy Type Safety (_safe_scalar())
Keepa returns numpy types (arrays, scalars) that fail Python boolean comparisons (if val and val > 0 raises "ambiguous truth value" errors). The _safe_scalar() utility converts numpy types to plain Python scalars before comparison. It handles: numpy integers/floats (.item()), single-element arrays, multi-element arrays (returns None), and passes through regular Python values unchanged.
Price & Rank Data Handling
- Prices stored in cents: Keepa stores all prices as integers in cents — divided by 100 for dollar values
- Price priority: Amazon price first, falls back to New (marketplace) price if unavailable
- Buy Box price: Extracted from stats index 17 (
BUY_BOX_SHIPPING) in the current array - Invalid values: -1 = out of stock/unavailable, NaN = no data. Both filtered before analysis
- Keepa epoch: Times stored as integer minutes since 2011-01-01 00:00:00 UTC
POST /api/keepa/backfill. Check enrichment status via GET /api/keepa/status. The scanner also auto-enriches products during scans.
Deployment & Infrastructure
Railway Platform
The application is hosted on Railway, a modern cloud platform:
- Auto-deploy: Every push to the
mainbranch on GitHub triggers an automatic deployment - Build time: ~1-2 minutes for Docker image build
- Container creation: ~3-5 minutes for new container startup
- Total deploy time: ~5-7 minutes end to end
- Zero-downtime: Railway performs rolling deploys (new container starts before old one stops)
- Persistent volume: SQLite database stored on mounted volume (
web-volume) that persists across deploys
Procfile & Gunicorn
The Procfile tells Railway how to run the app:
--workers 2— Two Gunicorn worker processes (sufficient for single-user dashboard)--timeout 120— 120-second worker timeout (important: long-running scan API calls can take time)$PORT— Railway automatically sets this environment variable
Environment Variables
All sensitive configuration is stored as Railway environment variables (never in code). See the From-Scratch Setup page for the complete reference table. Key groups:
| Group | Variables | Purpose |
|---|---|---|
| SP-API | SP_API_REFRESH_TOKEN, SP_API_LWA_APP_ID, SP_API_LWA_CLIENT_SECRET, SP_API_ACCESS_KEY, SP_API_SECRET_KEY, SP_API_ROLE_ARN | Amazon Selling Partner API credentials |
| Supabase | SUPABASE_URL, SUPABASE_ANON_KEY, SUPABASE_JWT_SECRET | Authentication (leave JWT secret empty for local dev) |
| Keepa | KEEPA_API_KEY | Historical data enrichment (optional) |
| App | MARKETPLACE_ID, DB_PATH, ALLOWED_EMAILS | App configuration |
Orphan Session Cleanup
When Railway redeploys, any running scan processes are killed. On startup, the application automatically calls _cleanup_orphaned_sessions() which:
- Queries all sessions with
status = 'running' - Updates each to
status = 'stopped'withstopped_at = current timestamp - Logs the count and IDs of orphaned sessions found
This prevents stale "Running" indicators on the dashboard after a redeploy.
Database Persistence
The SQLite database (checked_asins.db) is stored on a Railway persistent volume. The DB_PATH environment variable points to the volume mount location. If DB_PATH is not set, the database is created in the script's working directory. The database is auto-created with all tables and indexes on first run via init_db().
Enrichment Service
The enrichment service runs as a separate Railway worker using the same GitHub repo but with ENRICHMENT_MODE=1 environment variable. The Procfile conditional routes to python enrichment.py instead of Gunicorn.
Required variables: DATABASE_URL, KEEPA_API_KEY, ENRICHMENT_DAILY_BUDGET, ENRICHMENT_INTERVAL_HOURS, ENRICHMENT_MODE=1
Discord Webhook Alerts
Set DISCORD_WEBHOOK_URL on the web service to receive Discord notifications for high-quality Green Light products (score 7.0+ with 100+ monthly sales). The webhook is non-blocking and fires from the /api/analysis endpoint during dashboard polling.
Authentication & Security
The dashboard uses Supabase Authentication (an open-source Firebase alternative) for user management.
Supabase Setup
To set up authentication from scratch:
- Create a free project at supabase.com
- Enable Email/Password auth in Authentication → Providers
- Copy three values from Project Settings → API:
SUPABASE_URL— Project URL (e.g.,https://xxxxx.supabase.co)SUPABASE_ANON_KEY— Public anon key (safe to embed in frontend)SUPABASE_JWT_SECRET— JWT signing secret (under Settings → API → JWT Secret)
- Create your user account in Authentication → Users → Invite User
How Authentication Works
| Component | How It Works |
|---|---|
| Login | User submits email/password → Supabase validates → returns JWT + refresh token → stored in localStorage |
| API requests | All API calls go through authFetch() which attaches Authorization: Bearer <token> header |
| Server validation | @require_auth decorator decodes JWT using HS256 algorithm with SUPABASE_JWT_SECRET, audience "authenticated" |
| Token refresh | On 401 response, authFetch() calls _sb.auth.refreshSession(). If refresh succeeds, retries request with new token. If refresh fails, redirects to login screen. |
| Password reset | Email-based flow through Supabase's built-in password reset |
Email Allowlist
The ALLOWED_EMAILS environment variable restricts dashboard access to specific email addresses:
Comma-separated list. If empty or not set, all authenticated Supabase users can access the dashboard. If set, users whose email is not in the list receive a 403 "Access denied" response even with a valid JWT.
Local Development Mode
When SUPABASE_JWT_SECRET is empty or not set, the @require_auth decorator allows all requests through without authentication. This lets you develop and test locally without setting up Supabase:
authFetch() Flow
Every API call from the frontend JavaScript uses authFetch() instead of plain fetch():
@require_auth decorator checks token validity, expiration, and email allowlist. Expired tokens return 401 (triggering auto-refresh), invalid tokens return 401, and disallowed emails return 403.
Dashboard Tabs Explained
Overview Tab
Your command center. Shows real-time scan progress, key metrics, and quality distribution:
- Scan Progress: Phase indicator, categories scanned, ASINs discovered/checked, eligible count
- Discovery & Eligibility bars: Live progress bars showing Phase 2 (category scanning) and Phase 3 (ASIN checking) completion
- Product Intelligence: Estimated monthly profit, average fee margin, top category, average competition
- Keepa status: How many products have been enriched with historical data
- Quality Distribution: Visual breakdown of Green/Yellow/Red scores across all eligible products
- Top Categories & Price Distribution: Bar charts showing where your best opportunities cluster
Products Tab
The product research table. Every row is a fully eligible, ungated FBA product:
- Search: Filter by product name, ASIN, or category in real-time
- Green Light Only: Toggle to show only top-scored (Green) products
- Hide Amazon Buy Box: Exclude products where Amazon holds the Buy Box
- Category filter: Dropdown filter for focused research
- Price range: Set min/max price bounds
- Column sorting: Click any column header to sort ascending/descending
- Product Modal: Click any product row to see detailed analysis including Per-Seller Forecast, Competition Analysis, Keepa data, and Investment Thesis
- SAS Export: Export your product list as ASINs or full CSV for use with SAS (Seller Assistant) or other tools
New features include: Watchlist (star icon to save products), Product Comparison (select 2-3 products for side-by-side), Keyboard Shortcuts (press ? for help), and Mobile Card Layout (responsive cards on phones <768px).
Categories Tab
View all scanned Amazon categories with performance metrics:
- Discovered: How many ASINs were found in this category
- Checked: How many ASINs have been eligibility-checked
- Eligible: How many passed all checks (green badge = high count)
- Hit Rate: Percentage of checked ASINs that were eligible (higher = better category)
- Pages: How many pages were scanned in this category
- Scanned At: When this category was last scanned (relative time)
Insights Tab
Data analytics dashboard providing strategic intelligence about your scan results:
- Category Intelligence: Ranks all scanned categories by a composite score (0-100) based on hit rate, margin, sales volume, and competition
- Price Sweet Spots: Bar charts showing product distribution and green rate across price bands ($10-15, $15-25, etc.)
- Competition Landscape: Average seller count by category, Amazon/retailer presence rates
- Keepa Trend Analysis: Aggregated trend data showing improving vs declining products
- Smart Scan Plan: Preview of the AI-generated scan plan with tier breakdown before launching
History Tab
Complete record of every scan session:
- Session cards: Each card shows timestamp, type (Full/Targeted/Smart), status (Completed/Stopped), duration, and results
- Type badges: FULL (standard scan), TARGETED (specific categories), SMART (AI-optimized), PRE-HISTORY (data from before session tracking)
- Quality breakdown: Green/Yellow/Red count per session
- Click to drill down: Click any session card to see its products
Brands Tab
Brand-level intelligence across all your scan data. Uses a two-pass brand detection algorithm:
- Pass 1 — Known Brands: Matches product names against a curated dictionary of brand keywords. Single-word keywords (e.g., "Starburst") must start the product name. Multi-word keywords (e.g., "Arm & Hammer") can appear within the first 60 characters. Matched brands are labeled as Ungated.
- Pass 2 — Auto-detected: For unmatched products, extracts the first meaningful word of the product name (filtering out articles, colors, numbers). These brands are labeled as auto-detected.
- Minimum 3 products: Brands with fewer than 3 products are filtered out as noise
- Deduplication: Case-insensitive merging (e.g., "CLOROX" and "Clorox" become one brand)
Each brand card shows: product count, ungated status, parent company, categories, and all associated listings with prices, margins, sales, and scores. Brands are sorted by number of listings with 100+ monthly sales. CSV export is available for bulk analysis.
Top Opportunities (Overview Tab)
The Overview tab features a Top 10 Opportunities section — the highest-ranked products that pass all quality gates (no Amazon, 5+ sellers, 1+ FBA seller, non-Red, $3+ profit). Each card shows the opportunity score, letter grade (A+ through C), and key metrics. Products are ranked by the composite opportunity score which factors in sales volume, margin, competition, and Keepa trends.
Help Tab
Quick reference for the dashboard's features, filter options, modal fields, and scoring criteria.
FAQ & Troubleshooting
pip install -r requirements.txt, set SP-API environment variables (Supabase optional for local dev), then run python dashboard.py. Dashboard loads at http://localhost:5000 with no login required when SUPABASE_JWT_SECRET is empty.ATVPDKIKX0DER (Amazon.com retail, also the US marketplace ID) and A2R2RITDJNW1Q6 (Amazon.com Services LLC). If either appears as the Buy Box holder or in the offer list, the product is flagged as having Amazon on the listing.SaaS Scaling — Overview & Architecture
Last updated: 2026-03-02
scanner.adawllc.com from a single-tenant internal FBA scanner into a multi-tenant SaaS product listed on the Amazon Selling Partner Appstore, with Stripe subscription billing.
Why Go Multi-Tenant?
Right now, the scanner uses our own Amazon SP-API credentials (refresh token, seller ID, etc.) hardcoded in environment variables. Only we can scan. The eligibility endpoint (getListingsRestrictions) is per-seller — it returns whether that specific seller can list a product. So if another seller wants eligibility results personalized to their account, they must connect their own Amazon account via OAuth. That’s the fundamental driver for multi-tenancy.
Current Architecture (As-Is)
| Component | Technology | Details |
|---|---|---|
| Backend | Python 3.12 / Flask | dashboard.py (11,696 lines) |
| Scanner Engine | Python subprocess | finder.py (2,951 lines) |
| Keepa Client | Python | keepa_client.py (569 lines) |
| Database | SQLite (WAL mode) | checked_asins.db on Railway volume |
| Auth | Supabase JWT (optional) | HS256, audience="authenticated" |
| Frontend | Vanilla JS/HTML/CSS | Embedded in dashboard.py (~7,000 lines) |
| Deployment | Railway (3 services) | Web (Gunicorn), Enrichment (enrichment.py), PostgreSQL |
| Job Queue | None | Scan runs as subprocess |
| Cache | None | — |
Target Architecture (To-Be)
+---------------------------+
| Amazon Selling Partner |
| Appstore (discovery) |
+------------+--------------+
| OAuth authorize
v
+----------------------------------------------------------------+
| scanner.adawllc.com |
| |
| +----------+ +--------------+ +---------------------+ |
| | React | | Flask API | | Celery Workers | |
| | Frontend |-->| (REST + Auth)|-->| (per-tenant scans) | |
| | (Vercel) | | (Railway) | | (Railway) | |
| +----------+ +------+-------+ +----------+----------+ |
| | | |
| v v |
| +----------------------------------+ |
| | PostgreSQL (multi-tenant, RLS) | |
| | tenants, checked_asins, | |
| | scan_sessions, usage_tracking | |
| +----------------------------------+ |
| | |
| +--------+--------+ |
| | Redis | |
| | (Celery broker | |
| | + rate limit | |
| | + cache) | |
| +------------------+ |
| |
| +-------------+ +--------------+ +------------------+ |
| | Stripe | | Amazon SP-API| | Keepa API | |
| | (payments) | | (per-tenant | | (enrichment, | |
| | | | credentials)| | shared API key) | |
| +-------------+ +--------------+ +------------------+ |
+----------------------------------------------------------------+
What Changes From Current Architecture
| Aspect | Current (Single-Tenant) | Target (Multi-Tenant SaaS) |
|---|---|---|
| Database | SQLite on Railway volume | PostgreSQL with tenant_id on every table |
| SP-API Credentials | Hardcoded env vars (one seller) | Per-tenant encrypted refresh tokens in DB |
| Auth | Optional Supabase JWT | Required Supabase Auth + tenant provisioning |
| Scan Execution | Subprocess on same machine | Celery workers with per-tenant queues |
| Payments | None | Stripe Checkout + webhooks |
| Frontend | Embedded in Flask | Separate React app (Vercel) — later |
| Rate Limiting | Per-endpoint in-memory | Per-tenant token buckets in Redis |
| Keepa | Shared API key | Shared API key (cost absorbed or tiered) |
| User Onboarding | None | Signup → Pay → Connect Amazon → Scan |
Core Files (Current)
dashboard.py— Flask web server. Serves the entire UI (embedded HTML), all REST API endpoints, scan orchestration (start/stop/monitor), Supabase JWT auth, Keepa enrichment triggers, and stats/analytics.finder.py— Scanner engine. Runs as a subprocess spawned bydashboard.py. Three-phase scan: (1) download Amazon browse tree XML, (2) discover ASINs via CatalogItems API, (3) check eligibility via parallel API calls. Uses ThreadPoolExecutor with Wave 1 (3 threads) and Wave 2 (2 threads).keepa_client.py— Keepa API wrapper. Lazy init with monkey-patch for keepa library bug #221. Providesenrich_product()andenrich_batch(). Extracts 18 metric columns.backfill_offers.py,backfill_sales_estimates.py— Utility scripts for backfilling missing data.
Current Environment Variables
| Variable | Required | Purpose |
|---|---|---|
SP_API_REFRESH_TOKEN | Yes | Amazon SP-API auth |
SP_API_LWA_APP_ID | Yes | Login with Amazon app ID |
SP_API_LWA_CLIENT_SECRET | Yes | LWA client secret |
SP_API_AWS_ACCESS_KEY | Yes | AWS IAM access key |
SP_API_AWS_SECRET_KEY | Yes | AWS IAM secret key |
SP_API_ROLE_ARN | Yes | AWS IAM role ARN |
SELLER_ID | Yes | Amazon seller ID (A10MW1C5RN0DR7) |
MARKETPLACE_ID | No | Default: ATVPDKIKX0DER (US) |
KEEPA_API_KEY | No | Keepa API key |
SUPABASE_URL | Yes | Supabase project URL |
SUPABASE_ANON_KEY | Yes | Supabase anon key |
SUPABASE_JWT_SECRET | Yes | For JWT verification |
ALLOWED_EMAILS | No | Comma-separated email allowlist |
DB_PATH | No | Default: ./checked_asins.db |
Amazon SP-API Public App Registration
Last updated: 2026-03-02
What Is a Public SP-API App?
A public app is one that can be authorized by any Amazon seller (not just you). It gets listed on the Selling Partner Appstore inside Seller Central, where sellers discover and connect third-party tools. This is how every competitor (Jungle Scout, Helium 10, SellerAmp, BoxEm, etc.) integrates with Amazon.
Contrast with a private app: only works for your own seller account, doesn’t need Appstore listing, and is exempt from developer fees. Our current setup is effectively a private app.
Prerequisites
- Professional Amazon Seller Account ($39.99/mo from Amazon) — OR register via Solution Provider Portal without one
- Must be the primary account user (not a sub-user)
- AWS account with an IAM Role configured for SP-API
- Publicly accessible website (HTTPS) with privacy policy, terms of service, and clear business description
- Business email address
10-Step Registration Process
Step 1: Prepare
Read and understand these three documents before starting:
- Acceptable Use Policy (AUP) — What you can and cannot do with seller data
- Data Protection Policy (DPP) — Security requirements (encryption, pen testing, vuln scanning)
- Solution Provider Agreement — Legal terms for being an SP-API developer
Step 2: Create Solution Provider Portal Account
Go to https://developer.amazonservices.com and sign up. This is separate from Seller Central — it’s the developer portal where you manage your API applications.
Step 3: Create Developer Profile
- Identity verification (~20 min) — Amazon verifies your business identity
- Security control questionnaire — Answer questions about your data handling practices
- Use case description (under 500 words) — Describe what your app does. For us: “Product scanning tool that helps FBA sellers discover eligible products by scanning Amazon’s category tree, checking eligibility restrictions, analyzing competitive pricing, and calculating profitability with a proprietary Green Light scoring system.”
Step 4: Register Sandbox Application
Create a test application in the SP-API sandbox environment. This lets you make API calls against Amazon’s test data without affecting real listings.
Step 5: Make First Sandbox API Call
Verify connectivity by making a simple API call (e.g., getCatalogItem) against the sandbox. This proves your IAM role, credentials, and OAuth flow all work.
Step 6: Set Up Authorization Workflow
Implement the full OAuth 2.0 flow (Login with Amazon / LWA). This is the mechanism by which sellers authorize your app to access their data. See the OAuth Flow page for full details.
Step 7: Register Production Application
Create your production app and select the API roles you need:
- Product Listing — gives access to CatalogItems API and Listings Restrictions API
- Pricing — gives access to Product Pricing API and Product Fees API
Neither role is “restricted” (restricted roles involve PII and require extra security controls).
Step 8: Call SP-API in Production
Test with the version=beta parameter to verify your production app works with real data.
Step 9: Test Your Application
Validate all endpoints work with real seller data. Run through the full scan pipeline: browse tree download → ASIN discovery → eligibility check → pricing/fees.
Step 10: List Your Application
Submit to the Selling Partner Appstore for review. You’ll need:
- App name and description
- Feature list (bulleted)
- Target audience description
- Product URL (must be HTTPS)
- Pricing model (free, starting at $X/month, free trial with duration, etc.)
- Search page image + up to 6 screenshots
- Categories matching your registered app
Approval timeline: Amazon targets 3–4 weeks. If issues found, they contact via case log.
Key URLs
| Resource | URL |
|---|---|
| Solution Provider Portal | https://developer.amazonservices.com |
| SP-API Documentation | https://developer-docs.amazon.com/sp-api |
| Appstore Listing Guide | https://developer-docs.amazon.com/sp-api/docs/list-your-app-on-the-selling-partner-appstore |
Authorization Limits
| Stage | Max Seller OAuth Connections |
|---|---|
| Before Appstore listing approved | 25 sellers |
| After Appstore listing approved | Unlimited |
| Self-authorizations (for testing) | 10 |
OAuth Authorization Flow
Last updated: 2026-03-02
What Is OAuth and Why Do We Need It?
OAuth 2.0 is a protocol that lets a user (an Amazon seller) grant our app permission to access their Amazon data without sharing their password. Amazon’s implementation is called Login with Amazon (LWA).
We need OAuth because the getListingsRestrictions API (which checks if a seller can sell a specific product) requires a sellerId parameter that must match the seller whose OAuth token is being used. You literally cannot check Seller A’s eligibility with Seller B’s token.
Two Entry Points
Flow 1: From Your Website
User clicks “Connect Amazon Account” on scanner.adawllc.com, which triggers the OAuth flow.
Flow 2: From the Selling Partner Appstore
Seller finds your app inside Seller Central → Apps & Services, clicks “Authorize.” Same OAuth flow, different entry point.
Complete OAuth Flow (Step by Step)
1. User clicks "Connect Amazon Account" on scanner.adawllc.com
2. Your app redirects to Amazon consent page:
https://sellercentral.amazon.com/apps/authorize/consent
?application_id=YOUR_APP_ID
&state=RANDOM_CSRF_TOKEN
&redirect_uri=https://scanner.adawllc.com/api/auth/amazon/callback
3. Seller logs into Seller Central (if not already logged in)
4. Amazon shows consent page listing requested permissions
5. Seller clicks "Confirm"
6. Amazon redirects back to YOUR redirect URI with params:
?spapi_oauth_code=AUTH_CODE
&state=RANDOM_CSRF_TOKEN
&selling_partner_id=SELLER_ID
7. Your backend validates the state parameter (CSRF protection)
8. Your backend POSTs to https://api.amazon.com/auth/o2/token:
{
"grant_type": "authorization_code",
"code": "AUTH_CODE",
"redirect_uri": "https://scanner.adawllc.com/api/auth/amazon/callback",
"client_id": "YOUR_LWA_CLIENT_ID",
"client_secret": "YOUR_LWA_CLIENT_SECRET"
}
9. Amazon returns:
{
"access_token": "short-lived (1 hour)",
"refresh_token": "long-lived (365 days)",
"token_type": "bearer",
"expires_in": 3600
}
10. Store refresh_token ENCRYPTED (AES-256) in tenants table
11. User is now connected — can start scanning
Token Lifetimes
| Token | Lifetime | Notes |
|---|---|---|
| Authorization code | 5 minutes | Must exchange immediately after user authorizes |
| Access token | 1 hour | Use for API calls, refresh when expired |
| Refresh token | 365 days | Must re-authorize annually. Amazon sends reminder 30 days before expiry |
At Runtime (Making API Calls for a Tenant)
1. Retrieve tenant's encrypted refresh_token from PostgreSQL
2. Decrypt refresh_token
3. POST to https://api.amazon.com/auth/o2/token:
{
"grant_type": "refresh_token",
"refresh_token": "TENANT_REFRESH_TOKEN",
"client_id": "YOUR_LWA_CLIENT_ID",
"client_secret": "YOUR_LWA_CLIENT_SECRET"
}
4. Receive new access_token (valid 1 hour)
5. Use access_token in Authorization header for SP-API calls
6. Cache access_token in Redis (TTL: 50 minutes) to avoid unnecessary refreshes
Security Requirements
- Two separate
stateparameters used to prevent CSRF attacks — validate both - Refresh tokens must be stored encrypted at rest (AES-256 or similar)
- Never log or expose tokens in error messages
- Handle token expiry gracefully — if 401 returned, refresh and retry once
- The
selling_partner_idreturned in the callback is the seller’s ID — store it in thetenantstable
SP-API Roles & Permissions
Last updated: 2026-03-02
What Are SP-API Roles?
When you register an SP-API application, you select which roles your app needs. Each role unlocks specific API endpoints. Amazon has two categories:
- Non-restricted roles — Product data, pricing, fees. No PII involved. Easier approval.
- Restricted roles — Orders, shipping, buyer messaging. Involves PII (names, addresses). Requires additional security controls and justification.
Roles We Need (Both Non-Restricted)
| Role | APIs Unlocked | Restricted? |
|---|---|---|
| Product Listing | CatalogItems API (searchCatalogItems, getCatalogItem), Listings Restrictions API (getListingsRestrictions) | No |
| Pricing | Product Pricing API (getCompetitivePricing, getItemOffers, getItemOffersBatch), Product Fees API (getMyFeesEstimate, getMyFeesEstimates) | No |
Neither role is restricted because our use case avoids PII entirely — we only deal with product data, pricing, and eligibility.
What Each API Does in Our Scanner
| API | Endpoint | Purpose in Our Scanner |
|---|---|---|
| CatalogItems | searchCatalogItems | Discovery phase: find ASINs by category/keyword |
| CatalogItems | getCatalogItem | Get product name, category, BSR |
| ListingsRestrictions | getListingsRestrictions | Check if THIS seller can list THIS ASIN |
| CompetitivePricing | getCompetitivePricing | Get sale price, seller count (batch of 20) |
| ItemOffers | getItemOffers / getItemOffersBatch | Amazon on listing? FBA count, Buy Box info |
| ProductFees | getMyFeesEstimates | Referral fee, FBA fee, total (batch of 20) |
| Reports | GET_XML_BROWSE_TREE_DATA | Download category tree (one-time per scan) |
Critical: Eligibility Is Per-Seller
getListingsRestrictions requires the sellerId parameter and it MUST match the seller whose OAuth token is being used. You cannot check Seller A’s eligibility with Seller B’s token. This is the fundamental reason each user must connect their own Amazon account.
Eligibility varies per seller because of:
- Brand approvals (ungating)
- Category gating (e.g., Grocery, Topicals, Health & Beauty)
- Hazmat certifications
- IP complaint history
- Account age and performance metrics
SP-API Rate Limits & Developer Fees
Last updated: 2026-03-02
Rate Limits Are Per App+Seller Pair
Rate Limit Table
| API Operation | Rate (req/sec) | Burst | Items/Request | Effective Throughput |
|---|---|---|---|---|
getListingsRestrictions | 5 | 10 | 1 ASIN | 5 ASINs/sec |
searchCatalogItems | 2 | 2 | 20 ASINs/page | 40 ASINs/sec |
getCompetitivePricing | 0.5 | 1 | 20 ASINs | 10 ASINs/sec |
getItemOffers | 0.5 | 1 | 1 ASIN | 0.5 ASINs/sec |
getItemOffersBatch | 0.5 | 1 | 20 ASINs | 10 ASINs/sec |
getMyFeesEstimates | 0.5 | 1 | 20 ASINs | 10 ASINs/sec |
Bottleneck Analysis (Single Seller)
The bottleneck is getListingsRestrictions at 5 ASINs/sec (no batch endpoint exists).
| Scan Size | Eligibility Time | Pricing Time | Total Estimate |
|---|---|---|---|
| 100 ASINs | ~20 sec | ~10 sec | ~30 sec |
| 1,000 ASINs | ~200 sec (3.3 min) | ~100 sec | ~5 min |
| 10,000 ASINs | ~33 min | ~17 min | ~50 min |
| 50,000 ASINs | ~2.8 hours | ~1.4 hours | ~4 hours |
Rate Limit Strategy for Multi-Tenant
- Implement per-tenant token bucket rate limiters in Redis
- Mirror SP-API’s limits exactly (refill rates, burst capacity)
- Queue requests that exceed limits — don’t drop them
- Use exponential backoff with jitter for 429 (throttled) responses
- Batch wherever possible: CompetitivePricing, ItemOffersBatch, FeesEstimates all accept 20 ASINs
- Cache catalog data aggressively (product names/categories change slowly)
SP-API Developer Fees (2026)
Annual Subscription
| Fee | Amount | Effective Date |
|---|---|---|
| Annual subscription | $1,400/year | January 31, 2026 (ALREADY IN EFFECT) |
Monthly Usage Tiers (GET Calls Only)
| Tier | Monthly GET Calls | Monthly Fee | Effective Date |
|---|---|---|---|
| Basic | Up to 2,500,000 | Free | April 30, 2026 |
| Pro | Up to 25,000,000 | $1,000/month | April 30, 2026 |
| Plus | Up to 250,000,000 | $10,000/month | April 30, 2026 |
| Enterprise | Custom | Custom pricing | April 30, 2026 |
| Overage | Beyond tier limit | $0.40 per 1,000 GET calls | April 30, 2026 |
What Counts as a GET Call?
- Only GET HTTP methods are metered
- POST, PUT, PATCH are NOT metered (free)
- This means fee estimates (
POST getMyFeesEstimates) and token exchanges (POST) don’t count
Cost Estimate by Scale
| Tenants | Scans/Month | Est. GET Calls | Tier | Monthly API Cost |
|---|---|---|---|---|
| 1–10 | 20–100 | ~200K–1M | Basic | $0 |
| 10–50 | 100–500 | ~1M–2.5M | Basic | $0 |
| 50–100 | 500–1,000 | ~2.5M–5M | Pro | $1,000 |
| 100–500 | 1,000–5,000 | ~5M–25M | Pro | $1,000 |
Important Notes
- Private developers are exempt from these fees (apps only for your own business). Since we’re going public (multi-tenant SaaS), the fees apply.
- Amazon does NOT take revenue share. The $1,400/yr + usage fees are your only costs to Amazon. This is fundamentally different from Apple/Google app stores (which take 15–30%).
Multi-Tenant PostgreSQL Schema
Last updated: 2026-03-02
Migration Strategy: SQLite → PostgreSQL
We’re migrating from SQLite (single file on Railway volume) to PostgreSQL. Options:
- Railway’s PostgreSQL add-on — Simple, same platform as our app
- Supabase Postgres — Free tier includes RLS, built-in auth integration
Schema Pattern: Pool Model
We’re using the pool model: a single shared database where every table has a tenant_id column. This is the simplest, cheapest, and recommended approach for our scale (tens to low hundreds of tenants).
Other patterns (silo model = separate DB per tenant, bridge model = separate schema per tenant) are overkill for our scale and add operational complexity.
New Tables
shared_catalog
| Column | Type | Description |
|---|---|---|
asin | TEXT PK | Amazon product identifier |
monthly_sold | INTEGER | Amazon "bought in past month" badge value |
estimated_sales | INTEGER | Computed sales estimate after stale detection |
sales_source | TEXT | "Badge", "BSR Drops", or "BSR Formula" |
sales_confidence | TEXT | "high", "medium", "low" |
enriched_at | TIMESTAMP | When Keepa data was last fetched |
enrichment_count | INTEGER | Number of times enriched |
tenants
CREATE TABLE tenants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES auth.users(id),
email TEXT NOT NULL,
display_name TEXT,
-- Stripe
stripe_customer_id TEXT,
stripe_subscription_id TEXT,
plan TEXT NOT NULL DEFAULT 'scout',
-- scout / pro / enterprise / admin
plan_status TEXT NOT NULL DEFAULT 'active',
-- active / past_due / canceled / trialing
-- Amazon SP-API (encrypted)
sp_api_refresh_token TEXT, -- AES-256 encrypted
seller_id TEXT,
marketplace_id TEXT DEFAULT 'ATVPDKIKX0DER',
amazon_connected_at TIMESTAMP,
amazon_token_expires_at TIMESTAMP,
-- refresh_token expiry (365 days from auth)
-- Limits
scan_limit_monthly INTEGER DEFAULT 2,
asin_limit_per_scan INTEGER DEFAULT 5000,
-- Timestamps
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
usage_tracking
CREATE TABLE usage_tracking (
id SERIAL PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
month TEXT NOT NULL, -- '2026-03'
scans_started INTEGER DEFAULT 0,
asins_scanned INTEGER DEFAULT 0,
api_calls_made INTEGER DEFAULT 0,
keepa_tokens_used INTEGER DEFAULT 0,
UNIQUE(tenant_id, month)
);
Modified Existing Tables
Every existing table gets tenant_id UUID NOT NULL REFERENCES tenants(id):
-- checked_asins: add tenant_id, change PK ALTER TABLE checked_asins ADD COLUMN tenant_id UUID NOT NULL; ALTER TABLE checked_asins DROP CONSTRAINT checked_asins_pkey; ALTER TABLE checked_asins ADD PRIMARY KEY (tenant_id, asin); -- Same pattern for all other tables: -- scan_categories: add tenant_id -- scan_asins: add tenant_id -- scan_sessions: add tenant_id -- scan_meta: add tenant_id
The primary key for checked_asins changes from (asin) to (tenant_id, asin) because different sellers can have different eligibility results for the same ASIN.
Row-Level Security (RLS)
If using Supabase Postgres, you can enable RLS so that queries automatically filter by the current tenant — an extra safety net beyond application-level filtering:
ALTER TABLE checked_asins ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON checked_asins
USING (tenant_id = current_setting('app.current_tenant')::uuid);
In your Flask app, set the tenant context at the start of each request:
@app.before_request
def set_tenant_context():
tenant_id = get_tenant_from_jwt(request)
db.execute("SET app.current_tenant = %s", [str(tenant_id)])
Current SQLite Schema (For Reference)
The full current database schema with all columns and types is documented on the Database Schema page under the Technical section. When migrating, every table gets tenant_id added and composite primary keys where applicable.
Background Job System (Celery + Redis)
Last updated: 2026-03-02
Why Do We Need a Job Queue?
Currently, scans run as a Python subprocess on the same machine as the web server. This works for a single user, but with multiple tenants scanning simultaneously, we need:
- Concurrent scan execution — multiple scans at once without blocking the web server
- Per-tenant isolation — one user’s scan can’t starve another’s
- Retries and error handling — failed API calls retry automatically
- Monitoring — track progress, cancel scans, report status
What Is Celery?
Celery is a Python distributed task queue. It takes jobs from your web application and runs them in separate worker processes. Redis acts as the “message broker” — the queue that connects your Flask API to Celery workers.
Architecture
Flask API
|
| celery_app.send_task('run_scan', args=[tenant_id, config])
v
Redis Queue (broker)
|
| Worker picks up task
v
Celery Worker Process
|
| Uses tenant's SP-API credentials
v
SP-API (Amazon) + PostgreSQL (results)
Implementation Example
# tasks.py
from celery import Celery
celery_app = Celery('scanner', broker='redis://localhost:6379/0')
@celery_app.task(bind=True, max_retries=3)
def run_scan(self, tenant_id, scan_config):
# Run a full scan for a specific tenant.
tenant = get_tenant(tenant_id)
sp_client = create_sp_client(
tenant.sp_api_refresh_token,
tenant.seller_id
)
# Phase 1: Tree (skip if already downloaded)
# Phase 2: Discovery (searchCatalogItems)
# Phase 3: Eligibility (Wave 1 + Wave 2)
# All results written to PostgreSQL with tenant_id
Railway Setup
# Procfile (updated for multi-service) web: gunicorn dashboard:app --bind 0.0.0.0:$PORT --workers 4 --timeout 120 worker: celery -A tasks worker --loglevel=info --concurrency=4
Railway runs the web and worker as separate services (each with their own container). Redis is also a separate Railway service. All three share the same internal network.
Why Celery + Redis (Not Alternatives)?
| Option | Pros | Cons |
|---|---|---|
| Celery + Redis (chosen) | Python ecosystem standard, mature, well-documented, Railway-compatible | Adds operational complexity |
| RQ (Redis Queue) | Simpler than Celery | Less features, fewer tutorials |
| Subprocess (current) | Simple, no dependencies | No concurrency, no retry, no monitoring |
| AWS Lambda | Serverless, auto-scaling | Different platform, cold starts, 15 min limit |
enrichment.py) instead of Celery+Redis. It runs as a separate Railway worker with ENRICHMENT_MODE=1 and processes the shared_catalog table on a configurable interval. Celery+Redis is planned for future scaling when concurrent per-tenant scan queuing is needed.
Stripe Payment Integration
Last updated: 2026-03-02
What Is Stripe?
Stripe is a payment processing platform that handles subscription billing, credit card processing, invoicing, and tax compliance. It’s the industry standard for SaaS products.
Components Implemented
| Component | Purpose |
|---|---|
| Stripe Checkout | Hosted payment page for initial subscription. User picks a plan, enters card, Stripe handles everything. |
| Stripe Customer Portal | Self-service page where users manage billing: change plan, update card, download invoices, cancel. |
| Stripe Webhooks | Backend events that notify us when things happen (payment succeeded, failed, subscription canceled, etc.). |
| Stripe Products/Prices | Plan tiers (Scout, Pro, Enterprise) and their prices configured in the Stripe dashboard. |
User Flow (End to End)
1. User signs up (Supabase Auth — email/password or Google OAuth) 2. Redirect to Stripe Checkout (plan selection page) 3. User enters credit card and pays 4. Stripe fires checkout.session.completed webhook 5. Backend creates tenant record (plan='starter', status='active') 6. User sees "Connect Amazon Account" button 7. Amazon OAuth flow → refresh_token stored 8. User can now scan
Webhook Events Handled
# Endpoint: POST /api/webhooks/stripe checkout.session.completed → User completed payment → Activate tenant, set plan, update plan_status='active' invoice.paid → Recurring payment succeeded → Extend subscription, reset monthly usage counters invoice.payment_failed → Payment failed → Set plan_status='past_due' → Grace period (7 days), then suspend scans customer.subscription.updated → Plan change (upgrade/downgrade) → Update plan and limits in tenants table customer.subscription.deleted → Subscription canceled → Set plan_status='canceled' → Revoke scan access, keep data for 30 days
Webhook Security
Every webhook request from Stripe includes a Stripe-Signature header. You must verify this signature using your STRIPE_WEBHOOK_SECRET to prevent spoofed webhook events.
event = stripe.Webhook.construct_event(
payload=request.data,
sig_header=request.headers['Stripe-Signature'],
secret=STRIPE_WEBHOOK_SECRET
)
Environment Variables
| Variable | Purpose |
|---|---|
STRIPE_SECRET_KEY | Backend API key (sk_live_...) |
STRIPE_PUBLISHABLE_KEY | Frontend key (pk_live_...) |
STRIPE_WEBHOOK_SECRET | Webhook signature verification (whsec_...) |
STRIPE_SCOUT_PRICE_ID | Stripe Price ID for Scout tier |
STRIPE_PRO_PRICE_ID | Stripe Price ID for Pro tier |
STRIPE_ENTERPRISE_PRICE_ID | Stripe Price ID for Enterprise tier |
Authentication & User Management
Last updated: 2026-03-02
Current State
We already use Supabase Auth with JWT verification. It’s optional (controlled by ALLOWED_EMAILS env var). For multi-tenant, auth becomes required and we add tenant provisioning on top.
Full Auth Flow
1. Supabase Auth handles signup, login, password reset, MFA 2. On signup, Supabase creates a user in auth.users 3. After Stripe payment, our backend creates a row in tenants table linked to auth.users.id 4. JWT from Supabase includes user_id claim 5. Backend maps user_id → tenant_id on every request 6. All database queries scoped to tenant_id
JWT Claims (Extended)
{
"sub": "user-uuid",
"email": "seller@example.com",
"aud": "authenticated",
"role": "authenticated",
"app_metadata": {
"tenant_id": "tenant-uuid",
"plan": "pro"
}
}
Request Middleware
@app.before_request
def require_auth():
token = request.headers.get('Authorization', '')
token = token.replace('Bearer ', '')
payload = jwt.decode(
token,
SUPABASE_JWT_SECRET,
algorithms=['HS256'],
audience='authenticated'
)
g.user_id = payload['sub']
g.tenant_id = get_tenant_id_for_user(payload['sub'])
# Enforce plan limits
tenant = get_tenant(g.tenant_id)
if tenant.plan_status != 'active':
abort(402, 'Subscription required')
What Changes From Current Auth
| Aspect | Current | Multi-Tenant |
|---|---|---|
| Auth required? | Optional | Required for all API endpoints |
| User → tenant mapping | N/A (single tenant) | user_id → tenant_id lookup on every request |
| Plan enforcement | N/A | Check plan_status before allowing scans |
| Signup flow | Email allowlist | Open signup → Stripe payment → tenant creation |
Compliance & Security Requirements
Last updated: 2026-03-02
Amazon Data Protection Policy (DPP)
As a public SP-API developer, Amazon requires you to meet ongoing security obligations. Non-compliance can result in revocation of your API access.
| Requirement | Frequency | Details |
|---|---|---|
| Vulnerability scanning | Every 180 days | Automated scanning of your app and infrastructure |
| Penetration testing | Every 365 days | Must be done by an entity DIFFERENT from whoever built the app |
| Code scanning | Before each release | Static analysis for security issues |
| Encryption at rest | Always | Refresh tokens must be encrypted (AES-256) |
| Encryption in transit | Always | HTTPS everywhere |
| Incident response plan | Always | Document what happens if tokens are leaked |
Amazon Acceptable Use Policy (AUP)
You MUST NOT:
- Aggregate data across sellers to sell to third parties
- Scrape Amazon outside of the SP-API
- Facilitate violations of Amazon’s Business Solutions Agreement
- Store data longer than necessary
- Share one seller’s data with another seller
You MUST:
- Only collect data you need for your stated purpose
- Delete seller data if they disconnect or request deletion
- Maintain audit logs of data access
- Respond to Amazon audit requests
Penetration Testing Notes
Our Security Checklist
- ☐ Encrypt refresh tokens at rest (AES-256)
- ☐ HTTPS on all endpoints (Railway provides this)
- ☐ Webhook signature verification (Stripe + Amazon)
- ☐ CSRF protection on OAuth flow (state parameter)
- ☐ Rate limiting per tenant (Redis token buckets)
- ☐ Audit logging for data access
- ☐ Incident response plan document
- ☐ First vulnerability scan (within 180 days of launch)
- ☐ First pen test (within 365 days of launch)
- ☐ Data deletion on tenant disconnect
Competitor Analysis
Last updated: 2026-03-02
Why This Matters
We researched 6+ competitors to validate that our planned architecture is correct. The result: every single competitor uses the exact same pattern — public SP-API app + OAuth per-seller + Appstore listing. This confirms our approach.
Direct Competitors (Product Sourcing/Scanning)
| Tool | Type | SP-API? | Appstore? | Pricing | Notes |
|---|---|---|---|---|---|
| SellerAmp | Product analysis, eligibility | Yes | Yes | ~$14–28/mo | Shows “No” eligibility without connection |
| ScanUnlimited | Wholesale list scanning | Yes | Yes (Carbon6) | ~$40–80/mo | Upload spreadsheets, bulk analysis |
| Tactical Arbitrage | Retail/online arbitrage | Yes | Yes (Threecolts) | ~$50–100/mo | Scans 1,500+ retail sites |
| BuyBotPro | Product analysis Chrome ext | Yes | Yes | ~$30–50/mo | Chrome extension focused |
| Seller Assistant | Product analysis | Yes | Yes | ~$16–50/mo | Integrates Keepa + SP-API |
Adjacent Competitors (Broader FBA Tools)
| Tool | Type | Pricing | Notes |
|---|---|---|---|
| Jungle Scout | Full suite (research + tracking) | ~$49–129/mo | Massive dataset, own API |
| Helium 10 | Full suite (research + optimization) | ~$39–249/mo | Owned by Pacvue |
| BoxEm | Shipment/inventory/analytics | $23–70/mo | US only, 14-day trial |
Key Validation Findings
- All use public SP-API app + OAuth — No competitor uses a different integration method
- Eligibility requires per-seller credentials —
getListingsRestrictionsneedssellerIdmatching the OAuth token - Only 2 non-restricted roles needed — Product Listing + Pricing (no PII involved)
- Rate limits scale linearly — Per app+seller pair, so more tenants = more capacity
- Amazon takes NO revenue share — Only developer fees ($1,400/yr + usage)
- PostgreSQL pool model is industry standard — Shared DB with tenant_id column
Our Differentiation
Pricing Model & Revenue
Last updated: 2026-03-02
Recommended Tiers
| Tier | Price | Includes |
|---|---|---|
| Scout | $49/mo | 10,000 scans/mo, Keepa 100 tokens/day (~1,000 enrichments/mo), score-gated auto-enrichment |
| Pro | $99/mo | 50,000 scans/mo, Keepa 500 tokens/day (~5,000 enrichments/mo), all eligible enriched |
| Enterprise | $179/mo | Unlimited scans, Keepa 1,500 tokens/day (~15,000 enrichments/mo), priority queue |
What’s Included in All Tiers
- Green Light scoring (7-point system with Keepa bonuses)
- Eligibility checking (per-seller, real-time via their own Amazon account)
- Competitive pricing analysis
- Fee estimation (referral + FBA fees)
- Category tree scanning
- Keepa enrichment (if available)
- Opportunity ranking and filtering
Pro/Enterprise Extras
- Pro+: Keepa integration, category insights, full scan history
- Enterprise: Priority scan queue, API access for custom integrations, team members / sub-accounts
Revenue Projections
| Scenario | Tenants | Avg Revenue/Tenant | Monthly Revenue | Annual Revenue |
|---|---|---|---|---|
| Early | 25 | $40 | $1,000 | $12,000 |
| Growth | 100 | $55 | $5,500 | $66,000 |
| Scale | 500 | $65 | $32,500 | $390,000 |
Cost Structure
| Cost | Monthly | Annual |
|---|---|---|
| Railway (web + worker + Redis + Postgres) | ~$20–50 | ~$240–600 |
| Amazon SP-API developer fee | — | $1,400 |
| Amazon SP-API usage (Basic tier) | $0 | $0 |
| Keepa API | ~$55 (€49) | ~$660 |
| Stripe fees (2.9% + $0.30/txn) | Variable | ~3% of revenue |
| Pen testing | — | ~$2,000–5,000 |
| Total fixed costs | ~$200 | ~$5,000–8,000 |
Break-Even Analysis
With ~$200/month in fixed costs, break-even is at ~5–7 paying customers on the Starter plan, or ~3 customers on Pro.
Implementation Phases
Last updated: 2026-03-02
Phase 1: Foundation (2–4 weeks) ✅ Complete
| # | Task | Status |
|---|---|---|
| 1 | Register as public SP-API developer on Solution Provider Portal | ☐ |
| 2 | Set up PostgreSQL on Railway (or Supabase Postgres) | ☐ |
| 3 | Migrate SQLite schema → PostgreSQL with tenant_id on all tables | ☐ |
| 4 | Create tenants and usage_tracking tables | ☐ |
| 5 | Set up Stripe: create products, prices, checkout session | ☐ |
| 6 | Build signup → payment → dashboard flow | ☐ |
| 7 | Add tenant_id enforcement to all API queries | ☐ |
Phase 2: Multi-Tenant Scanning (3–5 weeks) ✅ Complete
| # | Task | Status |
|---|---|---|
| 1 | Implement Amazon OAuth flow (LWA authorization + token exchange) | ☐ |
| 2 | Build “Connect Amazon Account” UI flow | ☐ |
| 3 | Store per-tenant SP-API refresh tokens (encrypted AES-256) | ☐ |
| 4 | Modify finder.py to accept tenant context and use tenant’s credentials | ☐ |
| 5 | Set up Celery + Redis for background scan jobs | ☐ |
| 6 | Implement per-tenant scan queuing and rate limiting | ☐ |
| 7 | Add scan progress WebSocket/SSE for real-time updates | ☐ |
Phase 3: Polish & Launch (2–3 weeks) 🔄 In Progress
| # | Task | Status |
|---|---|---|
| 1 | Stripe webhook handling (payment failures, upgrades, cancellations) | ☐ |
| 2 | Usage metering and plan enforcement (scan limits, ASIN limits) | ☐ |
| 3 | Tenant settings page (billing, Amazon connection status, usage) | ☐ |
| 4 | Onboarding tour for new users | ☐ |
| 5 | Landing page with pricing table | ☐ |
| 6 | Submit app to Selling Partner Appstore for review | ☐ |
| 7 | Vulnerability scan + pen test (for Amazon DPP compliance) | ☐ |
Phase 4: Growth (Post-Launch)
| # | Task | Status |
|---|---|---|
| 1 | Chrome extension for on-page product analysis | ☐ |
| 2 | Additional marketplaces (UK, CA, EU) | ☐ |
| 3 | Team/sub-account support | ☐ |
| 4 | API access for Enterprise tier | ☐ |
| 5 | Separate React frontend (if needed) | ☐ |
Key Technical Decisions
Decided
| Decision | Choice | Rationale |
|---|---|---|
| Database | PostgreSQL | Multi-tenant RLS, scalability, industry standard |
| App type | Public SP-API app on Appstore | Required for multi-seller OAuth, unlimited connections |
| Payments | Stripe | Industry standard SaaS billing |
| Job queue | Celery + Redis | Python ecosystem standard, mature, Railway-compatible |
| Auth | Supabase Auth | Already in use, handles full user lifecycle |
To Decide
| Decision | Options | Notes |
|---|---|---|
| PostgreSQL host | Railway Postgres vs Supabase Postgres | Supabase has free tier + built-in RLS |
| Frontend | Keep embedded HTML (MVP) vs React | Embedded is faster to ship |
| Keepa cost | Absorb in plan price vs separate add-on | Most competitors include data enrichment |
| SP-API app count | Single app vs multiple registrations | Single is fine until rate limits become an issue |
Glossary
Last updated: 2026-03-02
Terms and definitions used throughout the SaaS scaling documentation. If you encounter an unfamiliar term, check here first.
| Term | Definition |
|---|---|
| ASIN | Amazon Standard Identification Number — a unique 10-character alphanumeric product ID assigned by Amazon to every product in its catalog. |
| BSR | Best Seller Rank — a number representing how well a product sells compared to others in its category. Lower = better. Updated hourly by Amazon. |
| Buy Box | The “Add to Cart” button area on a product page. Multiple sellers can list the same product, but only one “wins” the Buy Box at any time. Winning the Buy Box means your offer is the default purchase option. |
| Celery | A Python distributed task queue library for processing background jobs asynchronously. Used to run scans in separate worker processes. |
| DPP | Data Protection Policy — Amazon’s security requirements for SP-API developers, including encryption, vulnerability scanning, and penetration testing. |
| FBA | Fulfillment by Amazon — a service where sellers ship inventory to Amazon warehouses. Amazon handles storage, packing, shipping, and customer service. |
| FBM | Fulfillment by Merchant — the seller handles their own storage, packing, and shipping directly to the customer. |
| Green Light Score | Our proprietary weighted 0–10 scoring system that evaluates product opportunity quality based on profit margin, sales velocity, competition (FBA sellers), price point, and Keepa trends. |
| IAM | Identity and Access Management — an AWS service for managing API credentials and permissions. SP-API requires an IAM Role for authentication. |
| LWA | Login with Amazon — Amazon’s implementation of OAuth 2.0. Used to let sellers authorize our app to access their SP-API data. |
| MWS | Marketplace Web Service — Amazon’s old API for seller tools. Fully sunset in March 2024. Replaced by SP-API. |
| OAuth | Open Authorization — a protocol that lets a user (Amazon seller) grant our app permission to access their data without sharing their password. |
| Pool Model | A multi-tenant database pattern where all tenants share one database and tables, with a tenant_id column on every row for isolation. Simplest and cheapest approach. |
| Redis | An in-memory data store used as a message broker (Celery queue), cache (access tokens), and rate limiter (token buckets). |
| RLS | Row-Level Security — a PostgreSQL feature that restricts which rows a query can see based on the current session context. Provides an extra layer of tenant isolation beyond application code. |
| SP-API | Selling Partner API — Amazon’s current REST API for third-party seller tools. Replaced MWS in 2024. |
| Stripe | A payment processing platform that handles subscription billing, credit card processing, invoicing, and tax compliance for SaaS products. |
| Tenant | A single customer (Amazon seller) in our multi-tenant system. Each tenant has their own data, credentials, and scan results, isolated from other tenants. |
| WAL | Write-Ahead Logging — a SQLite journaling mode that allows concurrent reads while writing. Used in our current single-tenant setup. |
| Webhook | An HTTP callback — when an event happens (payment succeeded, subscription canceled), the external service (Stripe/Amazon) sends a POST request to our server to notify us. |
Cost Architecture
The scanner uses two external APIs with usage-based costs. Our optimization strategy reduces total API calls by ~80% without sacrificing data quality.
SP-API Call Reduction
| Strategy | How It Works | Savings |
|---|---|---|
| Eligibility-First Pipeline | Check eligibility BEFORE pricing/catalog. 92% of ASINs are rejected at this gate (1 call) instead of running the full 5-call pipeline. | 75% fewer calls |
| Shared Eligibility Cache | Non-eligible results cached globally with 7-day TTL. Subsequent scans skip SP-API entirely for known-rejected ASINs. | 80% on re-scans |
| Early Price Rejection | Products under $12 or with 15+ sellers skip Wave 2 (fees + offers), saving 2 API calls per product. | ~20% of eligible |
Keepa Token Optimization
| Strategy | How It Works | Savings |
|---|---|---|
| Global Keepa Cache | 289K+ ASINs cached globally. When any user scans an ASIN already enriched, data served from cache (0 tokens). | 60-80% at scale |
| Smart Budget Management | When daily budget < 50%, daemon skips Red products and focuses tokens on Green + Yellow products only. | 70% token savings |
| Tiered Freshness TTL | BSR <10K = 2-day refresh, BSR <50K = 5-day, BSR >50K = 10-day. Fast movers get fresher data. | 50% daemon savings |
| Pre-Score Threshold | Only products scoring ≥3.5 pre-Keepa get enriched. Low-quality products skip enrichment entirely. | ~30% fewer enrichments |
The Shared Catalog architecture further reduces costs by 90%: one Keepa plan (~$100/mo) serves all tenants via the shared_catalog table, vs $500+/mo for per-tenant enrichment.
Per-Product API Cost Breakdown
Critical Fixes
| Bug | Root Cause | Fix |
|---|---|---|
| Scan crashes after 5 ASINs | UnboundLocalError on _keepa_batch_buffer — reassignment inside function made Python treat global as local | Added global _keepa_batch_buffer declaration |
| PostgreSQL transaction abort | init_db() DDL failures left connection in InFailedSqlTransaction state, poisoning all subsequent queries | Wrapped each DDL in try/except with explicit commit/rollback |
| Brands 500 error | SELECT * loading 135K+ rows into memory caused timeout. Also, pagination code mutated cached objects via pop() | Added is_eligible=1 filter (11K vs 135K rows) + shallow copy instead of mutation |
| 502 server overload | Frontend polling every 1 second with 2 gunicorn workers. 600+ requests queued. | Reduced polling to 5 seconds, added analysis cache (30s TTL) |
| 12.3M duplicate ASINs | Clicking Top Sellers multiple times stacked ASINs in scan_asins without dedup. No guard against concurrent starts. | Added is_scan_running() check + dedup on insert + cleanup endpoint |
| Silent Keepa failures | 50+ except: pass blocks hiding real errors. Production debugging impossible. | Replaced with except Exception as e: print() for visibility |
| Amazon grade incorrect | Products with Amazon on Buy Box scored C/Hold instead of F/Avoid | Hardcoded grade override: Amazon BB = F grade + AVOID badge |
| 0/10 scores on valid products | Score computation used fee_margin from DB (often NULL) instead of calculating from price/fees | Compute margin inline if DB value is NULL |
| NULL sort crashes | Sorting by score put NULL-score products randomly in results | Added NULLS LAST to all sort columns |
| History card shows zeros | Running scan session queried only finalized data, not live counts | Added live query fallback for active sessions |
| Keepa price mismatch | Badge data diverging >3x from sale price displayed misleading prices | Hidden with warning when divergence exceeds 3x |
| Stale badge CHECK 3 | Drops-vs-badge ratio >5x not detected, inflating sales estimates | Added cross-validation check; capped at 3x BSR estimate |
| N+1 query | shared_catalog per-product queries caused slow dashboard loads | Replaced with batch preload on analysis endpoint |
| Font fallback | 17+ buttons using Arial instead of system font | Global font-family: inherit fix for all buttons/inputs |
| Watchlist table creation | PostgreSQL CREATE TABLE IF NOT EXISTS not running on startup | Added watchlist table to init_db() migration |
UI/UX Fixes
| Bug | Fix |
|---|---|
HTML tags rendered in Quality Distribution (<span> visible) | Fixed template escaping |
| Product Type showing underscores (PRECISION_MEASURING) | Added title case + space conversion |
| Yield chart x-axis showing "Mar 5, Mar 5, Mar 5..." repeating | Deduplicated labels, show only on date change |
| Mobile horizontal scroll on all pages | Added overflow-x:hidden, responsive breakpoints |
| Top Opportunities not sorted correctly | Fixed opportunity_score sort to use DESC |
| Alert/confirm dialogs blocking UI | Replaced with non-blocking showToast() notifications |
| Docs page not mobile-friendly | Added responsive sidebar, stacked layout |
Paste a Discord webhook URL to get alerts when new Green-light products hit your dashboard (score ≥ 7.0, 100+ monthly sales, 2+ sellers, Amazon not in buy box). How do I create a webhook?
All brands detected in your scan data — ungated brands are highlighted.