How do I validate product feed URLs before ingesting them?

The problem

A marketplace ingests product feeds from thousands of sellers. Each product listing contains URLs: product pages, seller storefronts, image sources, and documentation links. Dead product URLs create broken listings that frustrate buyers and erode trust in your platform.

Parked seller domains mean the seller has likely closed, but their products are still showing up in search results. Brand impersonation in seller sites signals potential counterfeit products. Without validation, your marketplace becomes a graveyard of broken links and a vector for fraud.

How Unphurl solves it

Validate product feed URLs as they arrive, before they enter your catalogue. The product feed scoring profile weights signals that indicate dead listings, defunct sellers, and potential counterfeiting. AI agents processing feeds can automatically quarantine flagged products for human review instead of publishing them directly.

Signals that matter for this use case

• – Parked means the seller's domain is a placeholder. The seller has likely closed or abandoned their storefront.
• – Chain incomplete means the product URL doesn't resolve. Dead listing.
• – Redirects 5+ flags product URLs that bounce through excessive redirects, often indicating moved or discontinued products.
• – Brand impersonation detects seller domains that mimic well-known brand names. Potential counterfeit signal.
• – Expiring soon flags seller domains about to lapse. Products from these sellers may become orphaned.
• – No MX record means the seller can't receive email. Harder to resolve disputes or communicate about orders.
• – Domain status bad (pendingDelete, serverHold) confirms the seller's domain is effectively dead.
• – SSL invalid means the seller's site has a broken certificate. Trust concern for buyers.
• – HTTP only signals a seller site without basic encryption. Not a secure shopping experience.

Suggested scoring profile

{
  "name": "product-feed",
  "weights": {
    "parked": 25,
    "chain_incomplete": 25,
    "redirects_5": 20,
    "brand_impersonation": 20,
    "expiring_soon": 15,
    "no_mx_record": 10,
    "domain_status_bad": 15,
    "ssl_invalid": 10,
    "http_only": 10
  }
}

What a result looks like

Your platform ingests 5,000 product URLs from 200 sellers. Unphurl validates each one during ingestion:

The 4,600 clean URLs publish immediately. The 400 flagged URLs are quarantined for review. Your marketplace stays clean, buyers see working product pages, and counterfeit concerns get caught before they reach customers.

Cost

Daily feed ingestion is the typical pattern. Most major e-commerce domains are cached after the first validation pass, so repeat ingestion costs drop significantly. The Pro package handles ongoing validation for marketplaces processing thousands of product URLs per day.

Get started

# Validate a product feed URL
curl -X POST https://api.unphurl.com/v1/analyse \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://seller-store.com/product/widget-123",
    "profile": "product-feed"
  }'