ProductCompat — AI Enrichment & Search Queue

1. How it works

Each product that enters the AI Search Queue goes through a five-stage pipeline:

• Sources: unlike the classic pipeline (first source that answers wins), the queue collects results from every configured source — Icecat, SearXNG, Google CSE, SerpApi, official/trusted sites — plus their raw text snippets.
• AI pass: the gathered text is sent to your AI provider (via AiCore) which extracts part numbers, device models, barcodes, corrections for wrong data, a clean description, brand, specs and image URLs.
• Hallucination guards: the AI may only report values that appear verbatim in the sources; barcodes must pass the GTIN check-digit; AI-only findings are capped at 60% confidence (corroborated ones get boosted). Anything that fails validation is silently dropped.
• Review-first: structured data lands as needs-review rows; rich data (description, name, brand, specs, images) lands as pending candidates. Nothing touches the product until you click Apply / Approve.

2. Data sources explained

These are the entries you see in Product Compatibility → Settings. They are data sources the pipeline reads from — each one is optional, and the queue uses every source that is configured:

Product Compatibility → Settings: SearXNG / Google / SerpApi source settings (left) and the new AI enrichment section (right) with the green AiCore status indicator.

3. AI provider & failover chain (AI Core → Settings)

The AI engine is shared by the whole system (WhatsApp auto-reply, VetAI, ProductCompat…). Setup is a two-step flow — configure services once, then just pick them in the chain. The main form holds only behavior (business type, on/off, temperature, budget, persona, prompts) plus a read-only Active primary line — credentials live exclusively in the AI services box. Copies upgraded from older versions get their manually-configured provider auto-converted into a service card, linked as Default.

AI Settings main form after simplification: no credential fields — just behavior settings and the "Active primary" status line (★ Groq).

Step 1 — AI services (configure & test each service once)

• The box starts empty — use Add provider to add only the services you need: Groq, Claude, DeepSeek, z.ai, Bynara, naraya.ai, Custom. Endpoint and suggested model come prefilled; you only paste the API key (for Bynara / naraya.ai also their endpoint URL).
• Each panel has its own Save service, Test Connection and Remove buttons; a green Configured pill shows which services are ready.
• Make default: each configured card has a radio selector — the chosen service becomes the Primary the engine calls first. The first service you configure becomes the default automatically (unless a working manual primary already exists). Unconfigured cards show the selector disabled until saved with a key.
• Editing a service later automatically updates the default and every chain slot that uses it.

AI Core → Settings → AI services: Groq / Claude / DeepSeek panels with prefilled endpoints — configure and connection-test each service once.

Step 2 — Failover chain (selection only)

• The Primary card is read-only — it shows whichever service carries the Default radio (or "manual configuration" if you configured the main form directly).
• Each fallback slot (fallback → fallback2 → fallback3) is just a dropdown of your configured services — no credentials re-entered, one Save chain button.
• If the primary call fails for any reason (outage, rate limit, timeout), the engine automatically retries the chain in order. The user never sees the failure. Persona, voice and prompt settings are never touched by the chain.

Failover chain: pick a configured service per slot — "none / disabled" skips a slot.

4. The AI Search Queue

The queue is the list of products waiting for a full data check. Products enter it four ways:

Product Compatibility → AI Search Queue: add products, backfill, filter by status, retry failed / skip entries.

Running the queue

Three ways — pick what suits you:

1. Control Panel (live progress): Product Compatibility → Control Panel → AI Search Queue → Run queue now. Processes in small chunks with a progress bar; you can stop anytime.
2. Scheduled (recommended): enable it in Settings → AI enrichment. Requires the server cron to call php artisan schedule:run every minute (already the case on your servers).
3. Command line: php artisan productcompat:process-queue --limit=10

Control Panel: the classic pipeline cards plus the new AI Search Queue card with queued/failed counters.

5. Product create / edit hooks

• Creating a product — it is queued automatically and silently (a small note appears under the compatibility block). Disable via Auto-queue new products on create in settings.
• Editing a product — the compatibility block shows an unchecked box: "Re-check this product with the AI search queue after saving", next to a badge with the product's current queue status. Tick it only when you want a re-check (edited products re-enter with higher priority).

Product edit form: the re-check checkbox with the green "AI queue status: Done" badge, rendered inside the Product Compatibility block.

6. Review queue & applying data

Everything the pipeline finds waits for your decision in Product Compatibility → Review Queue. Products with AI findings show an N AI badge:

Review Queue: SKU0001 carries a "3 AI" badge — three AI data suggestions pending.

Opening the product card shows the AI data suggestions section:

Per product: image thumbnails with Download & attach, description Current-vs-Suggested comparison, name/brand suggestions — each with its source and confidence badge.

7. Change report (audit trail)

Every pipeline run and every applied suggestion is recorded in Product Compatibility → Change Report. AI activity is labeled AI check (queue run: what was found) and AI applied (what you approved: before & after values), so any change can be traced back or manually reverted.

Change Report: the AI check row for SKU0001 — snapshot of the old data alongside the run result (3 candidates, top source Icecat).

8. Settings reference (Product Compatibility → Settings → AI enrichment)

9. Console commands & cron

# Process the AI search queue manually
php artisan productcompat:process-queue --limit=10 --sleep=2

# Restrict to one business
php artisan productcompat:process-queue --business=1 --limit=5

# Classic commands (unchanged)
php artisan productcompat:enrich --missing --limit=50
php artisan productcompat:parse-custom-fields --dry-run

The scheduled run requires the standard Laravel cron entry (once per server):

* * * * * php /path/to/saas-one/artisan schedule:run >> /dev/null 2>&1

Deploying to another copy

1. Pull the branch / copy the two module folders (Modules/AiCore, Modules/ProductCompat).
2. Visit /product-compat/update then /ai-core/update as an admin — runs migrations, registers the new permission, bumps versions.
3. Grant the AI search queue permission to the roles that need it, configure AI Settings, done.

10. Permissions (Role management)

11. Troubleshooting