root@mindgraph:~# cat case-studies/multi-source-lead-scraping-scoring-engine.md
Multi-Source Lead Scraping & Scoring Engine
DOCUMENT: Multi-Source Lead Scraping & Scoring Engine
INDUSTRY: Software & SaaS
PROJECT: GTM Intelligence
CLASSIFICATION: CASE STUDY // PUBLIC RELEASE
SUMMARY:
An 11-platform prospecting habit, replaced by one scored pipeline.
DETAIL:
10 scraper sources landing in one deduplicated Postgres table, with 4-key dedup, ICP filtering across 99 titles and 9 domains, and a deterministic + LLM blended scoring engine the client controls.
IMPACT:
SOURCES UNIFIED: 11 platforms → 1 pipeline
07 / SOFTWARE · SAAS / GTM INTELLIGENCE
An 11-platform prospecting habit, replaced by one scored pipeline.
Client: B2B ESG SaaS company (scaling outbound)
Tags: B2B SaaS · Lead Generation · Multi-Source Scraping · LLM Scoring · Deduplication
// Problem
Manual prospecting across 11+ platforms — job boards, events, LinkedIn, news, regulatory filings — with no dedup between them and no shared scoring logic. Reps re-discovered the same prospects under different names, and pipeline throughput was bottlenecked by manual research time, not lead availability.
// What Ships Now
10 scraper sources → 1 table. Luma, LinkedIn Jobs, LinkedIn New Hires, Sales Navigator, ICP News, Competitor News, Blog, LinkedIn Commenters, ESG Regulatory Signals, and Slack-native inbound — all on independent schedules, all landing in one deduplicated Postgres table.
4-key dedup, every lead, every time. LinkedIn slug, email, phone, and name+company+city fingerprinting — checked in-batch and against the full historical database before a single enrichment credit is spent.
99 titles, 9 domains, one filter. Every profile is scored against the client's own ICP title map before it's allowed anywhere near the scoring engine.
Transparent scoring, not a black box. A deterministic point system — seniority, geography, regulatory exposure, repeat-signal bonuses — blended against an LLM score on a slider the client controls.
// Technical Details
- 10 independent scraper sources unified into one Postgres lead table via Celery beat scheduling
- 4-key deduplication fingerprinting (LinkedIn slug, email, phone, name|company|city), in-batch and against full DB history
- ICP relevance filtering against a 99-title map across 9 ESG domains, LLM-evaluated per profile
- Deterministic + LLM blended scoring engine, transparent point breakdown, adjustable via a single slider
- Company compliance-framework resolution (BRSR, CSRD, GRI, TCFD) cached per account
- On-demand Apollo contact enrichment and NeverBounce email validation before handoff to sales
- Slack-native inbound signal capture (website-visitor identification → structured lead)