# AI Outbound Calling and Lead Qualification System

> A system I built that calls a list of consented contacts with an AI voice agent, holds a short qualifying conversation and records every result as a tracked outcome or lead. A NestJS API runs the dialing, compliance checks and webhooks, and a Next.js dashboard with a built-in AI assistant lets the owner run campaigns and ask for reports in plain language.

- Author: Sami Ullah (https://samiullah.tech/about)
- Type: Own product
- Role: Full build
- Stack: NestJS, TypeScript, Prisma, SQLite, Next.js, Tailwind CSS, ElevenLabs, Twilio, Gemini, Docker
- Year: n/a
- Updated: 2026-11-13
- Canonical: https://samiullah.tech/work/ai-calling-agent

## Overview

This system takes a list of phone numbers from people who agreed to be contacted, calls them with an AI voice agent, has a short qualifying conversation and stores the result. Every number ends with a clear outcome: qualified lead, interested, callback requested, not interested, no answer, voicemail, opted out and so on.

The owner works from a web dashboard: upload a list, create a campaign, check the cost estimate, start calling, and watch results and leads come in. An AI assistant inside the dashboard answers questions such as "give me today's report" or "who asked for a callback" in English, Urdu or Hindi.

## My role

I did the full build: the API, the dialing scheduler, compliance rules, webhook handling, the dashboard, the AI assistant, tests and the Docker deployment.

## The problem

Calling a list of leads by hand is slow and expensive, and most calls end with no answer. AI voice agents can hold a natural conversation now, but a voice agent on its own is not a calling system. Someone still has to decide who to call and when, respect calling hours and opt-outs, retry sensibly, stay within budget and turn each conversation into a record a salesperson can act on.

## Approach & architecture

The voice itself is handled by ElevenLabs agents connected to a Twilio number. Everything around the calls is mine: a [NestJS](/skills/nestjs) API owns contacts, campaigns, compliance, scheduling, webhooks, leads and reports, and a [Next.js](/skills/nextjs) dashboard sits in front of it.

```flow
Dashboard (Next.js) -> server-side proxy -> NestJS API -> database
NestJS scheduler -> compliance checks -> ElevenLabs batch calls -> Twilio -> contact's phone
ElevenLabs -> signed post-call webhook -> NestJS API -> outcome, lead, transcript
AI assistant page -> NestJS assistant -> Gemini model -> read-only and safe tools
```

A scheduler runs every few minutes, picks contacts that may be called right now, and submits them in batches. When a call ends, a signed webhook delivers the outcome and transcript, and the API records it. If a webhook never arrives, the API falls back to polling so no number is left without a result.

The dashboard browser never sees the API key. It talks only to Next.js, which checks the session and adds credentials on the server.

## Key challenges

**Compliance first.** Only consented numbers are imported, a do-not-call list blocks numbers permanently, and opt-outs during a call are added to it automatically. Calls are placed only inside allowed local hours. For countries with several time zones and no known zone for the contact, the system only calls when every zone is inside the window, and numbers with an unknown country are skipped.

**Retries and budgets.** Unanswered calls are retried a limited number of times with a gap between attempts. Each campaign can have a minute budget, a daily call limit and a lead target. The system reserves estimated minutes for calls in flight and pauses a campaign when it reaches any limit.

**An assistant that cannot dial by itself.** The first version exposed the system to a desktop AI assistant through an MCP server. I later replaced that with an AI assistant built into the dashboard, powered by a Gemini model. It can list campaigns, read reports, change limits and pause campaigns, but it can never start or resume dialing. When asked to start, it only shows the cost summary with a confirm button, and the owner's own click starts the campaign.

**Testing without real calls.** A dry-run mode sends contacts through the real outcome handling with simulated results, and the test suite uses a mocked voice provider and a scripted fake model, so no network or keys are needed.

## Tech stack

- **NestJS and TypeScript** for the API and scheduler, with **Prisma** on SQLite (portable to PostgreSQL).
- **Next.js and Tailwind CSS** for the dashboard, with dependency-free charts.
- **ElevenLabs Agents and Twilio** for voice and telephony.
- **Gemini** through an OpenAI-compatible client for the dashboard assistant.
- **Jest and Supertest** for unit and end-to-end tests.
- **Docker Compose with Caddy** for HTTPS deployment on a VPS.

## Results

All planned milestones are implemented: list upload with an import report, campaigns with cost preview, compliant scheduling, retries, webhooks with polling fallback, leads and per-number results, reports, CSV export, the do-not-call list, an audit log, data retention, backups and the AI assistant. It runs locally in dry-run mode and is ready for VPS deployment.

## Screenshots

Screenshots of the dashboard will be added soon.