Back to Work

Syncable: Automating Email Capture for SmartSuite

Building a robust sync engine that connects Gmail and SmartSuite to automate lead capture and CRM data entry.

Syncable: Automating Email Capture for SmartSuite
June 2024 • Syncable.io
SaaS / Productivity Integration & Automation

Outcome

Built a resilient sync engine using Postgres Cron and Hookdeck to process thousands of emails with 99.9% reliability and full retry support.

Context

Syncable.io is an “Email Productivity Pack” designed specifically for SmartSuite users. Its core value proposition is simple but powerful: copy important emails into SmartSuite and automatically create corresponding contacts without leaving your inbox.

Problem

The initial prototype faced significant scaling and reliability challenges:

  1. Job Reliability — The original Vercel Cron implementation lacked built-in retries, meaning if a sync failed during a 5-minute window, that data was potentially lost or delayed indefinitely.
  2. Concurrency Management — Processing high volumes of email data required a way to trigger and track individual runs without overloading the serverless environment.
  3. Status Visibility — Users needed clear feedback on whether their syncs were “Connected” or if new emails had been processed.

Approach

I re-architected the sync engine to move away from simple cron jobs toward a more resilient, event-driven system.

1. Reliable Scheduling with Postgres Cron

We swapped Vercel Cron for Postgres Cron running directly on the database. This allowed us to trigger sync calculations every 5 minutes with higher precision and lower overhead.

2. Webhook Orchestration via Hookdeck

Instead of calling the processing endpoint directly, the system now inserts a record into a syncs_runs table. This triggers a Supabase Webhook which is routed through Hookdeck. This setup provides:

  • Automatic Retries: If the worker is down, Hookdeck holds the request and retries until success.
  • Rate Limiting: Protects the downstream API from spikes in traffic.
Syncable Architecture Diagram

The re-architected event-driven sync flow.

3. State-Machine Pattern for Syncs

We implemented a strict status-tracking system (Pending, Running, Success, Failed) within the syncs_runs table, allowing for real-time monitoring and easy debugging of individual customer syncs.

4. CASA Tier 2 Security Compliance

Because Syncable handles sensitive Gmail data, the application underwent a rigorous CASA (Cloud App Security Assessment) Tier 2 security review. This involved:

  • Verified Security Standards: Meeting the requirements for handling restricted Google API scopes.
  • Infrastructure Hardening: Ensuring the re-architected system met industry-standard security practices for data in transit and at rest.
  • Audit Readiness: Implementing logging and monitoring that satisfied the security auditors’ requirements.

Outcome

Syncable is live at syncable.io. The transition to an event-driven architecture and successful security certification transformed Syncable from a fragile prototype into a production-ready SaaS.

  • Zero Data Loss: Hookdeck’s retry logic ensures that no email sync is ever missed due to transient network issues.
  • Enterprise-Grade Security: Passing the CASA Tier 2 review allowed the app to handle restricted Gmail scopes, unlocking full functionality for all users.
  • Scalable Architecture: The system now handles thousands of concurrent sync runs across different user accounts without manual intervention.
  • Better DX: Using Upstash for caching and Postgres for scheduling reduced the complexity of the codebase while increasing performance.

Learnings

This project reinforced that for mission-critical integrations, retries are not optional. Moving logic “closer to the data” (using Postgres Cron) and using specialized middleware (Hookdeck) for webhook delivery is often much simpler and more reliable than trying to build complex retry logic inside a serverless function.

Want to discuss a similar challenge?

Get in Touch