Home: Iago Bonifácio

Second Nature, 2024 to present

A GCP data platform from an empty project

I design and operate Second Nature's analytics platform: 4M+ events a month from 1,500+ clients, at 99.9% SLA, with CDC replication from AWS.

Data platform: AWS sources replicated by CDC into BigQuery in bronze, silver and gold layers, orchestrated by Composer, governed by Dataplex and provisioned with Terraform.SourcesAWS RDS, APIsIngestionDatastream, Pub/SubBigQuerybronze, silver, goldBIComposer10+ DAGsDataplexgovernanceTerraformCloud, OIDC
Data platform: AWS sources replicated by CDC into BigQuery in bronze, silver and gold layers, orchestrated by Composer, governed by Dataplex and provisioned with Terraform.

Context

Second Nature (United States) processes rental events from more than 1,500 property management clients. Operational data lived in PostgreSQL and MariaDB on AWS RDS, and analytics needed a single warehouse on GCP.

Problem

Go from an empty GCP project to a platform a small team can run: reliable ingestion from another cloud, governance that meets SOC2 and PCI DSS without slowing analysts down, and BigQuery cost under control.

What I did

  • Medallion architecture (bronze, silver, gold) with BigQuery datasets, Dataflow pipelines, Dataform transformations and Dataplex governance.
  • Datastream CDC from RDS to BigQuery over VPN, with no database exposed.
  • Orchestration in Cloud Composer (Airflow) with 10+ DAGs for ingestion, transformation and analytics.
  • Terraform as the only change path: Terraform Cloud with GitHub OIDC (no long-lived credentials), pre-commit, tfsec and SonarCloud in CI.
  • Moved the NetSuite to Salesforce integration off MuleSoft onto Apigee X with Pub/Sub and BigQuery.
  • FinOps: BigQuery slot commitments, tiered storage, budget alerts and cost dashboards.
  • Naming standard across 50+ GCP resources, and SOC2 and PCI DSS assessments in Drata.

Result

  • 4M+ events a month from 1,500+ clients, at 99.9% SLA.
  • US$140k in annual savings (28% year over year) from FinOps.
  • Infrastructure deployment cycle cut from days to hours.