Skip to content

About

Config files for my GitHub profile.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

47 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 

Repository files navigation

Hi, I'm Guilherme Carvalho 👋

Senior Data & AI Engineer · Databricks · GCP · Azure · Lakehouse, Governance & Agentic AI
📍 Sydney, Australia · 🇧🇷 Brazilian

LinkedIn Email


About me

I'm a Senior Data & AI Engineer with 4+ years of experience designing and building data platforms end to end — from ingestion and modelling to governance, CI/CD and AI-powered applications. I work across Google Cloud, Azure and Databricks, and I'm focused on what I call AI Data Engineering: bringing agentic AI systems into the data engineering lifecycle as a core part of how pipelines are specified, built and operated.

  • 🏗️ Lakehouse & Data Warehouse architecture — Medallion (Bronze → Silver → Gold) on Databricks and BigQuery, terabyte-scale workloads.
  • 🛡️ Data governance & quality — Unity Catalog, Dataplex, Policy Tags, lineage and data quality gates with quarantine of rejected records.
  • 🤖 Agentic AI for data engineering — Claude Code workflows integrated via MCP (Model Context Protocol) with Databricks, Jira and SharePoint; reusable agent skills; LLM agents with ADK, LangChain and LangGraph; RAG.
  • 🔁 DataOps — CI/CD with Azure DevOps and Cloud Build, isolated DEV / HML / PROD deployments, orchestration with Airflow (Cloud Composer).
  • 📈 Machine Learning — Vertex AI and BigQuery ML, feature engineering, training, evaluation and deployment.

I'm currently living in Sydney, improving my English and preparing for a Master by Research, while staying hands-on with data and AI projects. I'm open to Data / AI Engineering opportunities in Sydney (on-site, hybrid or remote).

🏅 Certifications

Certification Issuer
Professional Data Engineer Google Cloud
Professional Machine Learning Engineer Google Cloud
Databricks Certified Data Engineer Associate Databricks
Forward Program McKinsey & Company

🧰 Tech stack

Cloud & platforms

Data engineering

AI & ML

Languages, APIs & DevOps

💼 Experience highlights

Period Role Company Focus
2025 – 2026 Senior Data Engineer — Azure & Databricks NEORIS Terabyte-scale Lakehouse, Delta Live Tables, Unity Catalog, agentic workflow on Claude Code + MCP
2025 – 2026 Senior Data Engineer — Google Cloud NEORIS End-to-end GCP architecture, Dataform, BigQuery, Dataplex, CI/CD with Azure DevOps
2025 Machine Learning Engineer IPNET by Vivo Vertex AI, BigQuery ML, LLM agents (ADK, LangGraph), RAG, FastAPI
2024 – 2025 Data Engineer IPNET by Vivo ETL/ELT on GCP: BigQuery, Dataflow, Composer, Dataproc
2024 Systems Development Analyst Prognum Brazil's largest bank real-estate credit system (SQL, Python, REST/SOAP)
2023 – 2024 Data Engineer Kairós Solutions GCP pipelines, PySpark on Dataproc, Looker Studio dashboards

🌐 Featured projects

Project What it does Stack
News Content Collector & API Async scraper for The Guardian, loads into BigQuery and serves the data through a REST API Python, asyncio/aiohttp, FastAPI, BigQuery, Docker
GCP ETL Pipeline with dbt Validates and loads files from Cloud Storage into BigQuery, then models them with dbt; deployed to Cloud Run Python, Flask, BigQuery, dbt, Cloud Run, Cloud Build
Automated Market Data ETL on GCP Serverless daily ETL of stock prices from Yahoo Finance into BigQuery Cloud Functions, Cloud Scheduler, Cloud Storage, BigQuery, IAM
FIPE API Extractor with Airflow Orchestrated extraction of Brazil's FIPE vehicle price tables Python, Apache Airflow (Astronomer), Docker

Let's connect — I'm always happy to talk about data platforms, Lakehouse architecture and agentic AI. 🚀

About

Config files for my GitHub profile.

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors