Jerry

Data Engineer

A data engineer who turns large-scale service logs and financial data into structures people can trust, from ingestion through transformation, quality, serving, and infrastructure.

Across Kakao's recommendation team and CryptoQuant, I have designed and operated the pipelines and data platforms behind large-scale user logs and financial data.

At Kakao, I ran pipelines that integrate and preprocess high-volume user logs in real time across services such as KakaoTalk, Gift, advertising, and Daum News, and I led the adoption of Apache Airflow 2.0 to modernize the team's batch orchestration.

At CryptoQuant, I owned financial data products for institutional clients, including US brokerages. I standardized BigQuery transformation pipelines with dbt, moved heavy computation to Spark to cut latency, and migrated a PostgreSQL-centered API serving layer to Databricks and Delta Lake. Delta UniForm exposes an Iceberg catalog, so customers can query the data directly from Snowflake or BigQuery. I also built and ran the infrastructure under these pipelines, including HA Airflow and the EKS/Karpenter migration.

What I care about at work is not handling requests one by one but building structures so the same kind of request does not come back. Replacing the consumption channel itself with an open table format, instead of adding another API for each customer with a different way of consuming data, came from the same thinking.

On my own, I build and run a quant trading system on crypto derivatives and DeFi, which shows me firsthand how data turns into decisions and P&L. Lately I apply AI coding agents actively to engineering work, and I am especially interested in data structures that AI agents, not only people, can understand and use.

Experience

CryptoQuant

Aug 2023 โ€“ Present

Web3 Data Engineer

Owned the data platform end to end: blockchain and market data ingestion, transformation, data serving for institutional clients, and pipeline infrastructure

Data platform modernization: open table format serving

  • Migrated the PostgreSQL-centered API serving layer to Delta Lake on Databricks to meet the varied data consumption needs of institutional clients, including US brokerages
  • Exposed an Iceberg catalog through Delta UniForm so clients can connect Snowflake and BigQuery directly, opening a data sales channel beyond the API

Analytics pipeline standardization and improvement

  • Managed and optimized the data pipelines of the BigQuery-based analytics platform
  • Standardized transformation pipelines with dbt and moved large-scale computation to distributed Spark processing, shortening data latency

XRP Ledger data product

  • Built pipelines that collect XRP Ledger and market data into PostgreSQL and BigQuery
  • Secured a competitive data edge with real-time updates and transaction data decoding

Workflow orchestration and infrastructure

  • Built and operated HA Airflow managed with ArgoCD, Kustomize, and Helm, securing pipeline stability and operational efficiency
  • Moved EC2 services to EKS and introduced Karpenter autoscaling; provisioned batch workloads quickly and flexibly with SPOT instances and Kubernetes restart logic

Kakao

Dec 2019 โ€“ Aug 2023

ML/Data Engineer, Recommendation Team

Developed and operated the recommendation systems of Kakao and Daum's major services and the high-volume log pipelines behind them

Integrated pipeline for high-volume user logs

  • Developed and operated pipelines that integrate and preprocess high-volume user logs in real time across KakaoTalk, Gift, advertising, Daum News, and Daum Story
  • Developed and maintained recommendation systems for multiple services, improving user engagement and experience

Led the adoption of Airflow 2.0

  • Led the adoption of Apache Airflow 2.0, improving the scalability and operational efficiency of task orchestration (presented at if(kakao))

Search keyword ad recommendation pipeline

  • Built a data pipeline that predicts pCTR from Daum Search query-consumption history and recommends keyword ads
  • Reached responses within 50 ms with vector DB (ANN) serving

Projects

Quant trading system: build and operation

Jan 2025 โ€“ Present

Personal project

A personal project to design, implement, and run systematic trading strategies on crypto derivatives and DeFi

  • Designed and ran a delta-hedged market-making strategy that combines Uniswap LP positions with futures positions
  • Implemented synthetic strategies on futures and options, and automated position management and risk monitoring
  • Built pipelines that collect and analyze on-chain and off-chain market data, used for signal generation and strategy backtesting

Skills

Core skills

  • dbt
  • Airflow
  • Spark

Skills

  • Python
  • SQL
  • BigQuery
  • Databricks
  • Delta Lake
  • Apache Iceberg
  • Kafka
  • PostgreSQL
  • Kubernetes
  • AWS
  • ArgoCD
  • Helm
  • Karpenter
  • Prometheus
  • Grafana
  • Redis
  • MongoDB

Industries

  • Finance
  • Content
  • Advertising & marketing

Education

Yonsei University

Feb 2013 โ€“ Aug 2019

Bachelor's in Industrial Engineering

Activities

Decipher blockchain research group

Aug 2023 โ€“ Aug 2024

Study sessions, conference talks, and research articles

  • Research on running Lightning Network nodes (YouTube, in Korean)
  • Scaling solutions for monolithic chains (article, in Korean)
  • Hands-on on-chain data analysis (in Korean and English)
  • Interplanetary Money (article, in Korean)

Languages

Korean

Native

English

Conversational