Resume
Open to Data Engineer roles · Remote, hybrid or on-site

Hi, I'm Ayush. I build data pipelines that keep fintech running.

Data Engineer in Pune with 3.5+ years of experience in real-time streaming, AWS lakehouses and AI agents that work on top of company data.

Download resume Get in touch
Ayush Kumar Singh

Who is he?

Senior Data Engineer at Bajaj Broking

I started as a self-taught mobile developer and moved into data engineering during my internship at Bajaj Broking. Today I design change data capture pipelines, AWS data lakes with Hudi and Iceberg, PII and DPDP compliance controls, and Copilot agents that let teams ask questions of their data in plain language.

Pune, India

Education

Symbiosis Institute of Technology
B.Tech in Computer Science and Engineering, 2019 to 2023
Honours in Data Science

What I'm working on

  • Copilot agents that answer questions from Power BI data
  • Real-time CDC pipelines on Kafka
  • AI agents that watch ETL pipelines on AWS

Achievements

Super Heroes Award, Bajaj Broking. Recognised for consistently exceeding expectations and delivering exceptional value.

What colleagues say

I had the pleasure of working with Ayush on multiple projects, and he consistently impressed me with his strong data engineering skills and problem-solving mindset. He’s reliable, detail-oriented, and always ready to support the team.

One of his key contributions was optimizing a critical data pipeline, which significantly improved performance. Ayush is a great team player and brings real value to any project he’s part of. I highly recommend him.

Harish Yarlagadda AI Decision Scientist, Accenture Strategy & Consulting
Former colleague, Data Engineer at Bajaj Broking

Skills

Data Engineering & Streaming

  • Apache Kafka
  • Change Data Capture (CDC)
  • Real-time streaming pipelines
  • Batch pipelines
  • Data modeling

AWS & Cloud

  • AWS
  • S3
  • Glue
  • Lambda
  • API Gateway
  • SNS
  • Athena
  • KMS
  • IAM
  • EventBridge

Lakehouse & Frameworks

  • Apache Hudi
  • Apache Iceberg
  • Apache Spark
  • Parquet
  • Docker
  • Jupyter

Databases

  • MSSQL
  • MySQL
  • PostgreSQL
  • MongoDB
  • Elasticsearch
  • Redis

Languages

  • Python
  • PySpark
  • Go
  • R

Machine Learning & AI

  • AI agents
  • Microsoft Fabric agent
  • TensorFlow
  • PyTorch
  • Scikit-learn
  • Keras
  • XGBoost
  • Deep learning

Monitoring & Tools

  • Grafana
  • Fluentd
  • Elasticsearch
  • NGINX
  • Git

Soft Skills

  • Cross-functional collaboration
  • Problem solving
  • Stakeholder communication
  • Technical leadership

Where I've worked and studied

  •   Bajaj Broking
  •   Persistent Systems
  •   Symbiosis Institute of Technology

When I'm not building pipelines

Off the clock

  • 🎮 PC and PS5 games
  • ⚡ Harry Potter books
  • ⚽ Football
  • 🏸 Badminton
Gaane
My playlist on YouTube Music. Play it while you scroll. Press play for a random song from my playlist.
Open playlist

Work

Bajaj Broking

Senior Data Engineer

  • Built and published a Copilot agent in Microsoft Fabric on a Power BI semantic model, with row-level security so each Relationship Manager sees only their clients, and natural language Q&A in Microsoft Teams.
  • Implemented CDC on MS SQL Server with Kafka streaming, reducing query load on the transactional database by 40–60%.
  • Deployed an AI agent on AWS that monitors ETL pipelines and flags failures, anomalies and bottlenecks.

Data Engineer

  • Architected an AWS data lake on Hudi, Iceberg and Parquet, with encryption and governance controls for PII.
  • Built a monitoring solution with Grafana, Fluentd and Elasticsearch: reports 80% faster to create, API performance up 40%.
  • Developed the ETL framework for a Customer Data Platform using Glue, S3 and Athena, and led vendor pipeline integration into the lakehouse.

Data Engineer Intern

  • Built a PII discovery tool for names, phone numbers, emails, Aadhaar and PAN, with encryption aligned to DPDP regulations.
  • Raised campaign CTR by 18–35% with real-time engagement systems, and lifted retention 12% with RFM segmentation.

Persistent Systems

Internship · Martian Summer Internship Program

  • Took part in a structured learning program covering key areas of computer science.
  • Prepared for and completed assessments that tested understanding and application of the material.
  • Finished all assigned modules and assessments, showing a strong grasp of the concepts.

Articles

Data engineering · 5 min read

Streaming database changes into a lakehouse with CDC

A general overview of how Debezium, Kafka and a lakehouse keep heavy reads off your production database.

  • CDC
  • Debezium
  • Kafka
  • Parquet
  • Apache Iceberg
Read article
Data engineering · 6 min read

Parquet vs Iceberg vs Hudi: picking a table format

What an open table format adds over plain files, how Iceberg and Hudi differ, and a simple guide to picking one.

  • Parquet
  • Apache Iceberg
  • Apache Hudi
  • Lakehouse
Read article

Projects

Real-time Data Streaming Pipeline

End-to-end streaming pipeline: Airflow schedules the fetching of mock user data from a public API, Kafka streams it, Spark processes it in real time and writes it to Cassandra. Everything runs in Docker.

  • Airflow
  • Kafka
  • Spark
  • Cassandra
  • Docker
View on GitHub

AR Quiz

Quiz app that uses face tracking to lock answer options based on the player's facial movements.

  • Python
  • Flutter
  • MediaPipe

Heart Disease Prediction

Android app with an ML model that predicts heart disease risk at 81.9% accuracy and gives health tips.

  • Android
  • ML
View on GitHub

Covid Tracker

Daily cases, recoveries and vaccination stats for India and the world, plus vaccination centres with open slots.

  • Android
  • REST APIs
View on GitHub

Quizone

Web platform for running secure, interactive online assessments for institutions and companies.

  • Web
View on GitHub

Akshaya Patra Donation App

Donation app with Razorpay payment integration.

  • Android
  • Razorpay
View on GitHub

See all repositories on GitHub

Hit me up if you want to chat about

  • Data engineering
  • Real-time ETL pipelines
  • AWS lakehouse design
  • Kafka and CDC
  • AI agents on data
ayushksingh7@gmail.com