Aaron Leevord Fernandes
Machine Learning Engineer working on generative models and computer vision, taking projects from research prototype to trained model at scale.
I build and ship machine learning systems end to end: training models in PyTorch, optimizing them for inference, and deploying them behind real infrastructure. My work spans computer vision, generative models, and reinforcement learning, with a strong focus on performance, scalable training, and production deployment (ONNX, CoreML, Docker, AWS, FastAPI).
Currently a research assistant on synthetic driving data for autonomous systems at Indiana University. I came to ML from software engineering (production backend work in enterprise banking before a master's in AI), and I'm comfortable owning a model from experiment to deployment.
- 2026 – now Graduate Research Assistant, Astemo × Indiana University, working on generative models for autonomous driving: image editing & generation, video / driving world models, vLLM-Omni multimodal inference, and geometry & computer-vision benchmarking to validate synthetic data. Earlier in 2026: backend engineering at Heartland.
- 2024 – 2025 ML engineering & research, spanning computer vision at Leung Research Group and reinforcement learning at Indiana University.
- 2024 NCAA Women's Basketball Prediction, a Random Forest tournament model; ranked top 5 of 67 in a national competition.
- 2023 – 2025 M.S. Intelligent Systems Engineering, Indiana University Bloomington (GPA 3.9). The pivot into AI: computer vision, diffusion, reinforcement learning.
- 2022 – 2023 Software Engineer, Tata Consultancy Services, building Java / Spring Boot microservices for HDFC Bank (100K+ daily transactions).
- 2017 – 2021 B.S. Electronics & Telecommunications, Goa College of Engineering. Networking & IoT; built a fuzzer for the Contiki IoT operating system.
A from-scratch latent Diffusion Transformer trained with rectified flow on 2M LAION-Aesthetics images. Attention, AdaLN-Zero conditioning, patchify, the rectified-flow objective, Euler sampler, and classifier-free guidance are all hand-written (~600 lines). A profiler-driven attention fix gave 2.2× throughput and 2× less memory; streaming data pipeline measured at 757 imgs/s over 1,976 WebDataset shards.
A small, readable diffusion repo that starts from a vanilla DDPM ε-prediction model and moves toward modern diffusion in minimal additive steps. U-Net with timestep embeddings and selective self-attention on CIFAR-10, EMA, checkpointing, and FID reporting every 10k steps, built to be instrumented and legible rather than a heavy framework.
Unsupervised Industrial Anomaly Detection
A Vision Transformer autoencoder with a Capsule Network and Gaussian mixture scoring for detecting and localizing defects in industrial imagery. Trained on the BeanTech (BTAD) dataset, improving AUROC from 0.78 to 0.94 over ResNet-50 baselines.
An LLM-assisted retrieval and knowledge-graph system for grounded-theory qualitative research, built with researchers at NICC Brussels. Extracts knowledge graphs from interview transcripts and supports source-traceable querying across 200+ documents, pairing vector embeddings with graph-backed retrieval so every answer traces back to its origin passage.
Goal-conditioned reinforcement learning combining Soft Actor-Critic, Hindsight Experience Replay, and Stein Variational Goal Generation for curriculum-style goal sampling in sparse-reward robotics environments like FetchReach. Hydra-configured, W&B-tracked.
A model-based RL algorithm inspired by Deep Value-and-Predictive-Model Control: learned dynamics, value approximation, and Cross-Entropy Method planning over a prediction horizon for continuous control. Effective in dense-reward settings, with sparse rewards analyzed as the open challenge.
ComicPaliGemma: a Vision-Language Model from scratch
A from-scratch implementation of Google's PaliGemma VLM (SigLIP ViT vision encoder plus a Gemma decoder), validated against the full pretrained checkpoint. Fine-tuned on 5K+ comic panels with 4-bit QLoRA to cut peak VRAM by 65%, with a long-context autoregressive generator producing coherent narration across 10+ panels. Custom Triton / FlashAttention-2-style kernels gave 5 to 10× faster inference than standard PyTorch attention.
Apr 2024 – Dec 2025
Machine Learning Engineer, Computer Vision
Leung Research Group
Self-supervised vision and computational imaging for OCT video-to-histology translation. Built U-Net surrogate models approximating FDTD optical simulations on HPC (SLURM/Linux), cutting inference from ~3 min to sub-second while holding SSIM > 0.85.
Jan 2025 – Jun 2025
Machine Learning Engineer, Reinforcement Learning
Indiana University
Goal-conditioned RL (HER, curriculum policy optimization) for sparse-reward navigation, with experiment infrastructure orchestrating 100+ multi-seed runs via Hydra, MLflow, and W&B.
Mar 2026 – present
Graduate Research Assistant, Synthetic Driving Data
Indiana University (Astemo-sponsored)
Building and benchmarking generative systems for autonomous driving: diffusion-based image editing and generation, video generation, and driving world models for synthetic data. Serving large-scale multimodal inference with vLLM-Omni, and applying geometry- and computer-vision-based benchmarking and validation to ensure the generated data is good enough to train on.