Unified Text-to-Text Transformer (T5-style)

Completed
2025

Built a custom multitask transformer from scratch in PyTorch with multi-head self-attention and cross-attention, using a shared encoder with per-task decoder heads for a unified text-to-text framework. Trained on translation, summarisation, and question-answering using task-conditioned prompts and masking strategies. Evaluated with BLEU, ROUGE, and Exact Match metrics.

Unified Text-to-Text Transformer (T5-style) - Image 1
About This Project

Built a custom multitask transformer from scratch in PyTorch with multi-head self-attention and cross-attention, using a shared encoder with per-task decoder heads for a unified text-to-text framework. Trained on translation, summarisation, and question-answering using task-conditioned prompts and masking strategies. Evaluated with BLEU, ROUGE, and Exact Match metrics.

Technologies Used
PyTorchPythonNumPyTransformers
Project Links
Resume
Unified Text-to-Text Transformer (T5-style) — Project