Unified Text-to-Text Transformer (T5-style)
Completed2025
Built a custom multitask transformer from scratch in PyTorch with multi-head self-attention and cross-attention, using a shared encoder with per-task decoder heads for a unified text-to-text framework. Trained on translation, summarisation, and question-answering using task-conditioned prompts and masking strategies. Evaluated with BLEU, ROUGE, and Exact Match metrics.
About This Project
Built a custom multitask transformer from scratch in PyTorch with multi-head self-attention and cross-attention, using a shared encoder with per-task decoder heads for a unified text-to-text framework. Trained on translation, summarisation, and question-answering using task-conditioned prompts and masking strategies. Evaluated with BLEU, ROUGE, and Exact Match metrics.
Technologies Used
PyTorchPythonNumPyTransformers
Project Links