Multilingual Modeling
How do we go beyond current transfer architectures and improve transfer across languages?
How to reduce language divergences in the multiNMT architecture?
Better multilingual transfer for generation tasks like one to many translation
What are the language-specific biases in current multilingual modeling approaches, which can be reduced to enable better multilingual transfer.
Self-Supervised Learning
Pre-training models have driven progress in AI. Some important directions to explore
Additional pre-training loss functions for different NLG applications: Alternative pre-training objectives, data augmentation, connection to information theory, non-contrastive objective functions for faster training. Look at work in vision domain.
Combining supervised and unsupervised data. In this context, understand when and how pre-training helps.
Limited finetuning to optimize computational budget
Training with Noisy data
Across projects, we are relying on mined data for training our AI models. The noisy nature of these datasets is a fact of life. How do we train our models to perform better the face of such noisy training data.
Repairing noisy training data
Training objectives that take noise into account
Knowledge Distillation
Building Indic language specific semantic models (like LaBSE) as well as divergent semantic models
AI for extremely low-resource languages
Modern AI relies on large amount of data (raw or annotated). For low-resource languages beyond top-12 Indian languages, novel methods have to be developed.
Pre-training for low-resources languages
Utilizing dictionaries and other lexical resources
Better use of language relatedness
Better zeroshot transfer
Translation between Indian languages
While the major focus has been on translation between English and Indian languages, this is also an important need by itself as well as its potential to improve English ←→ IL translation for low-resource languages. The area has been under-investigated and some directions to explore include:
Utilizing similarity between Indian languages
Models combining all translation directions
Multi-source translation systems