# MANGO: A Corpus of Human Ratings for Speech

**MANGO** (*MUSHRA Assessment corpus using Native listeners and Guidelines to understand human Opinions at scale*) is the first large-scale dataset designed for evaluating Text-to-Speech (TTS) systems in Indian languages. 

### Key Features:
- **246,000 human ratings** of TTS-generated outputs and ground-truth human speech.
- Covers two major Indian languages:
  - **Hindi**
  - **Tamil**
- Based on the **MUSHRA** (Multiple Stimuli with Hidden Reference and Anchor) test methodology.
- Ratings are provided on a continuous scale from **0 to 100**, with discrete quality categories:
  - **100-80**: Excellent
  - **80-60**: Good
  - **60-40**: Fair
  - **40-20**: Poor
  - **20-0**: Bad
- Includes evaluations involving:
  - *MUSHRA*: with explicitly mentioned high-quality references.
  - *MUSHRA-NMR*: without explicitly mentioned high-quality references. 
  - *MUSHRA-DG*: with detailed guidelines across fine-grained dimensions
  - *MUSHRA-DG-NMR*: with detailed guidelines across fine-grained dimensions and without explicitly mentioned high-quality references.  

### Available Splits

The dataset includes the following splits based on the test type and language. 

| **Split**            | **Number of Ratings** |
|---------------------------|:---------------------------------------------------------------------:|
| **Hindi__MUSHRA**           | 56500 |
| **Hindi__MUSHRA_DG**        | 10000 |
| **Hindi__MUSHRA_DG_NMR**    | 10000 |
| **Hindi__MUSHRA_NMR**       | 51000 |
| **Tamil__MUSHRA**           | 50000 |
| **Tamil__MUSHRA_DG**        | 10000 |
| **Tamil__MUSHRA_DG_NMR**    | 10000 |
| **Tamil__MUSHRA_NMR**       | 48500 |



### Why Use MANGO?
- Addresses limitations of traditional **MOS** and **CMOS** tests.
- Enables robust benchmarking for:
  - Comparative analysis across multiple TTS systems.
  - Evaluations in diverse linguistic contexts.
  - Large-scale studies with multiple raters.

This dataset is a valuable resource for researchers and practitioners working on speech synthesis, TTS evaluation, and related fields.


### License

This dataset is released under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).