Explore LiFE projects, datasets, and models.

## About Nyishi Nyishi is an under-resourced Tibeto-Burman language spoken by the Nyishi people, the largest ethnic group in Arunachal Pradesh, India. According to the 2011 Census, there are approximately **3 lakh speakers** of the language. The language belongs to the Tani branch of the Tibeto-Burman language family. Owing to its status as one of the largest languages of Arunachal Pradesh, some resources for language development, such as a corpus and a language model, have been developed. However, the language lacks any large, significant speech or text corpus. ## Dataset Description The Nyishi Speech Dataset, developed as part of the [Speech Datasets and Models for Tibeto-Burman Languages (Project SpeeD-TB)](https://sites.google.com/view/speed-tb/ "SpeeD-TB Project Website") funded under Mission Bhashini, is a transcribed speech corpus of Nyishi. The full dataset comprises over **200 hours of high-quality audio recordings paired with accurate transcriptions in both IPA and Roman script**, making it the ** largest speech resource for the language** that not only enables building and evaluating voice models in low-resource linguistic settings but also enables large-scale linguistic description and documentation of the language. The audio data captures a diverse range of speakers across different age groups, genders, and education levels, ensuring variability in pronunciation, speech patterns, and tone. It includes both spontaneous and read speech collected in naturalistic and semi-controlled environments, thereby reflecting real-world linguistic usage. The transcriptions are carefully prepared and normalised to maintain consistency, supporting robust model training. This dataset also contributes to the preservation and digital documentation of the Nyishi language and culture by transforming oral knowledge into structured, machine-readable formats. Almost 60% of the data in the corpus is included from domains of agriculture, education and science & technology. The rest of the data is from varied domains including culture, lifecycle, sports, entertainment, healthcare and oral history, thereby giving extensive coverage. We have also used a variety of elicitation methods for collecting the data, including translations, narrations, lectures, role-play, spontaneous conversations, interviews and picture and video descriptions. The released dataset is meticulously mapped to rich metadata, including demographic and linguistic metadata of the speakers, domains, elicitation methods and individual prompts. The audio included in the current dataset is already sliced at sentence level, thereby ready to be integrated into the model training pipeline out-of-the-box. The overall dataset of the project is collected over multiple phases and using multiple questionnaires. This repository contains the full dataset for the language collected till now. ## Ethical Considerations, IPR and Attribution This repository represents our commitment to not only fair remuneration to the speakers of the language but also to co-ownership and equal IPR to all the contributors who have built the dataset. We believe this is the first step to move away from extractive data collection and use practices and ensure fairness in our treatment of the community members. As such, we have listed all speakers and transcribers as Contributors to the dataset. This dataset is only licensed to other researchers for use in their research projects. More details about licensing and commercial use conditions are given in the License and Commercial Use sections.

### The Dataset The current data preview of Irula is being released as part of the **[Project BoLI](https://boli.unreal-tece.co.in)**. This preview is a reflection of the full dataset and consists of the following - 1. Speech Recordings of 100 sentences in the language. 2. Transcriptions in IPA and native script(s). 3. Translations in English (which also act as prompts for the translation sentences). 4. Detailed speaker metadata, including their demographic, educational and linguistic profile. 5. Prompt in English and Hindi. The full dataset contains the following - 1. Translations of a minimum of 1000 carefully selected sentences. These sentences are selected to represent diverse morphosyntactic categories generally found in Indian languages such as demonstratives, classifiers, TAM morphology, and different sentence structures such as transitives and ditransitives. These sets of sentences are based on the standardised questionnaires built by Linguists for writing the sketch grammar of any human language, thereby, representing almost full range of morphosyntactic properties exhibited in the language. Unlike other benchmarks which focus largely on lexical level evaluation (aka domains), this is the first benchmark dataset that evaluates model's performance on a range of morphosyntactic structures, even the most uncommon ones. 2. More than 600 narrative speeches collected across 8 domains. These are recorded by at least 2 speakers. Narrations, along with their prompts, can be used to evaluate AI models on a range of prompt-based tasks. 3. Along with transcriptions in IPA and other scripts and translation, all data is interlinearly glossed at morphemic level. This gives a word-by-word meaning and morphosyntactic information, thereby, enabling evaluation of models on their deep grammatical knowledge, ability for cross-linguistic comparison and generalisation and reasoning capacity and skills in language-related puzzles. This allows for evaluating the model's capacity on reasoning tasks beyond mathematical reasoning tasks as well as their capacity to arrive at typological generalisations. 4. In addition to the data and prompt itself, as mentioned earlier, we are also making available the detailed speaker metadata and prompt-level metadata viz domain, elicitation method, multilingual prompt, target grammatical category (for translation sentences), etc. 5. In accordance with our data governance policy, all contributors to the dataset, including those recording the dataset and those transcribing it, are named and listed as data contributors. ### Dataset Preparation The speech included as part of this dataset was recorded by native speakers using a data collection application on a mobile device (Karya or in-house app, Atekho), by translating sentences from English or Hindi to the target language. The recorded speech was validated and then transcribed manually by trained linguists, working with the native speakers, following the [guidelines for the project](https://docs.google.com/document/d/1h-EcxxaNiGzdWiwc1-V_S-Xt6n4MwE6F/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) using MATra Lab, a part of the [LiFE Suite Ecosystem](https://life.unreal-tece.co.in), developed by [Unreal Tece LLP](https://unreal-tece.co.in). [Project BoLI Guidelines](https://docs.google.com/document/d/1h-EcxxaNiGzdWiwc1-V_S-Xt6n4MwE6F/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) ### About Irula Irula belongs to the Dravidian, Southern Dravidian, Tamil-Kannada sub-branch (ISO 639-3: `iru`, Glottocode: `irul1243`). ### Speakers According to the [2011 Census of India](https://censusindia.gov.in/nada/index.php/catalog/42561), the Irula speech community is estimated at approximately 200,000 native speakers. ### Distribution in India Irula is primarily spoken in Southern India. Its primary geographic concentration includes the Western Ghats region of Tamil Nadu (specifically the Nilgiris, Coimbatore, and Erode districts, as well as northern districts like Tiruvallur and Kanchipuram) and adjacent areas in Kerala (primarily the Attappadi valley in Palakkad district), extending into border areas of Karnataka. ### Major grammatical features * **Phonology:** Contains a typical Southern Dravidian consonant system with a full series of retroflex stops, nasals, and liquids, as well as distinct centralized short and long vowels (such as /ɨ/ and /ɵ/). * **Morphology:** Strictly agglutinative. Substantives inflect for case (including dative, locative, sociative, and ablative) using suffixes, while verbs take suffixal markers for tense, aspect, mood, and person-number-gender (PNG) agreement. * **Syntax:** Characterized by a canonical Subject-Object-Verb (SOV) default word order, head-final noun phrases, use of postpositions, and non-finite clause subordination (such as verbal participles). ### Further reading * [Irula language on Wikipedia](https://en.wikipedia.org/wiki/Irula_language) * [Glottolog Language Details: Irula of the Nilgiri](https://glottolog.org/resource/languoid/id/irul1243) * [ISO 639-3 Code Aspect: `iru`](https://iso639-3.sil.org/code/iru) ### About Project BoLI [Project BoLI](https://boli.unreal-tece.co.in) is the flagship project of [UnReaL-TecE LLP](https://unreal-tece.co.in), which aims to build high-quality datasets for benchmarking and evaluating different kinds of AI tasks, including speech-to-text, machine translation, grammatical analysis and reasoning tasks and prompt-based evaluation of LLMs. While the project aims to build these datasets for every Indian language and variety, the primary focus is on over 1300 underserved languages and both first and second language varieties of major, scheduled languages. The project's uniqueness is not just limited to the kind of benchmarking tasks it supports (including proposing some novel tasks) and also the kind of languages and communities it supports but also in its contextualisation and implementation of a unique data governance model (not yet implemented anywhere across the globe), which mandates that all datasets released by the project and their derivatives (including the models) are co-owned by the community members and all contributors of the project, and any permission to use the dataset is not a transfer of ownership but a revocable license to use it. The conditions under which the license could be revoked are clearly mentioned as part of the BoLI License. The complete details of the project, languages and communities supported till now, the quantum of data available till now, its data governance model and other relevant documents are all publicly accessible at the [project website](https://boli.unreal-tece.co.in). ### Ethical Considerations, Consent, IPR and Attribution Project BoLI and this repository represents our commitment to not only fair remuneration to the speakers of the language but also to co-ownership and equal IPR to all the contributors who have built the dataset. We believe this is the first step to move away from the extractive data collection and use practices and ensure fairness in our treatment of the community members. As such, we have listed all speakers and transcribers as Contributors to the dataset (we insist that they are co-owners of the dataset, even though the HuggingFace platform does not provide us an explicit way of stating that) and they are further recognised as Speakers and Annotators of the dataset. This dataset is only licensed to other researchers for use in their research projects. More details about licensing and commercial use conditions are given in the License and Commercial Use sections. [Project BoLI - Data Governance Policy](https://docs.google.com/document/d/1DBGZVFJ4RG57j4rFphKIrvgbpfpj8HqZ/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [BoLI Ethics Principle & Pledge](https://docs.google.com/document/d/1hMQzwsOJF4QDfk_DOSIGLOF3GApLQOZA/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - Digital Consent Form](https://docs.google.com/document/d/1BTQUrgVTS-sVszUUDOkbnyUNusAHVYm4/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - Field Recording of Oral Consent](https://docs.google.com/document/d/1xoRxXcNES2oQqBajY3Z7HxMwpt0oOyDK/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - TnC](https://docs.google.com/document/d/1UuiIAHWXXKF202MYqLrtOy1ftUk__POW/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) ### Dataset Access Full dataset for the language can be browsed and queried on our [app](https://life.unreal-tece.co.in). The results of the evaluation of different models will also be made available on the same link. If you would like to access the full dataset for your own use or would like to work with us in collecting more data for any language or variety, or collaborate with us in this initiative in any other way, please get in touch with us.

### The Dataset The current data preview of Irula is being released as part of the **[Project BoLI](https://boli.unreal-tece.co.in)**. This preview is a reflection of the full dataset and consists of the following - 1. Speech Recordings of 10 narrations in the language. 2. Transcriptions in IPA and native script(s). 3. Translations in English (which also act as prompts for the translation sentences). 4. Detailed speaker metadata, including their demographic, educational and linguistic profile. 5. Prompt in English and Hindi. The full dataset contains the following - 1. Translations of a minimum of 1000 carefully selected sentences. These sentences are selected to represent diverse morphosyntactic categories generally found in Indian languages such as demonstratives, classifiers, TAM morphology, and different sentence structures such as transitives and ditransitives. These sets of sentences are based on the standardised questionnaires built by Linguists for writing the sketch grammar of any human language, thereby, representing almost full range of morphosyntactic properties exhibited in the language. Unlike other benchmarks which focus largely on lexical level evaluation (aka domains), this is the first benchmark dataset that evaluates model's performance on a range of morphosyntactic structures, even the most uncommon ones. 2. More than 600 narrative speeches collected across 8 domains. These are recorded by at least 2 speakers. Narrations, along with their prompts, can be used to evaluate AI models on a range of prompt-based tasks. 3. Along with transcriptions in IPA and other scripts and translation, all data is interlinearly glossed at morphemic level. This gives a word-by-word meaning and morphosyntactic information, thereby, enabling evaluation of models on their deep grammatical knowledge, ability for cross-linguistic comparison and generalisation and reasoning capacity and skills in language-related puzzles. This allows for evaluating the model's capacity on reasoning tasks beyond mathematical reasoning tasks as well as their capacity to arrive at typological generalisations. 4. In addition to the data and prompt itself, as mentioned earlier, we are also making available the detailed speaker metadata and prompt-level metadata viz domain, elicitation method, multilingual prompt, target grammatical category (for translation sentences), etc. 5. In accordance with our data governance policy, all contributors to the dataset, including those recording the dataset and those transcribing it, are named and listed as data contributors. ### Dataset Preparation The speech included as part of this dataset was recorded by native speakers using a data collection application on a mobile device (Karya or in-house app, Atekho), by translating sentences from English or Hindi to the target language. The recorded speech was validated and then transcribed manually by trained linguists, working with the native speakers, following the [guidelines for the project](https://docs.google.com/document/d/1h-EcxxaNiGzdWiwc1-V_S-Xt6n4MwE6F/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) using MATra Lab, a part of the [LiFE Suite Ecosystem](https://life.unreal-tece.co.in), developed by [Unreal Tece LLP](https://unreal-tece.co.in). [Project BoLI Guidelines](https://docs.google.com/document/d/1h-EcxxaNiGzdWiwc1-V_S-Xt6n4MwE6F/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) ### About Irula Irula belongs to the Dravidian, Southern Dravidian, Tamil-Kannada sub-branch (ISO 639-3: `iru`, Glottocode: `irul1243`). ### Speakers According to the [2011 Census of India](https://censusindia.gov.in/nada/index.php/catalog/42561), the Irula speech community is estimated at approximately 200,000 native speakers. ### Distribution in India Irula is primarily spoken in Southern India. Its primary geographic concentration includes the Western Ghats region of Tamil Nadu (specifically the Nilgiris, Coimbatore, and Erode districts, as well as northern districts like Tiruvallur and Kanchipuram) and adjacent areas in Kerala (primarily the Attappadi valley in Palakkad district), extending into border areas of Karnataka. ### Major grammatical features * **Phonology:** Contains a typical Southern Dravidian consonant system with a full series of retroflex stops, nasals, and liquids, as well as distinct centralized short and long vowels (such as /ɨ/ and /ɵ/). * **Morphology:** Strictly agglutinative. Substantives inflect for case (including dative, locative, sociative, and ablative) using suffixes, while verbs take suffixal markers for tense, aspect, mood, and person-number-gender (PNG) agreement. * **Syntax:** Characterized by a canonical Subject-Object-Verb (SOV) default word order, head-final noun phrases, use of postpositions, and non-finite clause subordination (such as verbal participles). ### Further reading * [Irula language on Wikipedia](https://en.wikipedia.org/wiki/Irula_language) * [Glottolog Language Details: Irula of the Nilgiri](https://glottolog.org/resource/languoid/id/irul1243) * [ISO 639-3 Code Aspect: `iru`](https://iso639-3.sil.org/code/iru) ### About Project BoLI [Project BoLI](https://boli.unreal-tece.co.in) is the flagship project of [UnReaL-TecE LLP](https://unreal-tece.co.in), which aims to build high-quality datasets for benchmarking and evaluating different kinds of AI tasks, including speech-to-text, machine translation, grammatical analysis and reasoning tasks and prompt-based evaluation of LLMs. While the project aims to build these datasets for every Indian language and variety, the primary focus is on over 1300 underserved languages and both first and second language varieties of major, scheduled languages. The project's uniqueness is not just limited to the kind of benchmarking tasks it supports (including proposing some novel tasks) and also the kind of languages and communities it supports but also in its contextualisation and implementation of a unique data governance model (not yet implemented anywhere across the globe), which mandates that all datasets released by the project and their derivatives (including the models) are co-owned by the community members and all contributors of the project, and any permission to use the dataset is not a transfer of ownership but a revocable license to use it. The conditions under which the license could be revoked are clearly mentioned as part of the BoLI License. The complete details of the project, languages and communities supported till now, the quantum of data available till now, its data governance model and other relevant documents are all publicly accessible at the [project website](https://boli.unreal-tece.co.in). ### Ethical Considerations, Consent, IPR and Attribution Project BoLI and this repository represents our commitment to not only fair remuneration to the speakers of the language but also to co-ownership and equal IPR to all the contributors who have built the dataset. We believe this is the first step to move away from the extractive data collection and use practices and ensure fairness in our treatment of the community members. As such, we have listed all speakers and transcribers as Contributors to the dataset (we insist that they are co-owners of the dataset, even though the HuggingFace platform does not provide us an explicit way of stating that) and they are further recognised as Speakers and Annotators of the dataset. This dataset is only licensed to other researchers for use in their research projects. More details about licensing and commercial use conditions are given in the License and Commercial Use sections. [Project BoLI - Data Governance Policy](https://docs.google.com/document/d/1DBGZVFJ4RG57j4rFphKIrvgbpfpj8HqZ/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [BoLI Ethics Principle & Pledge](https://docs.google.com/document/d/1hMQzwsOJF4QDfk_DOSIGLOF3GApLQOZA/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - Digital Consent Form](https://docs.google.com/document/d/1BTQUrgVTS-sVszUUDOkbnyUNusAHVYm4/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - Field Recording of Oral Consent](https://docs.google.com/document/d/1xoRxXcNES2oQqBajY3Z7HxMwpt0oOyDK/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) [Project BoLI - TnC](https://docs.google.com/document/d/1UuiIAHWXXKF202MYqLrtOy1ftUk__POW/edit?usp=sharing&ouid=106672251267017181111&rtpof=true&sd=true) ### Dataset Access Full dataset for the language can be browsed and queried on our [app](https://life.unreal-tece.co.in). The results of the evaluation of different models will also be made available on the same link. If you would like to access the full dataset for your own use or would like to work with us in collecting more data for any language or variety, or collaborate with us in this initiative in any other way, please get in touch with us.