Search

Word embeddings are used to convert human language into a numerical form by encoding the semantic properties of words. Using it each word can be transformed to a set of N-dimensional vectors. It plays a vital role in processing of linguistic applications like natural language inference, information retrieval, sentiment analysis, etc. The goal of word embedding is to capture the meaning of words in their context. And it also find the semantic relationships and similarities between words. The aim of this work is to summarize the existing embedding techniques for words and available corpus for Malayalam language. Since Malayalam is a resource-constrained Indian language, this paper is expected to help NLP researchers in Malayalam to identify the existing resources and to improve the current research trend.

The morphological richness and the agglutinative nature of Malayalam make it necessary to retrieve the root word from its inflected form in most of the NLP tasks. This paper presents an approach to identify the suffixes of Malayalam words using MBLP approach. The idea here is to use Memory Based Language Processing (MBLP) algorithm for Malayalam suffix identification. MBLP is an approach to language processing based on exemplar storage during learning and analogical reasoning during processing. Sandhi splitting is essential for morphological analysis, document indexing and topic modeling. Suffix separation improves the quality of machine translated text. Training instances created from words are manually annotated for their segmentation and the system is trained using TiMBL (Tilberg Memory Based Learner). The paper presents memory-based model of Malayalam suffix identification and its generalization accuracy.

Word Sense Disambiguation (WSD) is the task of identifying the correct sense of a word in a specific context when the word has multiple meaning. WSD is very important as an intermediate step in many Natural Language Processing (NLP) tasks, especially in Information Extraction(IE), Machine Translation(MT) and Question/Answering Systems. Word sense ambiguity arises when a particular word has more than one possible sense. The peculiarity of any language is that it includes a lot of ambiguous words. Since the sense of a word depends on its context of use, disambiguation process requires the understanding of word knowledge. Automatic WSD systems are available for structured languages like English, Chinese, etc. But Indian languages are morphologically rich and thus the processing task is very complex. The aim of this work is to develop a WSD system for Malayalam, a language spoken in India, predominantly used in the state of Kerala. The proposed system uses a corpus which is collected from various Malayalam web documents. For each possible sense of the ambiguous word, a relatively small set of training examples (seed sets) are identified which represents the sense. Collocations and most co-occurring words are considered as training examples. Seed set expansion module extends the seed set by adding most similar words to the seed set elements. These extended sets act as sense clusters. The most similar sense cluster to the input text context is considered as the sense of the target word.

Words are the important building blocks of every language. Morphological generator is used to get the inflected form of a word, given its root word and a set of properties such as lexical category and morphological properties. Morphological Generation and analysis are necessary for developing computational grammars as well as machine translation systems. This paper presents a morphological generator for Malayalam nouns using Memory Based Language Processing (MBLP) approach. MBLP is an approach to language processing based on exemplar storage during learning, and analogical reasoning during processing. For training the system, a training corpus is created. It contains the basic examples of root words and their features. The feature set for this Malayalam noun generation system includes number, case, and the last syllable of the root word. Tilburg Memory based Learner (TiMBL) is used for training the system. The system doesn't require a dictionary or rules for its working. It gives a satisfactory result, having an accuracy of 93.68%

Text chunking consists of dividing a text into syntactically correlated parts of words. Given the words and their morphosyntactic class, a chunker will decide which words can be grouped as chunks. Malayalam is a free word order language and has relatively unrestricted phrase structures that make the problem of chunking quite challenging. This paper aims to develop a text chunker for Malayalam using Memory-Based Learning (MBL) approach. Memory-Based Learning is a machine learning methodology based on the idea that the direct reuse of examples using analogical reasoning is more suited for solving language processing problems than the application of rules extracted from those examples. The chunker was trained using the tool Memory-Based Tagger (MBT) with words and their POS tags as features. The chunker demonstrated an accuracy of 97.14%.

Transliteration forms an essential part of transcription which converts text from one writing system to another. The need for translating data has become larger than before as the world is getting together through social media. Machine transliteration has emerged as a part of information retrieval and machine translation projects to translate named entities, that are not registered in the dictionary, based on phonemes and graphemes. This paper proposes a machine learning technique that performs transliteration from Tamil to Malayalam, two languages that belong to Dravidian family. Transliteration can be used to supplement machine translation process by handling the issues that can happen due to the presence of named entities.

This paper presents an approach to morphological analysis of Malayalam words as a classification Problem. The idea here is to use Memory Based Language Processing (MBLP) algorithm for Malayalam morphological analysis. MBLP is an approach to language processing based on exemplar storage during learning, and analogical reasoning during processing. The aim of the system is to find the citation forms (or meaningful parts) of words rather than a detailed morphological analysis. Training instances created from words are manually annotated for their segmentation and the system is trained using TiMBL (Tilburg Memory based Learner). The paper presents memory based model of Malayalam morphological analysis and its generalization accuracy.

In this paper, Sentence level Sentiment Analysis of Malayalam movie reviews is done by classifying the polarity of opinions obtained from the user as positive, negative and neutral. Sentiment analysis is an application of Natural Language Processing and text analysis which helps to identify the emotions in a given context. In this work a hybrid approach for Sentiment Analysis is used in which Machine Learning method is used for tagging and Fuzzy Logic is used to find the membership of the review in each sentiment class. Fuzzy logic can be used to handle the vagueness in natural language. Manually tagged data are trained using TnT and Fuzzy rules are incorporated for identifying and classifying the emotions. Certain other rules are also incorporated to handle certain special cases. The fuzzy rules yield output that varies in degree between 0 and 1.

In this paper, we present the details of a phonotactic language identification (LID) system developed for five Indian languages, English (Indian), Hindi, Malayalam, Tamil and Kan-nada. Since there are no publicly available speech databases for English, Malayalam and Kannada, we developed the database for each of the target languages by downloading the audio files from YouTube videos and removing the non-speech signals manually. The system was tested using a test data set consisting of 40 utterances with duration of 30, 10, and 3 sees, in each of 5 target languages. The performance evaluation was done separately accordingly to the NIST benchmarking sessions, for 30s, 10s and 3s segments separately. For the baseline system, we got an overall EER of 10.41 %, 19.56 % and 31.45 % for 30, 10, and 3 sees segments when tested with a 3-gram language model. The use of 4-gram language model has helped enhance the performance of the LID system to 9.81 %, 19.38 % and 32.77% respectively for 30,10 and 3 sees test segments. Further, by using the n-gram smoothing, we were able to improve the EER of the LID system, 9.02 %, 18.70 % and 29.24 % for 3-gram language models and 8.88 %, 16.46 % and 32.03 % for 4-gram language models, respectively for 30,10, and 3 sec test segments. The study shows that the use of 4-gram language models can help enhance the performance of LID systems for Indian languages.

Malayalam, a classical language in India, is spoken by over 40 million people. This paper proposes an effective information retrieval system for Malayalam, which retrieve Malayalam documents relevant to the user's information need. The proposed system improves effectiveness by considering synonyms and negations of the terms specified in the query. Though the evaluation of the system is performed over a small corpus, the results are promising. The proposed system is thus relevant for various natural language processing tasks in an highly agglutinative language like Malayalam.

Stemming is the process of removing the affixes from inflections and to return the root form. Malayalam is highly agglutinative in nature and hundreds of inflections are possible for each word. An effective stemmer in Malayalam is not yet implemented. This paper presents a lightweight stemmer for Malayalam, which conflates terms by suffix removal. The proposed stemmer is both computationally inexpensive and domain independent and will serve as a vital part in many areas of Malayalam Language Computing.