Natural Language Processing: A Comprehensive Guide to Understanding and Analyzing Language, Summaries of Natural Language Processing (NLP)

Download Natural Language Processing: A Comprehensive Guide to Understanding and Analyzing Language and more Summaries Natural Language Processing (NLP) in PDF only on Docsity!

NATURAL LANGUAGE

PROCESSING

Module 5 1

NLP

2  Natural language processing (NLP) is a field of CS, AI and computational linguistics concerned with the interactions between computers and human (natural) languages,  In particular, concerned with programming computers to fruitfully process large natural language corpora.

NLP

4  Challenges in natural language processing frequently involve  speech recognition,  natural language understanding,  natural language generation (frequently from formal, machine-readable logical forms),  connecting language and machine perception, dialog systems.

NLP

5

• Language processing problem can be divided as:
  • Processing written text, semantic and syntactic knowledge of the language.
  • Processing spoken language.
• Applications of NLP:
  • Voice recognition, Machine translation, Information visualization, Grammar checking System etc.

Features of language

(that makes it difficult and useful) 7  English sentences are incomplete description of the info that they intend to convey.  Eg. Some dogs are outside.  The expression may mean different things.  Eg. Where’s the water?  No NL program can be complete bcoz new words, expressions etc can be generated easily.  The are many ways to say the same thing.

Useful Features

8  Language give flexibility to users to be vague or precise.  Language helps us to communicate wrt infinite world with finite words.  Language can evolve with experience.  When a lot of information is known, facts imply each other.

Phonetics

10  Pronunciation of different speakers.  Deals with physical building blocks of language sound system.  Pace of Speech.  Ex: I ate eight cakes, different 'k' sounds in 'kite', 'coat', That band is banned.  If we speak in foreign English ( I 8 8 cakes) similarly k (Hindi क for kite & coat).

Phonology

11  Processing of a speech.  Organization of speech sound with in language.  Ex: Bank (finance) v/s. Bank (River),  In hindi, aa-jayenge (will come) or aaj-ayenge (will come today).

13 Happy is a free morpheme because it can appear on its own un means "not", while ness means "being in a state or condition"

Syntactic Analysis

14  It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words.  The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.  In other words, Syntactic Analysis exploit the results of Morphological analysis to build a structural description of the sentence.

Syntactic Analysis

16  Main problems on this level are:  part of speech tagging (POS tagging),  chunking or detecting syntactic categories (verb, noun phrases) and  sentence assembling (constructing syntax tree).

Parsing

17  May be defined as the process of assigning structural descriptions to sequences of words in a natural language.  The goal of Parsing it to convert the flat list of words that forms the sentence into a structure that defines the units that are represented by the flat list.  Converted into a hierarchical structure

Syntactic Analysis

19 Explanation: a sentence is construct from

• noun phrase (NP) & verb phrase (VP).
• Noun Phrase construct from article (art) and noun (n).
• verb phrase are from verb (v) and noun phrase.

Lexical Analysis

20  Obtaining the properties of word.  Ex: 'dog' then you can easily bring an image of dog & its properties like 4 leg, carnivore and animate.  These properties also match with another animals like Lion.  It involves identifying and analyzing the structure of words.  Lexicon of a language means the collection of words and phrases in a language.  Lexical analysis is dividing the whole chunk of text into paragraphs, sentences, and words.