Package nltk :: Package corpus :: Module chat80
[hide private]
[frames] | no frames]

Module chat80

source code

Overview

Chat-80 was a natural language system which allowed the user to interrogate a Prolog knowledge base in the domain of world geography. It was developed in the early '80s by Warren and Pereira; see http://acl.ldc.upenn.edu/J/J82/J82-3002.pdf for a description and http://www.cis.upenn.edu/~pereira/oldies.html for the source files.

This module contains functions to extract data from the Chat-80 relation files ('the world database'), and convert then into a format that can be incorporated in the FOL models of nltk.sem.evaluate. The code assumes that the Prolog input files are available in the NLTK corpora directory.

The Chat-80 World Database consists of the following files:

   world0.pl
   rivers.pl
   cities.pl
   countries.pl
   contain.pl
   borders.pl

This module uses a slightly modified version of world0.pl, in which a set of Prolog rules have been omitted. The modified file is named world1.pl. Currently, the file rivers.pl is not read in, since it uses a list rather than a string in the second field.

Reading Chat-80 Files

Chat-80 relations are like tables in a relational database. The relation acts as the name of the table; the first argument acts as the 'primary key'; and subsequent arguments are further fields in the table. In general, the name of the table provides a label for a unary predicate whose extension is all the primary keys. For example, relations in cities.pl are of the following form:

  'city(athens,greece,1368).'

Here, 'athens' is the key, and will be mapped to a member of the unary predicate city.

The fields in the table are mapped to binary predicates. The first argument of the predicate is the primary key, while the second argument is the data in the relevant field. Thus, in the above example, the third field is mapped to the binary predicate population_of, whose extension is a set of pairs such as '(athens, 1368)'.

An exception to this general framework is required by the relations in the files borders.pl and contains.pl. These contain facts of the following form:

   'borders(albania,greece).'
   
   'contains0(africa,central_africa).'

We do not want to form a unary concept out the element in the first field of these records, and we want the label of the binary relation just to be 'border'/'contain' respectively.

In order to drive the extraction process, we use 'relation metadata bundles' which are Python dictionaries such as the following:

 city = {'label': 'city',
         'closures': [],
         'schema': ['city', 'country', 'population'],
         'filename': 'cities.pl'}

According to this, the file city['filename'] contains a list of relational tuples (or more accurately, the corresponding strings in Prolog form) whose predicate symbol is city['label'] and whose relational schema is city['schema']. The notion of a closure is discussed in the next section.

Concepts

In order to encapsulate the results of the extraction, a class of Concepts is introduced. A Concept object has a number of attributes, in particular a prefLabel and extension, which make it easier to inspect the output of the extraction. In addition, the extension can be further processed: in the case of the 'border' relation, we check that the relation is symmetric, and in the case of the 'contain' relation, we carry out the transitive closure. The closure properties associated with a concept is indicated in the relation metadata, as indicated earlier.

The extension of a Concept object is then incorporated into a Valuation object.

Persistence

The functions val_dump and val_load are provided to allow a valuation to be stored in a persistent database and re-loaded, rather than having to be re-computed each time.

Individuals and Lexical Items

As well as deriving relations from the Chat-80 data, we also create a set of individual constants, one for each entity in the domain. The individual constants are string-identical to the entities. For example, given a data item such as 'zloty', we add to the valuation a pair ('zloty', 'zloty'). In order to parse English sentences that refer to these entities, we also create a lexical item such as the following for each individual constant:

  PropN[num=sg, sem=<\P.(P zloty)>] -> 'Zloty'

The set of rules is written to the file chat_pnames.cfg in the current directory.

Classes [hide private]
  Concept
A Concept class, loosely based on SKOS (http://www.w3.org/TR/swbp-skos-core-guide/).
Functions [hide private]
list
clause2concepts(filename, rel_name, closures, schema)
Convert a file of Prolog clauses into a list of Concept objects.
source code
 
_str2records(filename, rel)
Read a file into memory and convert each relation clause into a list.
source code
Concept
unary_concept(label, subj, records)
Make a unary concept out of the primary key in a record.
source code
Concept
binary_concept(label, closures, subj, obj, records)
Make a binary concept out of the primary key and another field in a record.
source code
dict
process_bundle(rels)
Given a list of relation metadata bundles, make a corresponding dictionary of concepts, indexed by the relation name.
source code
list or a Valuation
make_valuation(concepts, read=False, lexicon=False)
Convert a list of Concepts into a list of (label, extension) pairs; optionally create a Valuation object.
source code
 
val_dump(rels, db)
Make a Valuation from a list of relation metadata bundles and dump to persistent database.
source code
 
val_load(db)
Load a Valuation from a persistent database.
source code
bool
alpha(str)
Utility to filter out non-alphabetic constants.
source code
Valuation
label_indivs(valuation, lexicon=False)
Assign individual constants to the individuals in the domain of a Valuation.
source code
list
make_lex(symbols)
Create lexical CFG rules for each individual symbol.
source code
list
concepts(items=('borders', 'circle_of_lat', 'circle_of_long', 'city', 'contai...)
Build a list of concepts corresponding to the relation names in items.
source code
 
main() source code
Variables [hide private]
  borders = {'closures': ['symmetric'], 'filename': 'borders.pl'...
  contains = {'closures': ['transitive'], 'filename': 'contain.p...
  city = {'closures': [], 'filename': 'cities.pl', 'rel_name': '...
  country = {'closures': [], 'filename': 'countries.pl', 'rel_na...
  circle_of_lat = {'closures': [], 'filename': 'world1.pl', 'rel...
  circle_of_long = {'closures': [], 'filename': 'world1.pl', 're...
  continent = {'closures': [], 'filename': 'world1.pl', 'rel_nam...
  region = {'closures': [], 'filename': 'world1.pl', 'rel_name':...
  ocean = {'closures': [], 'filename': 'world1.pl', 'rel_name': ...
  sea = {'closures': [], 'filename': 'world1.pl', 'rel_name': 's...
  items = ('borders', 'circle_of_lat', 'circle_of_long', 'city',...
  item_metadata = {'borders': {'closures': ['symmetric'], 'filen...
  rels = [{'closures': [], 'filename': 'cities.pl', 'rel_name': ...
  not_unary = ['borders.pl', 'contain.pl']
Function Details [hide private]

clause2concepts(filename, rel_name, closures, schema)

source code 

Convert a file of Prolog clauses into a list of Concept objects.

Parameters:
  • filename (string) - filename containing the relations
  • rel_name (string) - name of the relation
  • schema (list) - the schema used in a set of relational tuples
Returns: list
a list of Concepts

unary_concept(label, subj, records)

source code 

Make a unary concept out of the primary key in a record.

A record is a list of entities in some relation, such as ['france', 'paris'], where 'france' is acting as the primary key.

Parameters:
  • label (string) - the preferred label for the concept
  • subj (int) - position in the record of the subject of the predicate
  • records (list of lists) - a list of records
Returns: Concept
Concept of arity 1

binary_concept(label, closures, subj, obj, records)

source code 

Make a binary concept out of the primary key and another field in a record.

A record is a list of entities in some relation, such as ['france', 'paris'], where 'france' is acting as the primary key, and 'paris' stands in the 'capital_of' relation to 'france'.

More generally, given a record such as ['a', 'b', 'c'], where label is bound to 'B', and obj bound to 1, the derived binary concept will have label 'B_of', and its extension will be a set of pairs such as ('a', 'b').

Parameters:
  • label (string) - the base part of the preferred label for the concept
  • closures (list) - closure properties for the extension of the concept
  • subj (int) - position in the record of the subject of the predicate
  • obj (int) - position in the record of the object of the predicate
  • records (list of lists) - a list of records
Returns: Concept
Concept of arity 2

process_bundle(rels)

source code 

Given a list of relation metadata bundles, make a corresponding dictionary of concepts, indexed by the relation name.

Parameters:
  • rels (list of dictionaries) - bundle of metadata needed for constructing a concept
Returns: dict
a dictionary of concepts, indexed by the relation name.

make_valuation(concepts, read=False, lexicon=False)

source code 

Convert a list of Concepts into a list of (label, extension) pairs; optionally create a Valuation object.

Parameters:
  • concepts (list of Concepts) - concepts
  • read (bool) - if True, (symbol, set) pairs are read into a Valuation
Returns: list or a Valuation

val_dump(rels, db)

source code 

Make a Valuation from a list of relation metadata bundles and dump to persistent database.

Parameters:
  • rels (list of dictionaries) - bundle of metadata needed for constructing a concept
  • db (string) - name of file to which data is written. The suffix '.db' will be automatically appended.

val_load(db)

source code 

Load a Valuation from a persistent database.

Parameters:
  • db (string) - name of file from which data is read. The suffix '.db' should be omitted from the name.

alpha(str)

source code 

Utility to filter out non-alphabetic constants.

Parameters:
  • str (string) - candidate constant
Returns: bool

label_indivs(valuation, lexicon=False)

source code 

Assign individual constants to the individuals in the domain of a Valuation.

Given a valuation with an entry of the form {'rel': {'a': True}}, add a new entry {'a': 'a'}.

Parameters:
Returns: Valuation

make_lex(symbols)

source code 

Create lexical CFG rules for each individual symbol.

Given a valuation with an entry of the form {'zloty': 'zloty'}, create a lexical rule for the proper name 'Zloty'.

Parameters:
  • symbols (sequence) - a list of individual constants in the semantic representation
Returns: list

concepts(items=('borders', 'circle_of_lat', 'circle_of_long', 'city', 'contai...)

source code 

Build a list of concepts corresponding to the relation names in items.

Parameters:
  • items (list of strings) - names of the Chat-80 relations to extract
Returns: list
the Concepts which are extracted from the relations

Variables Details [hide private]

borders

Value:
{'closures': ['symmetric'],
 'filename': 'borders.pl',
 'rel_name': 'borders',
 'schema': ['region', 'border']}

contains

Value:
{'closures': ['transitive'],
 'filename': 'contain.pl',
 'rel_name': 'contains0',
 'schema': ['region', 'contain']}

city

Value:
{'closures': [],
 'filename': 'cities.pl',
 'rel_name': 'city',
 'schema': ['city', 'country', 'population']}

country

Value:
{'closures': [],
 'filename': 'countries.pl',
 'rel_name': 'country',
 'schema': ['country',
            'region',
            'latitude',
            'longitude',
            'area',
...

circle_of_lat

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'circle_of_latitude',
 'schema': ['circle_of_latitude', 'degrees']}

circle_of_long

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'circle_of_longitude',
 'schema': ['circle_of_longitude', 'degrees']}

continent

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'continent',
 'schema': ['continent']}

region

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'in_continent',
 'schema': ['region', 'continent']}

ocean

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'ocean',
 'schema': ['ocean']}

sea

Value:
{'closures': [],
 'filename': 'world1.pl',
 'rel_name': 'sea',
 'schema': ['sea']}

items

Value:
('borders',
 'circle_of_lat',
 'circle_of_long',
 'city',
 'contains',
 'continent',
 'country',
 'ocean',
...

item_metadata

Value:
{'borders': {'closures': ['symmetric'],
             'filename': 'borders.pl',
             'rel_name': 'borders',
             'schema': ['region', 'border']},
 'circle_of_lat': {'closures': [],
                   'filename': 'world1.pl',
                   'rel_name': 'circle_of_latitude',
                   'schema': ['circle_of_latitude', 'degrees']},
...

rels

Value:
[{'closures': [],
  'filename': 'cities.pl',
  'rel_name': 'city',
  'schema': ['city', 'country', 'population']},
 {'closures': [],
  'filename': 'world1.pl',
  'rel_name': 'circle_of_latitude',
  'schema': ['circle_of_latitude', 'degrees']},
...