Module chat80
source code
Overview
Chat-80 was a natural language system which allowed the user to
interrogate a Prolog knowledge base in the domain of world geography.
It was developed in the early '80s by Warren and Pereira; see http://acl.ldc.upenn.edu/J/J82/J82-3002.pdf for a
description and http://www.cis.upenn.edu/~pereira/oldies.html for the
source files.
This module contains functions to extract data from the Chat-80
relation files ('the world database'), and convert then into a format
that can be incorporated in the FOL models of nltk.sem.evaluate. The code assumes that the Prolog
input files are available in the NLTK corpora directory.
The Chat-80 World Database consists of the following files:
world0.pl
rivers.pl
cities.pl
countries.pl
contain.pl
borders.pl
This module uses a slightly modified version of
world0.pl
, in which a set of Prolog rules have been
omitted. The modified file is named world1.pl
. Currently,
the file rivers.pl
is not read in, since it uses a list
rather than a string in the second field.
Reading Chat-80 Files
Chat-80 relations are like tables in a relational database. The
relation acts as the name of the table; the first argument acts as the
'primary key'; and subsequent arguments are further fields in the
table. In general, the name of the table provides a label for a unary
predicate whose extension is all the primary keys. For example,
relations in cities.pl
are of the following form:
'city(athens,greece,1368).'
Here, 'athens'
is the key, and will be mapped to a
member of the unary predicate city.
The fields in the table are mapped to binary predicates. The first
argument of the predicate is the primary key, while the second argument
is the data in the relevant field. Thus, in the above example, the
third field is mapped to the binary predicate population_of, whose extension is a set of pairs such
as '(athens, 1368)'
.
An exception to this general framework is required by the relations
in the files borders.pl
and contains.pl
.
These contain facts of the following form:
'borders(albania,greece).'
'contains0(africa,central_africa).'
We do not want to form a unary concept out the element in the first
field of these records, and we want the label of the binary relation
just to be 'border'
/'contain'
respectively.
In order to drive the extraction process, we use 'relation metadata
bundles' which are Python dictionaries such as the following:
city = {'label': 'city',
'closures': [],
'schema': ['city', 'country', 'population'],
'filename': 'cities.pl'}
According to this, the file city['filename']
contains a
list of relational tuples (or more accurately, the corresponding
strings in Prolog form) whose predicate symbol is
city['label']
and whose relational schema is
city['schema']
. The notion of a closure
is
discussed in the next section.
Concepts
In order to encapsulate the results of the extraction, a class of Concepts
is introduced. A Concept object has a number of attributes, in
particular a prefLabel
and extension
, which
make it easier to inspect the output of the extraction. In addition,
the extension
can be further processed: in the case of the
'border'
relation, we check that the relation is
symmetric, and in the case of the 'contain'
relation, we carry out the transitive closure. The closure
properties associated with a concept is indicated in the relation
metadata, as indicated earlier.
The extension
of a Concept
object is then incorporated into a Valuation object.
Persistence
The functions val_dump and val_load are provided to allow a valuation to be
stored in a persistent database and re-loaded, rather than having to be
re-computed each time.
Individuals and Lexical Items
As well as deriving relations from the Chat-80 data, we also create
a set of individual constants, one for each entity in the domain. The
individual constants are string-identical to the entities. For example,
given a data item such as 'zloty'
, we add to the valuation
a pair ('zloty', 'zloty')
. In order to parse English
sentences that refer to these entities, we also create a lexical item
such as the following for each individual constant:
PropN[num=sg, sem=<\P.(P zloty)>] -> 'Zloty'
The set of rules is written to the file chat_pnames.cfg
in the current directory.
list
|
|
|
_str2records(filename,
rel)
Read a file into memory and convert each relation clause into a list. |
source code
|
|
Concept
|
|
Concept
|
binary_concept(label,
closures,
subj,
obj,
records)
Make a binary concept out of the primary key and another field in a
record. |
source code
|
|
dict
|
process_bundle(rels)
Given a list of relation metadata bundles, make a corresponding
dictionary of concepts, indexed by the relation name. |
source code
|
|
list or a Valuation
|
make_valuation(concepts,
read=False,
lexicon=False)
Convert a list of Concept s into a list of (label,
extension) pairs; optionally create a Valuation object. |
source code
|
|
|
|
|
|
bool
|
|
Valuation
|
label_indivs(valuation,
lexicon=False)
Assign individual constants to the individuals in the domain of a
Valuation . |
source code
|
|
list
|
|
list
|
concepts(items=( ' borders ' , ' circle_of_lat ' , ' circle_of_long ' , ' city ' , ' contai ... )
Build a list of concepts corresponding to the relation names in
items . |
source code
|
|
|
|
|
borders = { ' closures ' : [ ' symmetric ' ] , ' filename ' : ' borders.pl ' ...
|
|
contains = { ' closures ' : [ ' transitive ' ] , ' filename ' : ' contain.p ...
|
|
city = { ' closures ' : [ ] , ' filename ' : ' cities.pl ' , ' rel_name ' : ' ...
|
|
country = { ' closures ' : [ ] , ' filename ' : ' countries.pl ' , ' rel_na ...
|
|
circle_of_lat = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' rel ...
|
|
circle_of_long = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' re ...
|
|
continent = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' rel_nam ...
|
|
region = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' rel_name ' : ...
|
|
ocean = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' rel_name ' : ...
|
|
sea = { ' closures ' : [ ] , ' filename ' : ' world1.pl ' , ' rel_name ' : ' s ...
|
|
items = ( ' borders ' , ' circle_of_lat ' , ' circle_of_long ' , ' city ' , ...
|
|
item_metadata = { ' borders ' : { ' closures ' : [ ' symmetric ' ] , ' filen ...
|
|
rels = [ { ' closures ' : [ ] , ' filename ' : ' cities.pl ' , ' rel_name ' : ...
|
|
not_unary = [ ' borders.pl ' , ' contain.pl ' ]
|
clause2concepts(filename,
rel_name,
closures,
schema)
| source code
|
Convert a file of Prolog clauses into a list of Concept
objects.
- Parameters:
filename (string) - filename containing the relations
rel_name (string) - name of the relation
schema (list) - the schema used in a set of relational tuples
- Returns: list
- a list of Concepts
|
Make a unary concept out of the primary key in a record.
A record is a list of entities in some relation, such as
['france', 'paris'] , where 'france' is acting
as the primary key.
- Parameters:
label (string) - the preferred label for the concept
subj (int) - position in the record of the subject of the predicate
records (list of lists) - a list of records
- Returns: Concept
- Concept of arity 1
|
binary_concept(label,
closures,
subj,
obj,
records)
| source code
|
Make a binary concept out of the primary key and another field in a
record.
A record is a list of entities in some relation, such as
['france', 'paris'] , where 'france' is acting
as the primary key, and 'paris' stands in the
'capital_of' relation to 'france' .
More generally, given a record such as ['a', 'b', 'c'] ,
where label is bound to 'B' , and obj bound to
1, the derived binary concept will have label 'B_of' , and
its extension will be a set of pairs such as ('a', 'b') .
- Parameters:
label (string) - the base part of the preferred label for the concept
closures (list) - closure properties for the extension of the concept
subj (int) - position in the record of the subject of the predicate
obj (int) - position in the record of the object of the predicate
records (list of lists) - a list of records
- Returns: Concept
- Concept of arity 2
|
Given a list of relation metadata bundles, make a corresponding
dictionary of concepts, indexed by the relation name.
- Parameters:
rels (list of dictionaries) - bundle of metadata needed for constructing a concept
- Returns: dict
- a dictionary of concepts, indexed by the relation name.
|
make_valuation(concepts,
read=False,
lexicon=False)
| source code
|
Convert a list of Concept s into a list of (label,
extension) pairs; optionally create a Valuation object.
- Parameters:
concepts (list of Concepts) - concepts
read (bool) - if True , (symbol, set) pairs are read
into a Valuation
- Returns: list or a Valuation
|
Make a Valuation from a list of relation metadata bundles and
dump to persistent database.
- Parameters:
rels (list of dictionaries) - bundle of metadata needed for constructing a concept
db (string) - name of file to which data is written. The suffix '.db' will be
automatically appended.
|
Load a Valuation from a persistent database.
- Parameters:
db (string) - name of file from which data is read. The suffix '.db' should be
omitted from the name.
|
Utility to filter out non-alphabetic constants.
- Parameters:
str (string) - candidate constant
- Returns: bool
|
Assign individual constants to the individuals in the domain of a
Valuation .
Given a valuation with an entry of the form {'rel': {'a': True}}, add
a new entry {'a': 'a'}.
- Parameters:
- Returns: Valuation
|
Create lexical CFG rules for each individual symbol.
Given a valuation with an entry of the form {'zloty': 'zloty'}, create
a lexical rule for the proper name 'Zloty'.
- Parameters:
symbols (sequence) - a list of individual constants in the semantic representation
- Returns: list
|
concepts(items=( ' borders ' , ' circle_of_lat ' , ' circle_of_long ' , ' city ' , ' contai ... )
| source code
|
Build a list of concepts corresponding to the relation names in
items .
- Parameters:
items (list of strings) - names of the Chat-80 relations to extract
- Returns: list
- the Concepts which are extracted from the relations
|
borders
- Value:
{ ' closures ' : [ ' symmetric ' ] ,
' filename ' : ' borders.pl ' ,
' rel_name ' : ' borders ' ,
' schema ' : [ ' region ' , ' border ' ] }
|
|
contains
- Value:
{ ' closures ' : [ ' transitive ' ] ,
' filename ' : ' contain.pl ' ,
' rel_name ' : ' contains0 ' ,
' schema ' : [ ' region ' , ' contain ' ] }
|
|
city
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' cities.pl ' ,
' rel_name ' : ' city ' ,
' schema ' : [ ' city ' , ' country ' , ' population ' ] }
|
|
country
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' countries.pl ' ,
' rel_name ' : ' country ' ,
' schema ' : [ ' country ' ,
' region ' ,
' latitude ' ,
' longitude ' ,
' area ' ,
...
|
|
circle_of_lat
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' circle_of_latitude ' ,
' schema ' : [ ' circle_of_latitude ' , ' degrees ' ] }
|
|
circle_of_long
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' circle_of_longitude ' ,
' schema ' : [ ' circle_of_longitude ' , ' degrees ' ] }
|
|
continent
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' continent ' ,
' schema ' : [ ' continent ' ] }
|
|
region
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' in_continent ' ,
' schema ' : [ ' region ' , ' continent ' ] }
|
|
ocean
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' ocean ' ,
' schema ' : [ ' ocean ' ] }
|
|
sea
- Value:
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' sea ' ,
' schema ' : [ ' sea ' ] }
|
|
items
- Value:
( ' borders ' ,
' circle_of_lat ' ,
' circle_of_long ' ,
' city ' ,
' contains ' ,
' continent ' ,
' country ' ,
' ocean ' ,
...
|
|
item_metadata
- Value:
{ ' borders ' : { ' closures ' : [ ' symmetric ' ] ,
' filename ' : ' borders.pl ' ,
' rel_name ' : ' borders ' ,
' schema ' : [ ' region ' , ' border ' ] } ,
' circle_of_lat ' : { ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' circle_of_latitude ' ,
' schema ' : [ ' circle_of_latitude ' , ' degrees ' ] } ,
...
|
|
rels
- Value:
[ { ' closures ' : [ ] ,
' filename ' : ' cities.pl ' ,
' rel_name ' : ' city ' ,
' schema ' : [ ' city ' , ' country ' , ' population ' ] } ,
{ ' closures ' : [ ] ,
' filename ' : ' world1.pl ' ,
' rel_name ' : ' circle_of_latitude ' ,
' schema ' : [ ' circle_of_latitude ' , ' degrees ' ] } ,
...
|
|