See also: Bled, James Cussens, ICML'99, ILP'99, LLL

Learning Language in Logic (LLL) workshop

Bled, Slowenia, 30 June 1999

by James Cussens, University of York

The Learning Language in Logic (LLL) workshop took place on 30 June 1999 in Bled, Slovenia immediately after the Ninth International Workshop on Inductive Logic Programming (ILP'99) and the Sixteenth International Conference on Machine Learning (ICML'99). LLL is a growing research area lying at the intersection of computational linguistics, machine learning and computational logic. As such it is of interest to all those working in these three fields. I am pleased to say that the workshop attracted submissions from both the Natural Language Processing (NLP) community and the ILP community, reflecting the essentially multi-disciplinary nature of LLL. In addition the three invited speakers: Eric Brill, Ted Briscoe and Ray Mooney are well-known for work that brings together machine learning and NLP.

Eric Brill's talk Toward linguistically sophisticated models of language described using human linguistic proficiency as a tool for improving machine learning of natural language. People are observed postprocessing the output of an NLP system, with the idea that the linguistic cues they use are the ones we should include to improve our natural language models. Ted Briscoe looked at Bayesian learning of (stochastic) grammars. Arguing that current formulations of grammar learning in both theoretical linguistics and computational linguistics are inadequate; Ted put forward a parameter setting model linked to a constraint-based grammatical representation language. This model does not make the unrealistic assumptions of standard linguistic models such as assuming data is drawn from a stationary distribution or that a category set is available to the learner as background knowledge. Ray Mooney delivered a polemic: Learning for semantic
interpretation: Scaling up without dumbing down addressed to those applying ILP to NLP. Ray argued that far too much attention was being devoted to `low-level' tasks such as morphology and part-of-speech tagging. In fact, ILP has the most to offer NLP in learning for semantic interpretation - an area where Ray's group in Texas has done much work.

Grammar induction was a central topic of the workshop with four of the ten submitted papers in this area. Watkinson and Manandhar's paper Unsupervised lexical learning with categorial grammars using the LLL corpus presented a learner which, provided with a set of possible lexical CG categories and unannotated sentences is able to learn a reasonable lexicon. In CG induction using MDL and parsed corpora, Osborne described a Minimum Description Length based method to automatically extend an existing parameterised DCG using (i) raw text and (ii) a parsed corpus. James Cussens presented work with Stephen Pulman on their Experiments in inductive
chart parsing. Here `missing' grammar rules are found using ILP within a chart parsing framework - a central aim being to exploit the ability of chart parsing to efficiently determine what is needed to complete a parse. Adriaans and Haas approached the problem of grammar induction from a higher level in Grammar induction as substructural inductive logic programming, where they propose basing ILP approaches to grammar learning on substructural logic, rather than on the usual Horn logic. This has the advantage that substructural logic has just enough expressive power for grammars and grammars are represented ``intrinsically'' in substructural logic.

Part-of-speech tagging is a popular problem for those applying ILP to NLP (despite the view of Mooney!). At LLL, we had talks on Iterative part-of-speech tagging by Alí\'{\i}pio Jorge and Alneu de Andrade Lopes and on Towards disambiguation in Czech corpora by Lubo\v{s} Popel\'{\i}nsk\'{y}, Tom\'{a}\v{s} Pavelek and Tom\'{a}\v{s} Pt\'{a}\v{c}n\'{\i}k. In the former, the iterative approach is used to build up a recursive logic program which predicts tags based on the tags of surrounding words, as well as the words themselves. In the latter, Progol was used to learn disambiguation rules for Czech.

Claire N\'{e}dellec talked on Corpus-based learning of semantic relations by the ILP system, Asium. Asium is an ILP system (with, unusually for ILP, a rather nice GUI) which learns ontologies and verb subcategorization frames from parsed texts in specific domains. Luc Dehaspe presented joint work with Maarten Forrier on Transformation-based learning meets frequent pattern discovery. In true K.U. Leuven style, this involved upgrading an non-ILP ML technique (Brill's transformation-based learning) to a first order representation. Henrik Bostr\"{o}m examined techniques for the Induction of recursive transfer rules. Transfer rules translate between quasi-logical forms for different natural languages, here French and English. Such rule sets represent a challenge for ILP, since they are highly recursive and more than one rule may need to be produced from one example. Finally, we have the paper by Markus Junker, Michael Sintek and Matthias Rinck on Learning for text categorization and information extraction with ILP. Markus Junker presented an ILP learning algorithm using types for text, words and text positions which learned rules for text categorization and information extraction.


Natural language learning based on statistical approaches (e.g. n-gram language modelling) has proved successful, but it is well known that such linguistically impoverished approaches have severe limitations. The flexibility and expressivity of logical representations make them highly suitable for natural language analysis. I hope that the papers presented at the LLL workshop show that logical and inductive approaches to NLP are not necessarily competitors, and that they will encourage more LLL research. The LLL workshop itself was an enjoyable and stimulating event, so much so that there are plans for LLL'2000. This is likely to be co-located with the Fifth International Colloquium on Grammatical Inference (ICGI'2000) in Lisbon during 5-7 June 2000.

A LLL home page has been created at which provides an online bibliography and pointers to papers, datasets and algorithms. The workshop notes are available online at .

I would like to thank the programme committee for their help in reviewing submissions, those who submitted and presented papers, and also the three invited speakers. Thanks also to Sa\v{s}o D\v{z}eroski, the local chair, who organised everything so efficiently at the Slovene end. Finally many thanks to the ILP2 project, MLNet-II, ILPNet2 and CompulogNet for their generous financial support for the workshop.