Linguistics 884: Seminar in Vectorial Semantics

Natural languages associate meaning with recognized words at a range of granularities, from precise descriptions of familiar objects and events, to vague representations of new ideas, often within the same sentence. Traditionally these have been modeled in very different ways: using formal logic for precise meanings, dependency relations among head words for vague meanings, and mixtures of learned `sub-symbolic' features for gists and sentiments, which are vaguer still. This course will explore how these variously deep and shallow representations may be brought together using vectorial (or linear algebraic) models.

The course will focus on theory, occasionally dipping into engineering issues involved in implementing these models.

Instructor: William Schuler

Office: 209 Oxley Hall
Email: (my last name)@ling.osu.edu
Office hours: Thursdays 2:00-3:00, or by appointment (just email me)

Meeting time: Tuesday and Thursday 11:30-1:18pm in 213 Denney Hall (DE)

Web site: http://www.ling.ohio-state.edu/~schuler/courses/884. The updated syllabus, assignments, slides, etc. will be posted here, so check it regularly.

Topics:

Discovering the `gist' of a document in meaning space:
- Indexing by Latent Semantic Analysis ( Deerwester et al., 1990, Journal of the American Society for Information Science 41);
- A solution to Plato's problem: The latent semantic analysis theory of acquisition ( Landauer and Dumais, 1997, Psychological Review 104(2)211-240)
Helpful link: Hal Daume notes/code for conjugate gradient
Probabilistic gist models --- meaning as mixtures of archetypes:
- Probabilistic Latent Semantic Indexing ( Hoffmann, 1999, SIGIR);
- Latent Dirichlet Allocation ( Blei, Ng, and Jordan, 2003, Journal of Machine Learning Research 3 993-1022)
- Finding Scientific Topics ( Griffiths and Steyvers, 2004, Proceedings of the National Academy of Sciences 101 5228-5235)
- helpful links: Bayesian inference with tears, Gibbs sampling for the uninitiated.
Adding relational information to gist models: Relational Clustering
- Probabilistic Classification and Clustering in Relational Data ( Taskar, Segal, and Koller, 2001, IJCAI)
Infinite mixture models --- discovering unlimited numbers of archetypes: Chinese Restaurant Processes
- Hierarchical Topic Models and the Nested Chinese Restaurant Process ( Blei, Griffiths, Jordan, and Tenenbaum, 2003, NIPS 16)
- Hierarchical Dirichlet Processes ( Teh, Jordan, Beal, and Blei, 2005, Journal of the American Statistical Association)
Adding syntactic structure --- roles as relations over vectors: Kintsch and Mangalath
- The Construction of Meaning ( Kintsch and Mangalath, in press, Topics in Cognitive Science)
- this paper (under review)
Logical interpretation using structured vectorial semantics:
- this paper (under review)
5/23 (Andrew): The Role of Meaning in Word Recognition ( Balota, 1990, Comprehension Processes in Reading 9-32)
5/25 (Dominic): A Memory-based Theory of Verbal Cognition ( Dennis, 2005, Cognitive Science 29(2) 145--193)
6/1 (Michelle): The Representation of Polysemous Words ( Klein and Murphy, 2001, Journal of Memory and Language, 45, 259--282)
6/3 (Stephen): Probabilistic CFG with latent annotations ( Matsuzaki, Miyao, and Tsujii, 2005, Proc. ACL )

Course prerequisites:

Ling 684.01 or CSE 730 or CSE 732
Ling 684.02 is not a prerequisite, but it will help.

Students with Disabilities:

Students who need an accommodation based on the impact of a disability should contact me to arrange an appointment as soon as possible to discuss the course format, to anticipate needs, and to explore potential accommodations. I rely on the Office of Disability Services for assistance in verifying the need for accommodations and developing accommodation strategies. Students who have not previously contacted the Office for Disability Services are encouraged to do so (292-3307; http://www.ods.ohio-state.edu).

Academic Misconduct:

You must do your homeworks, programming assignments, and examinations yourself, ON YOUR OWN. Copying another's work, or allowing (even negligently) others to copy your work, or possession of electronic computing devices in the testing area, is cheating and grounds for penalties.

Academic dishonesty is not allowed and will be reported to the University Committee on Academic Misconduct.