CSCI 8980 sect 2: Spoken Language Interfaces Regular day, time, location: T/Th 11:15 to 12:30, Akerman 319 Altern. day, time, location: W 10:00 to 11:15, EECS 5-212 REQUIREMENTS: - lead discussion of 2-3 papers - prepare & present a 12-15 page report of a programming project PROJECTS: - develop software to perform SLI-related task (e.g. phone recognition) *OR* write a literature review on a SLI-related area of your interest - encouraged to (but don't need to) use nlp lab DBN recognizer / random variable template library - encouraged to (but don't need to) use training/evaluation data from nlp lab corpora - perform an evaluation of your system - prepare roughly 15-page report (30 pages for literature review) - prepare roughly 15-minute presentation (sign up for time slots by e-mail) - group projects w. any number of members encouraged, but each member must have separate responsibilities, project report, and presentation. - reports are due by e-mail at end of final exam day: Dec 20 DISCUSSION: please volunteer for 1) presenter and 2) question-asker for each session. (-)RESCHEDULED/(+)ALTERNATE | SESSION DATE PRESENTER QUESTIONER | PAPER TOPIC ----------------------------------------------------------------------------------------------------------- Session 01, 05 Sep : William --- Intro, stat modeling / RV library tutorial Session 02, 07 Sep : Andy William Lee&Hon'89 (Basics, HMMs for phone recognition) ----------------------------------------------------------------------------------------------------------- Session 03, 12 Sep : Andy William Lee&Hon'89 (Basics, HMMs for phone recognition) Session 04, 14 Sep : Tim Zach Robinson'94 (Basics, RNNs for phone recognition) ----------------------------------------------------------------------------------------------------------- -RESCHEDULE- 19 Sep - -RESCHEDULE- 21 Sep - ----------------------------------------------------------------------------------------------------------- Session 05, 26 Sep : Amer Dingcheng Jelinek'76 (Basics, hierarchic HMMs for word recognition) Session 06, 28 Sep : Amer Dingcheng Jelinek'76 (Basics, hierarchic HMMs for word recognition) ----------------------------------------------------------------------------------------------------------- Session 07, 03 Oct : Dingcheng Tim Murphy&Paskin'01 (Basics, hierarchic HMMs) -RESCHEDULE- 05 Oct - ----------------------------------------------------------------------------------------------------------- Session 08, 10 Oct : Lane Jiaping Chappelier&al'99 (Basics, Uncertain syntax + semantics) (ALT) 09, 11 Oct : Lane (#2) Jiaping + Miller&al'96 (Basics, Uncertain syntax + semantics) Session 10, 12 Oct : William Schuler&Miller'05 (Basics, Incremental syntax) ----------------------------------------------------------------------------------------------------------- Session 11, 17 Oct : Tim (#2) Amer Heeman&Allen'99 (Problems with syntax: Speech repairs) -RESCHEDULE- 19 Oct - ----------------------------------------------------------------------------------------------------------- Session 12, 24 Oct : Tim (#2) Amer Heeman&Allen'99 (Problems with syntax: Speech repairs) -RESCHEDULE- 26 Oct - ----------------------------------------------------------------------------------------------------------- Session 13, 31 Oct : Zac Stoness'01 (Need for integrated semantics) (ALT) 14, 01 Nov : Jiaping + Stent&al'99 (Integrated semantics) Session 15, 02 Nov : Stephen Roy&Mukherjee'05 (Integrated semantics) ----------------------------------------------------------------------------------------------------------- Session 16, 07 Nov : Andy (#2) Gorniak&Roy'04 (Integrated semantics) (ALT) 17, 08 Nov : Andy (#2) + Gorniak&Roy'04 (Integrated semantics) Session 18, 09 Nov : William pokey semantics (Integrated semantics) ----------------------------------------------------------------------------------------------------------- Session 19, 14 Nov : William pokey semantics (Integrated semantics) (ALT) 20, 15 Nov : Dingcheng (#2) + Allen&Ferguson'94 (Semantic representation) Session 21, 16 Nov : Jiaping (#2) Siskind'01 (Semantic representation) ----------------------------------------------------------------------------------------------------------- Session 22, 21 Nov : Jiaping (#2) Siskind'01 (Semantic representation) --HOLIDAY-- 23 Nov : (THANKSGIVING BREAK) ----------------------------------------------------------------------------------------------------------- Session 23, 28 Nov : Amer (#2) Allen&al'96 trains (Conversational interfaces, adding discourse) (ALT) 24, 29 Nov : Jack + Rich&al'98 collagen(Conversational interfaces, adding discourse) Session 25, 30 Nov : Zac (#2) Rayner&al'00 NASA (Conversational interfaces, adding discourse) ----------------------------------------------------------------------------------------------------------- Session 26, 04 Dec : Jack (#2) Oviatt&al'99 (Multi-modal interfaces) Session 27, 06 Dec : Stephen (#2) Chung&al'04 (Dynamic vocabulary) ----------------------------------------------------------------------------------------------------------- Session 28, 11 Dec : Andy, Tim, Stephen, Amer (student presentations) Session 29, 13 Dec : Jack, Zac, Jiaping, Dingcheng, Lane (student presentations) ----------------------------------------------------------------------------------------------------------- Lee&Hon'89: Speaker-Independent Phone Recognition Using Hidden Markov Models. Lee, K.F., Hon, H.W. IEEE Trans. Acoustic Speech and Signal Processing 37(11). webloria.loria.fr/~barreaud/ESIALRecherche/Lee89.pdf Robinson'94: An application of recurrent nets to phone probability estimation. Anthony J. Robinson. IEEE Transactions on Neural Networks 5(2). http://citeseer.ist.psu.edu/robinson94application.html Jelinek'76: Continuous Speech Recognition by Statistical Methods. Frederick Jelinek. Proceedings of the IEEE 64(4):532-556. (www.lib.umn.edu; journal catalog, "Proceedings of the IEEE", requires UofM login) Murphy&Paskin'01: Linear time inference in hierarchical HMMs. Kevin P. Murphy and Mark A. Paskin. NIPS'01 http://citeseer.ist.psu.edu/murphy01linear.html Chappelier&al'99: Lattice parsing for speech recognition. J. Chappelier and M. Rajman and R. Aragues and A. Rozenknop http://citeseer.ist.psu.edu/chappelier99lattice.html Miller&al'96: A Fully Statistical Approach to Natural Language Interfaces. Scott Miller and David Stallard and Robert Bobrow and Richard Schwartz. ACL'96. http://citeseer.ist.psu.edu/miller96fully.html Chelba&Jelinek'98: Exploiting syntactic structure for language modeling. Ciprian Chelba and Frederick Jelinek. ACL'98. http://citeseer.ist.psu.edu/chelba98exploiting.html Schuler&Miller'05: Integrating Denotational Meaning into a DBN Language Model. William Schuler and Tim Miller. Interspeech'05. http://www-users.cs.umn.edu/~schuler/isp.pdf Heeman&Allen'99: Speech Repairs, Intonational Phrases and Discourse Markers: Modeling Speakers' Utterances in Spoken Dialog. Peter Heeman and James Allen. Computational Linguistics, Vol. 25-4, 1999. www.cs.rochester.edu/u/james/CL99.pdf Stoness'01: Continuous Understanding: A First Look at CAFE. Scott Stoness. Technical Report, University of Rochester. 2001. citeseer.ist.psu.edu/stoness01continuous.html Stent&al'99: The CommandTalk Spoken Dialogue System. Amanda Stent, John Dowding, Jean Mark Gawron, Elizabeth Owen Bratt, Robert Moore. ACL'99. citeseer.ist.psu.edu/stent99commandtalk.html Roy&Mukherjee'05: Towards Situated Speech Understanding: Visual Context Priming of Language Models. Deb Roy and Niloy Mukherjee. Computer Speech and Language, 19(2), pages 227-24. web.media.mit.edu/~dkroy/papers/pdf/roy_niloy_2005.pdf Gorniak&Roy'04: Grounded Semantic Composition for Visual Scenes. Peter Gorniak and Deb Roy. Journal of Artificial Intelligence Research, Volume 21, pages 429-470. www.media.mit.edu/cogmac/publications/bishop.pdf Allen&Ferguson'94: Actions and Events in Interval Temporal Logic. James Allen and George Ferguson. Journal of Logic and Computation 4(5), 1994. citeseer.ist.psu.edu/allen94actions.html Siskind'01: Grounding the Lexical Semantics of Verbs in Visual Perception Using Force Dynamics and Event Logic. Jeffrey Mark Siskind. Journal of Artificial Intelligence Research (JAIR), 15:31-90, August 2001. www.cs.cmu.edu/afs/cs/project/jair/pub/volume15/siskind01a.ps.Z Allen&al'96: Robust Understanding in a Dialogue System. James F. Allen, Bradford W. Miller, Eric K. Ringger, Teresa Sikorski. ACL'96. http://citeseer.ist.psu.edu/allen96robust.html Rich&al'98: COLLAGEN: A Collaboration Manager for Software Interface Agents citeseer.ist.psu.edu/rich98collagen.html Rayner&al'00: A Compact Architecture for Dialogue Management Based on Scripts and Meta-Outputs http://citeseer.ist.psu.edu/rayner00compact.html Oviatt&al'99: Ten Myths of Multimodal Interaction citeseer.ist.psu.edu/oviatt99ten.html Chung&al'04: A Dynamic Vocabulary Spoken Dialogue Interface citeseer.ist.psu.edu/chung04dynamic.html [skipped: collagen problems (Conversational interfaces, adding discourse), Johnston&al'02 (Multi-modal interfaces), oviatt stamp (Multi-modal interfaces)] Johnston&al'02: citeseer.ist.psu.edu/johnston02match.html