Seminar in Computational Linguistics

Abstract

 

Despite   their  practical   success   and  impressive   performances, neural-network-based and  distributed semantics techniques  have often been criticized as  they remain fundamentally opaque  and difficult to interpret.   Several  recent  pieces  of work  have  investigated  the linguistic abilities  of these  representations, and whether  they can capture long agreement and thus  hierarchical notions.  Results are at present inconclusive.  In  this vein, we study  another core, defining and more  challenging property  of language:  the ability  to construe long-distance dependencies.   Human languages  exhibit the  ability to interpret discontinuous elements distant from each other in the string as if they  were adjacent.  This ability is blocked  if a similar, but extraneous, element  intervenes between the  discontinuous components. We present results  that show that word embeddings  and the similarity spaces they  define do  not unequivocally correlate  with experimental results  on  intervention  similarity  in  long-distance  dependencies narrowly defined,  even if translated  in more 'syntactic'  portion of the distributional space or tested in prediction tasks.  These results show that the linguistic  encoding in distributed representations does not appear to be human-like, and it also brings evidence to the debate on narrow  or broad definitions  of similarity in syntax  and sentence processing.

 

Bio

 

Paola Merlo  is associate professor  in the Linguistics  department of the University  of Geneva.  She  is the head of  the interdisciplinary research  group Computational  Learning and  Computational Linguistics (CLCL).   The  group  is  concerned  with  interdisciplinary  research combining  linguistic  modelling  with  machine  learning  techniques. Prof.  Merlo  has been editor of  Computational Linguistics, published by  MIT  Press  and  a  member  of  the  executive  committee  of  the ACL. Prof. Merlo  holds a  doctorate in  Computational Linguistics  from the University of Maryland,  USA.  She has been  associate research fellow at the  University of Pennsylvania,  and visiting scholar  at Rutgers, Edinburgh, and Stanford.