UChicago CompLing Lab

Projects

Members of our lab are engaged in a range of projects aimed at better understanding the linguistic competence and nature of sensitivities of pre-trained language models, studying and improving competence with compositional meaning understanding in NLP models, as well as using computational models to test hypotheses about the mechanisms underlying language processing in humans.


Controlled evaluation of compositional meaning understanding in pre-trained LMs

In this body of work our priority is to establish reliable and robust evaluations for the compositional meaning competence of pre-trained language models. This work employs controlled tests that tease apart compositional meaning understanding from more superficial predictive behaviors, and often draws on methodology from psycholinguistics and cognitive neuroscience.

Example papers:
~~~~~~~~~~~~~~~~~

Assessment of other linguistic competence and sensitivity in pre-trained LMs

Beyond assessment of compositional meaning understanding per se, our group has investigated a wide range of linguistic competences, behavioral sensitivities, and encoding patterns in pre-trained language models. Topics of interest include conceptual property knowledge and induction, discourse and pragmatic competence, response to syntactic anomaly, sensitivity to presence of "prime words", and encoding in contextual embeddings of different types of linguistic information from surrounding words.

Example papers:

Property knowledge and inductive reasoning Pragmatic competence Assorted dimensions of model linguistic sensitivity and representation
~~~~~~~~~~~~~~~~~

Computational modeling of real-time language processing in humans

In this body of work we design computational models to test hypotheses about the cognitive mechanisms driving the robust and rapid language processing in the human brain. We draw selectively on probabilistic and representational measures from pre-trained language models to increase stimulus-level statistical sensitivities of our cognitive models.

Example papers:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~