2024

1. Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models

   Mehrdad Farahani, and Richard Johansson

   In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

   Abs PDF

   Generative language models often struggle with specialized or less-discussed knowledge. A potential solution is found in Retrieval-Augmented Generation (RAG) models which act like retrieving information before generating responses. In this study, we explore how the Atlas approach, a RAG model, decides between what it already knows (parametric) and what it retrieves (non-parametric). We use causal mediation analysis and controlled experiments to examine how internal representations influence information processing. Our findings disentangle the effects of parametric knowledge and the retrieved context. They indicate that in cases where the model can choose between both types of information (parametric and non-parametric), it relies more on the context than the parametric knowledge. Furthermore, the analysis investigates the computations involved in \textithow the model uses the information from the context. We find that multiple mechanisms are active within the model and can be detected with mediation analysis: first, the decision of \textitwhether the context is relevant, and second, how the encoder computes output representations to support copying when relevant.

2. Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion

   Denitsa Saynova, Lovisa Hagström, Moa Johansson, Richard Johansson, and Marco Kuhlmann

   arXiv preprint arXiv:2410.14405, Nov 2024

   PDF
3. Can Large Language Models (or Humans) Disentangle Text?

   Nicolas Audinet Pieuchon, Adel Daoud, Connor Jerzak, Moa Johansson, and Richard Johansson

   In Proceedings of the 6th Workshop on Natural Language Processing and Computational Social Science (NLP+CSS), Nov 2024

   Abs PDF

   We investigate the potential of large language models (LLMs) to disentangle text variables–to remove the textual traces of an undesired forbidden variable in a task sometimes known as text distillation and closely related to the fairness in AI and causal inference literature. We employ a range of various LLM approaches in an attempt to disentangle text by identifying and removing information about a target variable while preserving other relevant signals. We show that in the strong test of removing sentiment, the statistical association between the processed text and sentiment is still detectable to machine learning classifiers post-LLM-disentanglement. Furthermore, we find that human annotators also struggle to disentangle sentiment while preserving other semantic content. This suggests there may be limited separability between concept variables in some text contexts, highlighting limitations of methods relying on text-level transformations and also raising questions about the robustness of disentanglement methods that achieve statistical independence in representation space.

4. What Happens to a Dataset Transformed by a Projection-based Concept Removal Method?

   Richard Johansson

   In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, Nov 2024

   Abs PDF

   We investigate the behavior of methods that use linear projections to remove information about a concept from a language representation, and we consider the question of what happens to a dataset transformed by such a method. A theoretical analysis and experiments on real-world and synthetic data show that these methods inject strong statistical dependencies into the transformed datasets. After applying such a method, the representation space is highly structured: in the transformed space, an instance tends to be located near instances of the opposite label. As a consequence, the original labeling can in some cases be reconstructed by applying an anti-clustering method.

2023

1. The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models

   Lovisa Hagström, Denitsa Saynova, Tobias Norlund, Moa Johansson, and Richard Johansson

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs PDF

   Large Language Models (LLMs) make natural interfaces to factual knowledge, but their usefulness is limited by their tendency to deliver inconsistent answers to semantically equivalent questions. For example, a model might supply the answer “Edinburgh” to “Anne Redpath passed away in X.” and “London” to “Anne Redpath’s life ended in X.” In this work, we identify potential causes of inconsistency and evaluate the effectiveness of two mitigation strategies: up-scaling and augmenting the LM with a passage retrieval database. Our results on the LLaMA and Atlas models show that both strategies reduce inconsistency but that retrieval augmentation is considerably more efficient. We further consider and disentangle the consistency contributions of different components of Atlas. For all LMs evaluated we find that syntactical form and task artifacts impact consistency. Taken together, our results provide a better understanding of the factors affecting the factual consistency of language models.

2. Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models

   Ehsan Doostmohammadi, Tobias Norlund, Marco Kuhlmann, and Richard Johansson

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Jul 2023

   Abs PDF

   Augmenting language models with a retrieval mechanism has been shown to significantly improve their performance while keeping the number of parameters low. Retrieval-augmented models commonly rely on a semantic retrieval mechanism based on the similarity between dense representations of the query chunk and potential neighbors. In this paper, we study the state-of-the-art Retro model and observe that its performance gain is better explained by surface-level similarities, such as token overlap. Inspired by this, we replace the semantic retrieval in Retro with a surface-level method based on BM25, obtaining a significant reduction in perplexity. As full BM25 retrieval can be computationally costly for large datasets, we also apply it in a re-ranking scenario, gaining part of the perplexity reduction with minimal computational overhead.

3. On the Generalization Ability of Retrieval-Enhanced Transformers

   Tobias Norlund, Ehsan Doostmohammadi, Richard Johansson, and Marco Kuhlmann

   In Findings of the Association for Computational Linguistics: EACL 2023, May 2023

   Abs PDF

   Recent work on the Retrieval-Enhanced Transformer (RETRO) model has shown impressive results: off-loading memory from trainable weights to a retrieval database can significantly improve language modeling and match the performance of non-retrieval models that are an order of magnitude larger in size. It has been suggested that at least some of this performance gain is due to non-trivial generalization based on both model weights and retrieval. In this paper, we try to better understand the relative contributions of these two components. We find that the performance gains from retrieval to a very large extent originate from overlapping tokens between the database and the test data, suggesting less of non-trivial generalization than previously assumed. More generally, our results point to the challenges of evaluating the generalization of retrieval-augmented language models such as RETRO, as even limited token overlap may significantly decrease test-time loss. We release our code and model at \urlhttps://github.com/TobiasNorlund/retro

4. An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems

   Mehrdad Farahani, and Richard Johansson

   In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), May 2023

   Abs PDF

   Autoregressive models used to generate responses in open-domain dialogue systems often struggle to take long-term context into account and to maintain consistency over a dialogue. Previous research in open-domain dialogue generation has shown that the use of \textitauxiliary tasks can introduce inductive biases that encourage the model to improve these qualities. However, most previous research has focused on encoder-only or encoder/decoder models, while the use of auxiliary tasks in \textitencoder-only autoregressive models is under-explored. This paper describes an investigation where four different auxiliary tasks are added to small and medium-sized GPT-2 models fine-tuned on the PersonaChat and DailyDialog datasets. The results show that the introduction of the new auxiliary tasks leads to small but consistent improvement in evaluations of the investigated models.

5. Class Explanations: the Role of Domain-Specific Content and Stop Words

   Denitsa Saynova, Bastiaan Bruinsma, Moa Johansson, and Richard Johansson

   In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), May 2023

   Abs PDF

   We address two understudied areas related to explainability for neural text models. First, \textitclass explanations. What features are descriptive across a class, rather than explaining single input instances? Second, the \textittype of features that are used for providing explanations. Does the explanation involve the statistical pattern of word usage or the presence of domain-specific content words? Here, we present a method to extract both class explanations and strategies to differentiate between two types of explanations – domain-specific signals or statistical variations in frequencies of common words. We demonstrate our method using a case study in which we analyse transcripts of political debates in the Swedish Riksdag.

2022

1. Controlling for Stereotypes in Multimodal Language Model Evaluation

   Manuj Malik, and Richard Johansson

   In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, Sep 2022
2. Coveting Your Neighbor’s Wife: Using Lexical Neighborhoods in Substitution-based Word Sense Disambiguation

   Richard Johansson

   In LIVE and LEARN – Festschrift in honor of Lars Borin, Sep 2022

   PDF
3. Cross-modal Transfer Between Vision and Language for Protest Detection

   Ria Raj, Kajsa Andéasson, Tobias Norlund, Richard Johansson, and Aron Lagerberg

   In Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE), Sep 2022

   PDF
4. Can We Use Small Models to Investigate Multimodal Fusion Methods?

   Lovisa Hagström, Tobias Norlund, and Richard Johansson

   In Proceedings of the 2022 CLASP Conference on (Dis)embodiment, Sep 2022

   PDF
5. Conceptualizing Treatment Leakage in Text-based Causal Inference

   Adel Daoud, Connor Jerzak, and Richard Johansson

   In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jul 2022

   PDF
6. What do Models Learn From Training on More Than Text? Measuring Visual Commonsense Knowledge

   Lovisa Hagström, and Richard Johansson

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, May 2022

   PDF
7. How to Adapt Pre-trained Vision-and-Language Models to a Text-only Input?

   Lovisa Hagström, and Richard Johansson

   In Proceedings of the 29th International Conference on Computational Linguistics (COLING), May 2022

   PDF
8. Semi-supervised Learning with Natural Language Processing for Right Ventricle Classification in Echocardiography – a Scalable Approach

   Eva Hagberg, David Hagerman, Richard Johansson, Nasser Hosseini, Jan Liu, Elin Björnsson, Jennifer Alvén, and Ola Hjelmgren

   Computers in Biology and Medicine, May 2022

   PDF

2021

1. Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?

   Tobias Norlund, Lovisa Hagström, and Richard Johansson

   In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, May 2021

   PDF
2. Knowledge Distillation for Swedish NER models: A Search for Performance and Efficiency

   Lovisa Hagström, and Richard Johansson

   In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa 2021), May 2021

   PDF