Towards Knowledgeable Foundation Models

@ ACL 2026 Workshop

San Diego, California, United States

Call for Papers 📹 Join on Zoom

Towards Knowledgeable Foundation Models

Foundation models have reshaped how AI systems acquire and utilize knowledge. Rather than relying solely on structured knowledge bases or curated documents, these models internalize vast amounts of world knowledge through large-scale pre-training, and can generate, retrieve, and reason over that knowledge at inference time.

Yet fundamental questions remain: Where does this knowledge come from? How much do foundation models know? Is their knowledge reliable and up-to-date? Can we control what they remember or forget? As models grow in scale and are deployed in multimodal, agentic, and retrieval-augmented settings, understanding and managing the knowledge lifecycle becomes increasingly critical.

This workshop examines the lifecycle of knowledge within foundation models across four key stages:

Emergence

How knowledge arises through pre-training and scaling

Injection

Augmenting models with external and retrieved knowledge

Updating

Editing, correcting, and erasing knowledge in models

Probing & Generation

Evaluating, extracting, and generating knowledge

This is the 4th KnowFM workshop. Previous editions: KnowFM@ACL2025, KnowFM@AAAI2025, and KnowLM@ACL2024.

Call for Papers

Foundation models have demonstrated remarkable capabilities in storing and utilizing knowledge acquired during pre-training. Yet as these models scale and are deployed in increasingly complex settings, critical challenges remain: How can we reliably assess what a model knows? How do we reconcile conflicting knowledge from parametric memory and retrieved context? How can we keep model knowledge up-to-date without compromising reasoning abilities? And how do we extend these capabilities beyond text to multimodal and agentic settings?

This workshop brings together researchers working on different stages and aspects of the knowledge lifecycle, from structured and unstructured knowledge sources to knowledge acquired and synthesized by models themselves, to discuss how knowledge should be represented, acquired, verified, and applied in the era of foundation models.

Submission Topics

We welcome submissions on all topics related to knowledgeable foundation models, including:

  • Analysis of knowledge within foundation models: how much they know, where that knowledge comes from, and how it is represented
  • Enhancing models with existing knowledge sources (knowledge graphs, domain-specific databases, manuals, rules, etc.) during training or inference
  • Analyzing and improving RAG (retrieval-augmented generation) systems
  • Updating, editing, and erasing knowledge in foundation models
  • Knowledge extraction, generation, and distillation using foundation models
  • Synthetic data quality and reliability for knowledge-intensive tasks
  • Evaluation of knowledge utilization (faithfulness, truthfulness, attribution) by foundation models
  • Identification and mitigation of hallucinations and factual errors
  • Knowledge conflicts: resolving inconsistencies between parametric memory and retrieved or long-context information
  • Knowledge in multimodal foundation models: visual knowledge, cross-modal grounding, and multimodal RAG
  • Knowledge-intensive agents: search agents, tool-augmented agents, and agentic RAG systems
  • Grounding and knowledge acquisition in multi-step reasoning and planning agents

Paper Awards

We will also announce a Best Paper Award and an Outstanding Paper Award at our workshop.

Submission Instructions

We welcome two types of papers: regular workshop papers and non-archival submissions. Only regular workshop papers will be included in the workshop proceedings. Review process will be double-blind. All submissions should be in PDF format following the ACL template (8 pages for main text) and made through OpenReview submission portal (https://openreview.net/group?id=aclweb.org/ACL/2026/Workshop/KnowFM)

Important Dates

All deadlines are 23:59pm UTC-12h ("Anywhere on Earth").
Submission deadline is set after the ACL 2026 notification of acceptance.

Submission Deadline
April 1st, 2026 April 7th, 2026
Decision Notifications
April 20th, 2026
Camera-Ready Deadline
May 1st, 2026
Workshop Date
July 3, 2026

Invited Speakers

Luna Dong

Luna Dong

Meta

Alexander Spangher

Alexander Spangher

Stanford University

Weijia Shi

Weijia Shi

Meta

Parisa Kordjamshidi

Parisa Kordjamshidi

Michigan State University

Lianhui Qin

Lianhui Qin

UC San Diego

Jiaxin Pei

Jiaxin Pei

Stanford & UT-Austin

Schedule

July 3, 2026 · KnowFM @ ACL 2026
All times shown in Pacific Time (PT)

📹 Join Online (Zoom) Meeting ID: 829 2472 1594 Passcode: 184184 Join Meeting →
9:30–10:00 am
Opening Remarks: Manling Li
Towards Knowledgeable Foundation Models
10:00–10:30 am
Keynote: Luna Dong
From Sight to Insight: Visual Memory for Smarter Assistants
10:30–11:00 am
Measuring Tacit Knowledge and Articulation Gaps in Human Preferences
11:00 am–12:30 pm
Oral Presentations
  1. 11:00–11:12 amRule vs. Consequence: Dissociable Internal Representations of Moral Reasoning in LLMsEugenie Shi
  2. 11:13–11:25 amWhen Do LLMs Admit Their Mistakes? Understanding the Role of Model Belief in RetractionYuqing Yang, Robin Jia
  3. 11:26–11:38 amTowards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model FinetuningLu Dai, Ziyang Rao, Yili Wang, Hanqing Wang, Hao Liu, Hui Xiong
  4. 11:39–11:51 amBaseCal: Unsupervised Confidence Calibration via Base Model SignalsHexiang Tan, Wanli Yang, Junwei Zhang, Xin Chen, Rui Tang, Du Su, Jingang Wang, Yuanzhuo Wang, Fei Sun, Xueqi Cheng
  5. 11:52 am–12:04 pmScienceMeter: Tracking Scientific Knowledge Updates in Language ModelsYike Wang, Shangbin Feng, Yulia Tsvetkov, Hannaneh Hajishirzi
  6. 12:05–12:17 pmEmergent Unfaithfulness: How Alignment Training Causes LMs to Silently Override Task FaithfulnessPardis Sadat Zahraei, Janvijay Singh, Gokhan Tur, Dilek Hakkani-Tür
  7. 12:18–12:30 pmFaithLens: Detecting and Explaining Faithfulness HallucinationShuzheng Si, Qingyi Wang, Haozhe Zhao, Yuzhuo Bai, Guanqiao Chen, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun
12:30–2:00 pm
Lunch Break
2:00–2:30 pm
Keynote: Weijia Shi
FlexOlmo: Open Language Models for Flexible Data Use
2:30–3:00 pm
Beyond Memorization: Conceptual Knowledge and Compositional Reasoning in LLMs
3:00–3:30 pm
Keynote: Lianhui Qin
Agents Reason, Learn, and Act in Complex Embodied and Scientific Environments
3:30–4:00 pm
Keynote: Jiaxin Pei
Multi-User Context Management for Collaborative Agents
4:00–4:10 pm
Conclusion & Award Announcement
4:10–5:30 pm
Virtual Poster Session
Xin Luna Dong
Keynote
From Sight to Insight: Visual Memory for Smarter Assistants

Imagine a personal assistant that, with user permission, persistently remembers moments from daily life—answering questions like “When and where did I see this lady?” or offering personalized suggestions like “You might enjoy The Little Prince—it relates to the statue you liked in Lyon.” Realizing this vision requires overcoming major challenges: capturing visual memories under hardware constraints (e.g., memory, battery, thermal limits, bandwidth), extracting meaningful personalization signals from noisy, task-agnostic visual histories, and supporting real-time question answering and recommendations under tight latency requirements.

In this talk, we present our early work toward this goal. Pensieve, our memory-based QA system, improves accuracy by 11% over state-of-the-art multimodal RAG baselines. VisualLens infers user interests from casual photos, outperforming leading recommendation systems by 5–10%. We also share initial results on efficient, event-triggered memory capture and compression. Our work points to a broad landscape of research opportunities in building richer, more context-aware personal assistants capable of learning from and reasoning over users’ visual experiences.

Bio. Xin Luna Dong is a Principal Scientist at Meta Wearables AI, where she leads the Agentic AI efforts for building trustworthy and personalized assistants on wearable devices. Previously, she spent over a decade advancing knowledge graph technology, including the Amazon Product Graph and the Google Knowledge Graph. She is co-author of Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases and Big Data Integration. She was named an ACM Fellow and an IEEE Fellow for “significant contributions to knowledge graph construction and data integration”, awarded the VLDB Women in Database Research Award and VLDB Early Career Research Contribution Award, and invited as an ACM Distinguished Speaker. She was a member of the VLDB endowment, a PC co-chair for ICDM’2027, KDD’2022 ADS track, WSDM’2022, VLDB’2021, and Sigmod’2018.

Alexander Spangher
Keynote
Alexander Spangher · Stanford University
Measuring Tacit Knowledge and Articulation Gaps in Human Preferences

Is a mathematical proof elegant? When is an experiment done? Why is a finding newsworthy? Human experts often agree on these decisions, but can struggle to completely describe, in natural language, why they agree: the decision involves intuition, experience and deep tacit knowledge. We call this phenomenon an “articulability gap”. However, as agents become more integrated into our daily lives, rubrics — or fully articulated preference specifications — have emerged as the primary tool for driving improvement. Thus, understanding and measuring articulability gaps is increasingly important in understanding when artificial intelligence systems may deviate from human preference. In this talk, we will discuss an ongoing research program that seeks to make progress in these areas. We will start by discussing a broad survey we have conducted of over 5,000 years of human preferences across a number of tasks: creative writing, law, math, coding, humor, and others. We then present a novel set of techniques for measuring articulation bounds on these preferences that will allow us to understand: how articulable are tastes in given fields; how much tacit knowledge is required to understand a taste; and how much of a taste is agreeable, verifiable and rule-based. Finally, we will empirically demonstrate the persistence of articulability gaps across the fields we study and discuss remedies and implications.

Bio. Alexander Spangher is a SAIL and an HAI post-doctoral fellow advised by Daniel Ho, Sanmi Koyejo and Diyi Yang at Stanford University. His research focuses on modeling human decision-making in creative domains, especially in contexts where rewards and goals are less clear. He is especially passionate about helping journalists and has framed tasks and trained reasoning LLMs to help journalists find stories and sources, structure narratives and track information updates. These tools have been incorporated into newsrooms at the New York Times and Bloomberg, impacting thousands of journalists. His work has received numerous awards including two outstanding paper awards at EMNLP 2024, one spotlight award at ICML 2024, one outstanding paper award at NAACL 2022 and he has been supported by a 4-year Bloomberg PhD Fellowship. His work is broad: in addition to his work in NLP and computational journalism, he has built piano-playing robots with EleutherAI, studied misinformation at Microsoft Research, and models plasma fusion processes at the MIT Plasma Science and Fusion Center.

Weijia Shi
Keynote
Weijia Shi · Meta
FlexOlmo: Open Language Models for Flexible Data Use

Language models are typically monolithic: a single model stores all knowledge and serves every use case. In this talk, I will introduce a new paradigm for building inherently modular language models from decentrally trained components. This approach enables flexible composition, data ownership, and an unprecedented level of control over how language models are built and deployed.

Bio. Weijia Shi is a Research Scientist at Meta, and recently completed her PhD at the University of Washington. Her research focuses on developing modular and augmented language model architectures, along with training algorithms that make language models more controllable, factual, and collaboratively developed. Her representative work includes REPLUG, In-Context Pretraining, Detecting Pretraining Data (Min-K% Prob), OpenScholar, and FlexOlmo. She received an Outstanding Paper Award at ACL 2024 and was recognized as a Rising Star in Machine Learning (2023) and Data Science (2024).

Parisa Kordjamshidi
Keynote
Parisa Kordjamshidi · Michigan State University
Beyond Memorization: Conceptual Knowledge and Compositional Reasoning in LLMs

Bio. Parisa Kordjamshidi is an Associate Professor of Computer Science and Engineering at Michigan State University. Her research focuses on Natural Language Processing, multimodal reasoning across vision and language, and neuro-symbolic learning. She received her Ph.D. from KU Leuven and conducted postdoctoral research at the University of Illinois Urbana-Champaign. She is a recipient of the NSF CAREER, Amazon Faculty Research, and Fulbright Scholar Awards, and her research team received the NAACL-2025 Outstanding and EMNLP-2025 SAC and MSLD-2026 best paper awards. Dr. Kordjamshidi serves as Associate Editor of JAIR, Co-editor in Chief of ARR, Action Editor for TACL and has held roles in organization committee of major conferences including ACL, NAACL, EACL, EMNLP, ECML-PKDD, and AAAI. Recently, she was visiting associate professor at UCLA and currently is visiting professor at Bloomberg.

Lianhui Qin
Keynote
Lianhui Qin · UC San Diego
Agents Reason, Learn, and Act in Complex Embodied and Scientific Environments

Large language and vision-language models are increasingly described as knowledgeable, but knowledgeable models are not yet knowledgeable agents. In complex environments, knowledge must be operationalized: agents need to reason robustly, learn from feedback, act over long horizons, and ground their behavior in the constraints of external worlds and tools.

In this talk, I will present a line of recent work toward building such agents across embodied and scientific settings. First, I will discuss LaDiR and LaDi-RL, which study reasoning as a space of diverse latent trajectories rather than a single brittle chain of thought. This enables global refinement, best-of-N robustness, and post-training methods that improve reasoning while preserving strategy diversity. Second, I will introduce AgentSpec, a framework for understanding how reasoning models become interactive agents through scaffolds such as perception, memory, reflection, action execution, and reinforcement learning. Third, I will present SimWorld Studio, which studies how to scale embodied-agent learning by scaling environment generation, using a self-evolving coding agent to generate verifiable, trainable 3D worlds for VLM agents. Finally, I will discuss SIGA, a scientific-agent use case where coding agents are grounded in the executable interface of scientific simulators through retrieval, procedural memory, validation, and enforced termination.

Together, these works argue that the path from knowledgeable models to knowledgeable agents requires four ingredients: robust reasoning, scaffolded interaction, generated environments, and grounded execution.

Bio. Lianhui Qin is an Assistant Professor in Computer Science at UC San Diego. Her research focuses on AI agents capable of reasoning, learning, and generalizing in complex environments. Her work advances machine reasoning and generation, large language and multimodal models, self-evolving agents, world modeling, and AI reasoning for science. Her recent focus includes SimWorld that simulates open-ended embodied environments and agents with coding, reinforcement learning, and environment-agent co-evolution. She completed her Ph.D. at the University of Washington under the supervision of Yejin Choi. She was named a 2021 Microsoft Research Ph.D. Fellow.

Jiaxin Pei
Keynote
Jiaxin Pei · Stanford & UT-Austin
Multi-User Context Management for Collaborative Agents

As foundation models are increasingly deployed as agents in collaborative settings, a critical challenge emerges: how should an AI agent manage knowledge and context not just from documents or databases, but from multiple users with distinct roles, preferences, and even conflicting objectives? In this talk, I will discuss context management for teams with AI agents, drawing on two lines of recent work. First, I will present a systematic study of multi-user LLM agents (arXiv), formalizing the problem as multi-principal decision-making where a single agent must navigate information asymmetry, privacy constraints, and competing interests across users. Second, I will examine the challenge of faithfully representing and aggregating the opinions of large groups through LLMs (arXiv). Together, these threads point toward a broader research agenda: extending the knowledge lifecycle (how knowledge is acquired, integrated, and applied) beyond single-user interactions to the multi-user, multi-stakeholder settings where AI agents are increasingly being deployed.

Bio. Jiaxin Pei is a Postdoctoral Fellow at the Stanford Institute for Human-Centered AI (HAI) and an incoming Assistant Professor at the School of Information at UT Austin (Fall 2026). He received his PhD from the School of Information at the University of Michigan, advised by David Jurgens. His research spans Large Language Models, Human-AI Collaboration, and AI Agents, including the economics of AI agents, collaborative agents and AI-native organizations, and the evaluation and governance of frontier AI systems. He is a core developer of POTATO, an open-source data labeling toolkit, and his work has received a Best Student Paper Award at ACM EAAMO and a Best Demo Paper Award at AAAI HCOMP.

Accepted Papers

We are pleased to announce the accepted papers for the KnowFM workshop. You can view the full list of accepted papers on OpenReview and their posters here.

Organizers

Organizing Committee

Canyu Chen

Canyu Chen

Northwestern University

Yuji Zhang

Yuji Zhang

University of Illinois Urbana-Champaign

Zoey Sha Li

Zoey Sha Li

Amazon

Zihan Wang

Zihan Wang

Northwestern University

Qineng Wang

Qineng Wang

Northwestern University

Jinyan Su

Jinyan Su

Cornell University

Priyanka Kargupta

Priyanka Kargupta

University of Illinois Urbana-Champaign

Sara Vera Marjanovic

Sara Vera Marjanović

University of Copenhagen

Jeff Z. Pan

Jeff Z. Pan

University of Edinburgh

Manling Li

Manling Li

Northwestern University

Advising Committee

Heng Ji

Heng Ji

University of Illinois Urbana-Champaign

Mohit Bansal

Mohit Bansal

University of North Carolina at Chapel Hill

Isabelle Augenstein

Isabelle Augenstein

University of Copenhagen

Jiawei Han

Jiawei Han

University of Illinois Urbana-Champaign

Contact

Please email know-fm-acl26@googlegroups.com if you have any questions.