Knowledge has been an important prerequisite for various NLP applications and is typically derived from either structured knowledge sources such as knowledge bases and dictionaries or unstructured knowledge sources such as Wikipedia documents and news articles.

It is known that language models already possess a significant amount of knowledge through pre-training: LLMs can be used to generate commonsense knowledge and factual knowledge when prompted to do so. However, beyond the surface, there are still many lingering questions such as “where the knowledge comes from”, “how do we quantify the amount of knowledge”, “is the knowledge reliable (and do LMs themselves know)”, “how can we augment LMs with domain-specific knowledge”, “how can we revise knowledge without hurting the reasoning abilities of LMs” and “how can we leverage knowledge to assist the self-correction of LMs”.

In this workshop, we want to bring together researchers who focus on different stages and different aspects (structured knowledge, unstructured knowledge, and knowledge acquired from LMs themselves) of the knowledge lifecycle to discuss the role of knowledge in the era of large language models.

Submission Topics

We welcome submissions on all topics related to knowledgable LMs, including:

• Analysis of knowledge within LMs: how much they know and where that knowledge is from.
• Enhancing LMs with existing knowledge sources (knowledge graphs, domain-specific databases, manuals, and rules, etc, either during training or inference).
• Analyzing and improving RAG (retrieval-augmented generation) systems
• Updating and editing knowledge in LMs.
• Knowledge extraction and generation using LMs
• Evaluation of knowledge utilization (faithfulness, truthfulness) by LMs.
• Identification and mitigation of LM hallucinations, factual error correction

We will also announce a Best Paper Award at our workshop sponsored by Amazon.

Submission Instructions

We solicit long papers (7 pages), short papers (4 pages), abstract papers (2 pages) with unlimited references/appendices. The contributions will be non-archival but will be hosted on our workshop website. Papers must be formatted in AAAI two-column, camera-ready style; see the AAAI-25 author kit for details. Please submit through OpenReview submission portal.

Coming soon!