Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Embeddings as epistemic states: Limitations on the use of pooling operators for accumulating knowledge

Schockaert, Steven ORCID: 2023. Embeddings as epistemic states: Limitations on the use of pooling operators for accumulating knowledge. International Journal of Approximate Reasoning: Uncertainty in Intelligent Systems , 108981. 10.1016/j.ijar.2023.108981

[thumbnail of 1-s2.0-S0888613X23001123-main.pdf] PDF - Published Version
Available under License Creative Commons Attribution.

Download (1MB)
License URL:
License Start date: 13 July 2023


Various neural network architectures rely on pooling operators to aggregate information coming from different sources. It is often implicitly assumed in such contexts that vectors encode epistemic states, i.e. that vectors capture the evidence that has been obtained about some properties of interest, and that pooling these vectors yields a vector that combines this evidence. We study, for a number of standard pooling operators, under what conditions they are compatible with this idea, which we call the epistemic pooling principle. While we find that all the considered pooling operators can satisfy the epistemic pooling principle, this only holds when embeddings are sufficiently high-dimensional and, for most pooling operators, when the embeddings satisfy particular constraints (e.g. having non-negative coordinates). We furthermore show that these constraints have important implications on how the embeddings can be used in practice. In particular, we find that when the epistemic pooling principle is satisfied, in most cases it is impossible to verify the satisfaction of propositional formulas using linear scoring functions, with two exceptions: (i) max-pooling with embeddings that are upper-bounded and (ii) Hadamard pooling with non-negative embeddings. This finding helps to clarify, among others, why Graph Neural Networks sometimes under-perform in reasoning tasks. Finally, we also study an extension of the epistemic pooling principle to weighted epistemic states, which are important in the context of non-monotonic reasoning, where max-pooling emerges as the most suitable operator.

Item Type: Article
Date Type: Published Online
Status: In Press
Schools: Computer Science & Informatics
Publisher: Elsevier
ISSN: 0888-613X
Funders: EPSRC
Date of First Compliant Deposit: 26 July 2023
Date of Acceptance: 5 July 2023
Last Modified: 19 Sep 2023 16:34

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics