Binder: Hierarchical Concept Representation through Order Embedding of Binary Vectors

Croix Gyurek, Niloy Talukder, Mohammad Hasan

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Concept Graph, Hierarchical Embedding, Order Embedding, Binary Vector Embedding

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

For natural language understanding and generation, embedding concepts using an
order-based representation is an essential task. Unlike traditional point vector based
representation, an order-based representation imposes geometric constraints on the
representation vectors for explicitly capturing various semantic relationships that may exist
between a pair of concepts. In existing literature, several approaches on order-based
embedding have been proposed, mostly focusing on capturing hierarchical relationships; examples include, Order embedding, Poincar\'e embedding on hyperbolic space, and Box embedding.
Each of the above approaches suffers from some significant weaknesses. Order embedding
fails to capture full spectrum of logical operations (such as, inverse, union)
on their embedding vectors, which are essential for deducing complementary or aggregated concepts. Box embedding overcomes this limitation by making the representation richer,
but along the process it sacrifices simplicity requiring custom-made optimization scheme
for learning the representation. Poincar\'e embedding improves embedding quality by
exploiting the ever-expanding property of hyperbolic space, but it also suffers from the
same fate as box embedding as gradient descent like optimization is not easy in the
hyperbolic space. In this work, we propose BINDER, a novel approach for order-based
representation. BINDER uses binary bits as representation vectors.
BINDER uses a simple, yet efficient algorithm for learning representation
vectors in a fraction of time in comparison to existing order-based representation learning
methods. Our experimental results show that BINDER is very accurate, yielding better
results than the existing state-of-the-art methods for both prediction and reconstruction tasks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8183