Structuring Semantic Embeddings for Principle Evaluation: A Kernel-Guided Contrastive Learning Approach

ICLR 2026 Conference Submission16359 Authors

Keywords: Contrastive Learning, Semantic Embeddings, Principle Alignment, Structured Representation, Prototype Learning

TL;DR: This paper presents a novel learning framework that creates specialized text embeddings to more reliably evaluate abstract principles by representing each principle with a distinct "prototype kernel" and enforcing separation between them.

Abstract: Evaluating principle adherence in high-dimensional text embeddings is challenging because principle-specific signals are often entangled with general semantic content. Our kernel-guided contrastive learning framework learns to disentangle these signals by projecting embeddings into a structured subspace. In this space, each principle is centered on a learnable **prototype kernel**---an optimized vector that embodies its core meaning---while a jointly learned **semantic basis** preserves context. A novel **offset penalty**, a loss term designed to create structure, then enforces a margin around each prototype. This ensures that even semantically similar principles are clearly separated while capturing their inherent contextual variability. Experiments show our optimized embeddings significantly improve performance on principle classification and ordinal regression, outperforming few-shot Large Language Models and demonstrating the value of specialized representations for reliable principle evaluation.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 16359