Dual-Channel Steering: Combining Explicit Prompting and Implicit Parameter Modulation for Reasoning Diversity

Takahito Tanimura, Kotaro Furuya

Track: tiny / short paper (up to 5 pages)

Keywords: Reasoning Diversity, Small Language Model, Deterministic Ensembles, Inference-Time Parameter Modulation

TL;DR: Prompt variation and Text-to-LoRA adapters induce complementary, low-correlation errors, and their hybrid ensemble improves majority voting on GSM8K under fixed compute.

Abstract:

Explicit Chain-of-Thought (CoT) prompting is a powerful lever for improving reasoning in language models, but it operates solely through input-level instructions. In this work, we compare two deterministic, language-indexed channels for building ensembles from a single language model:
(i) explicit context steering via prompt variation, and (ii) implicit parameter modulation via Text-to-LoRA (T2L),
where a fixed hypernetwork maps natural-language descriptors to LoRA adapters at inference time.
We refer to this unified setup as Dual-Channel Steering (DCS) and quantify diversity by pairwise error correlation and accuracy metrics.
On GSM8K with Mistral-7B-Instruct,
we find that while task-specific descriptors limit diversity due to semantic overlap,
using heterogeneous descriptors (describing unrelated tasks)
suggests more complementary behaviors
and lower error correlation among members, which is consistent with the improved majority voting observed for hybrid DCS.
Moreover, prompt-induced and T2L-induced errors are less correlated across channels than within a channel, indicating complementary failure patterns. Consistently, hybrid ensembles improve voting under a fixed budget: with $k{=}9$, majority-vote accuracy peaks at $\approx$ 62.5 % at an intermediate T2L ratio, compared to $\approx$ 59.1 % (prompt-only) and $\approx$ 60.1 % (T2L-only). These observations suggest that language-controlled parameter modulation can complement prompt steering as a practical source of ensemble diversity without retraining.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 35