# USER:

You are a senior AI researcher. Please help me complete the following paper's structure:

I am writing a paper with the topic: How to utilize LLMs to generate innovative ideas for scientific research.

I hope to discover some underlying algorithmic laws, preferably prove some mathematical assumptions about the fundamental aspects of LLMs, and propose improvement methods.

summary.txt contains some papers I have collected that may be relevant. You can selectively reference their ideas, but please pay attention to the novelty of your proposed methods, ensuring they do not overlap with previous papers.

I want you to think from the essence of LLMs: what are their detailed principles and how are they trained? Based on this, how can we design them to generate innovative ideas?

Please note that our resources are insufficient for fine-tuning/training the network. We hope to innovate through test-time operations.

----------------------------------------
# AGENT:

\<think\>

Initiating Research Focus

I'm starting by dissecting the user's request. My focus is on understanding the core task: formulating a research proposal exploring the application of Large Language Models (LLMs) to generate innovative scientific research. This involves breaking down the prompt into key elements, clarifying the desired output, and identifying the constraints. The goal is to establish a solid foundation for the proposal's structure and scope.

Unpacking the User's Intent

I'm now diving deep into the constraints. The core challenge is leveraging LLMs for innovation without training them. It's a test-time game! The "bottom-up" approach really resonated with me, pushing me to start from the LLM's architecture. I'm focusing on the inherent probability distribution and latent space. My initial brainstorming's centered on how to creatively manipulate the model's output at inference.

Formulating a Strategy

I've refined my approach based on the insights gained. My focus now lies on crafting a strategy to leverage LLMs for scientific innovation, while adhering to the strict "inference-time only" constraint. I'm exploring methods to manipulate the prompt, internal states, and sampling processes within the latent space, inspired by the "latent space exploration" and "multi-view embeddings" concepts.  This is all to build novelty without model training.

Refining Latent Space Manipulation

I've gone further into how to generate novelty. My exploration of the provided papers, especially Paper 9, highlighted the importance of Latent Space Exploration. I'm focusing now on directly manipulating the target LLM's latent space, without relying on external components. My thought process is now on methods to move within this space, inspired by the "facet recombination" approach from papers like 7 and 10. The aim is to make my approach more mathematically rigorous than just linear interpolation.

Synthesizing Novel Approaches

I'm now integrating the strengths of the reviewed papers. I'm focusing on "facet recombination" within the LLM's latent space, inspired by Papers 7 and 10. My method aims to directly manipulate the internal state, not the input, and will employ a mathematical method beyond linear interpolation. The goal is to identify less-dense areas of the latent space, potentially containing novel scientific ideas. I'll need to refine this concept to fit within the constraint of only inference-time manipulation.

Conceptualizing the Approach

I'm developing a test-time method, building on "facet recombination" from the papers. The goal is to innovate by manipulating the LLM's latent space to find less dense semantic regions. It's about moving beyond linear interpolation and crafting a more robust, mathematically grounded methodology.

Formulating the Core Method

I've finally cracked the core methodology. My approach, dubbed "Latent Orthogonal Projection for Scientific Innovation," leverages orthogonal projections in the LLM's latent space. We'll start with concept prompts, extract hidden states, and mathematically combine these to generate novel ideas. My next task is formalizing this method and integrating it with the "facet recombination" concept.

Formulating a Coherent Methodology

I'm solidifying my approach, now termed "Conceptual Orthogonal Projection (COP)." The core is to manipulate latent states to foster novelty. I'll prompt the LLM to provide core concepts and then use orthogonal projections to find novel intersections of ideas.  My plan is to quantify novelty and coherency via specific metrics, guided by the papers. I am ready to start testing my method.

Refining Methodology Details

I'm now fully immersed in developing the methodology. My key focus is on "Conceptual Orthogonal Projection (COP)" and formalizing the mathematical aspects, particularly the projection and perturbation steps. I'll utilize the papers to guide the novelty and evaluation phases. My method emphasizes test-time manipulation of latent states via mathematical operations. The goal is to rigorously quantify novelty and provide an evaluation.

Conceptualizing the Strategy

I'm solidifying my approach, now named "Conceptual Orthogonal Projection (COP)," rooted in linear algebra for novelty generation. I'll prompt the LLM to provide core concepts and use orthogonal projections to find novel idea combinations. The plan is to quantify novelty and coherency via specific metrics, guided by relevant research papers.

Formulating the Core Method

I'm now integrating the core "Conceptual Orthogonal Projection (COP)" concept into a coherent approach. The prompt will set up a 'Deconstruct, Intersect, Synthesize, Assess' workflow. The innovation step utilizes orthogonal projections in the latent space to foster novel idea combinations, moving beyond mere interpolation. I'll quantify novelty and coherence using relevant research metrics.

Finalizing Methodology Details

I'm now fully immersed in developing the methodology. My key focus is on "Conceptual Orthogonal Projection (COP)" and formalizing the mathematical aspects, particularly the projection and perturbation steps. I'll utilize the papers to guide the novelty and evaluation phases. My method emphasizes test-time manipulation of latent states via mathematical operations. The goal is to rigorously quantify novelty and provide an evaluation.

\</think\>

----------------------------------------
Of course. This is a fascinating and critical research direction. Building on the literature you've provided and focusing on the fundamental principles of LLMs, here is a detailed Idea construction for your paper.

The core of this proposal is a novel, test-time method that moves beyond simple prompt engineering or basic latent space interpolation. It's grounded in a mathematical hypothesis about the structure of an LLM's knowledge.

Working Title:
Conceptual Orthogonal Projection: Navigating the LLM Knowledge Manifold for Scientific Innovation

Abstract:
Large Language Models (LLMs) excel at synthesizing existing information but struggle to generate truly innovative scientific ideas, often defaulting to high-probability, well-trodden concepts. This limitation stems from their training objective, which optimizes for predicting plausible sequences based on a learned distribution of existing knowledge. We hypothesize that an LLM's internal latent space can be modeled as a high-dimensional "knowledge manifold," where dense regions represent established concepts and sparse regions represent novel intersections. This paper introduces Conceptual Orthogonal Projection (COP), a novel, inference-time algorithm designed to steer the generation process towards these sparse, innovative regions. COP operates by deconstructing research domains into core conceptual vectors (e.g., problem, mechanism), and then mathematically projecting them into orthogonal subspaces to discover non-obvious yet coherent relationships. By perturbing the generation process with these "orthogonal concept" vectors, our method forces the LLM to explore paths it would otherwise deem improbable. We propose a full framework, the DISA cycle (Deconstruct, Intersect, Synthesize, Assess), built around COP. We will demonstrate through experiments in materials science and computational biology that this method generates ideas rated significantly higher in novelty and creativity by human experts compared to current state-of-the-art prompting and latent-space techniques, all without any model fine-tuning.



1. Core Ideas and Theoretical Basis
1.1. The LLM as a Gravitationally-Bound Knowledge Manifold
Training Principle of LLMs: LLMs are trained through an auto-regressive process, with the core objective being to maximize the conditional probability P(token_{n+1} | token_1,...,token_n) of the next token appearing. This means that the model is essentially a massive probability distribution over sequences of human knowledge.

Mathematical Assumption - Knowledge Manifold: We can conceptualize the internal high-dimensional embedding space (latent space) of an LLM as a nonlinear manifold.

High-Density Regions: Regions of high density on the manifold represent frequently co-occurring, mature, and well-known concept combinations in the training data (for example, "CRISPR" and "gene editing"). The standard generation process (such as conventional prompting) is like a ball bearing that naturally rolls along geodesic paths into these high-density "gravity wells."

Sparse Regions: Regions of low density on the manifold represent unexplored or rare knowledge combinations. True innovation exists in these areas.

Challenge: How can we guide the generation process to these sparse regions without "tearing" the manifold (i.e., generating meaningless nonsense)?

1.2. Limitations of Existing Methods
Prompt Engineering (Refer to Papers 1, 11): Using multi-perspective inputs like Multi-Novelty essentially places ball bearings at multiple entry points on the manifold, hoping to find different paths. However, this is still limited by the manifold's own structure and makes it difficult to escape existing "gravity wells."

Simple Latent Space Exploration (Refer to Paper 9): The linear interpolation (v_new = λv_i + (1-λ)v_j) proposed in Paper 9 represents an advancement, but it can only move within the "valley" between two known points and cannot open up entirely new directions. It is exploring known paths on the manifold rather than creating new ones.

Element Reorganization (Refer to Papers 10, 7): The "element reorganization" of Scideator and Idea Novelty Checker is conceptually correct, but their implementation relies on the LLM's "understanding" and splicing of text, lacking underlying mathematical control.

2. Proposed New Method: Conceptual Orthogonal Projection (COP)
Our method does not change the model weights but directly manipulates the internal hidden states of the LLM during inference.

Core Idea: If two concept vectors are orthogonal, then they are "most unrelated" semantically. We can utilize this "unrelatedness" to create a meaningful "tension" that drives the generation of innovative ideas.

2.1. Algorithm Steps
Step 1: Concept Vector Extraction

Given a research field (for example, "nanomaterials for cancer treatment"), we do not directly use this long prompt.

We first have the LLM deconstruct the core concepts of the field. For example, we use two separate prompts to extract the internal representations of two core concepts:

Base Concept C_A: "Briefly describe the core problems of nanoparticles in drug delivery" -> Run the LLM, extract the hidden state vector of its final transformer layer, denoted as v_A ∈R^d.

Perturbation Concept C_B: "Briefly describe the biological mechanism of autophagy" -> Similarly extract its hidden state vector, denoted as v_B ∈R^d.

Step 2: Orthogonal Projection and Decomposition

We decompose vector v_B into two parts: one parallel to v_A, denoted v_B∥A, and one orthogonal to v_A, denoted v_B⊥A.

The parallel component represents the parts of concept B that are "similar" or "related" to concept A.

v_B∥A = proj_{v_A} (v_B) = (v_B·v_A)/(∥v_A∥^2) * v_A

The orthogonal component represents the parts of concept B that are "completely unrelated" to concept A and may bring innovative new information.

v_B⊥A = v_B - v_B∥A

Step 3: Innovative Vector Synthesis

We create a new "innovative direction" vector v_new by combining the base concept vector v_A with the orthogonal component v_B⊥A.

We introduce an innovation factor α (hyperparameter, e.g., 0<α≤1) to control the "step size" of the innovation.

v_new = v_A + α·v_B⊥A

Intuitive Explanation: We retain the core problem (v_A) but add to it a semantic direction (v_B⊥A) from a completely different field (autophagy) that is most unrelated to the original problem. This mathematically forces the model to think: "How can those properties in 'autophagy' that are unrelated to 'nanomedicine delivery' be applied to 'nanomedicine delivery'?"

Step 4: Controlled Decoding Generation

How can we have the LLM generate based on this synthesized vector v_new?

We use v_new as a soft prompt or control vector. We inject it into the initial steps of the LLM decoding process, replacing the embedding of the original prompt.

Then, we have the LLM perform autoregressive generation starting from this "perturbed" initial state. The final prompt could be: "Generate a novel research idea based on the following core concept: [inject v_new]".

3. Complete Scientific Idea Generation Framework: The DISA Cycle
We embed the COP algorithm in a complete, iterative framework.

Deconstruct: The user inputs a broad research field. The system (or with user guidance) decomposes it into multiple core concept pairs (e.g., Problem, Mechanism, Material, Method).

Intersect: The system automatically or with user selection chooses concept pairs (C_A, C_B), applies the COP algorithm to generate a series of innovative vectors {v_new_1, v_new_2,...}.

Synthesize: For each v_new, perform controlled decoding to generate preliminary, concise research ideas.

Assess:

Self-Assessment: Have the LLM score the generated ideas across dimensions including: Novelty, Feasibility, Impact.

Novelty Quantification: Drawing on the approach of Paper 6 (RND), calculate the relative density of the generated idea embedding vector within its neighborhood. A low-density idea is more likely to be novel.

Filtering and Ranking: Filter and select the highest-scoring ideas to present to the user.

This cycle can be iterated, with users able to take a good idea and reuse it as a "base concept" for the next round of deconstruction and intersection.

4. Experimental Design and Evaluation
Task Domains: Select 2-3 interdisciplinary fields, such as:

Material Science + Biology

NLP + Social Sciences

Baseline Methods:

Zero-Shot Prompting: Directly ask the LLM questions.

Chain-of-Thought (CoT) Prompting: The "Let's think step by step" approach.

Multi-View Prompting (Replicating Paper 1/11 approach): Generate multiple perspective prompts and then integrate them.

Latent Space Interpolation (Replicating Paper 9 approach): Implement simple linear interpolation methods.

Evaluation Methods:

Human Expert Evaluation (Primary): Drawing on the method of Paper 4, recruit graduate students or postdoctoral researchers in relevant fields to score anonymously generated ideas from different methods on a 1-5 scale, across dimensions including:

Novelty: Have you ever seen or thought of this idea before?

Feasibility: Is this idea theoretically and practically possible to implement?

Clarity: Can you clearly understand the core of this idea?

Inspiration: Does this idea inspire you to generate subsequent thoughts?

Automated Evaluation (Auxiliary): Calculate the average/minimum cosine similarity between embeddings of generated ideas and paper abstracts in existing literature databases (such as arXiv) as a proxy metric for novelty.

5. Expected Contributions and Novelty
Theoretical Contributions: First proposing the "Knowledge Manifold" hypothesis for LLMs, and building upon this to construct a mathematically grounded and interpretable theory of innovative generation.

Algorithmic Innovation: Proposing the COP algorithm, which goes beyond simple prompt engineering and linear interpolation, offering a new paradigm for precise control over LLM generation direction at the underlying (hidden state) level.

No Training Cost: The entire method is performed entirely **at inference time (test-time)**, requiring no model fine-tuning, thus possessing high generality and practical utility.

Framework Completeness: Proposing the DISA Cycle, which packages the core algorithm into a usable, human-computer collaborative research assistance framework.

This approach provides you with a clear, complete path from theoretical assumptions to algorithm design to experimental validation, while ensuring the novelty of the method relative to existing research.




----------------------------------------
# USER：

I think your analysis is excellent! "Challenge: How to guide the generation process to these sparse regions without "tearing" the manifold (i.e., generating meaningless gibberish)?" But as a research idea, its feasibility is also very important. How to ensure its feasibility? You mentioned "combining the basic concept vector vA with the orthogonal component vB⊥A to create a new "innovation direction" vector vnew", how can we ensure that this generated vector is within a reasonable region? If I can understand the combination of two ideas from a semantic perspective, but can directly adding embeddings of orthogonal vectors produce some reasonable content?

----------------------------------------

# AGENT:

\<think\>

Assessing User Concerns

I'm currently focused on deconstructing the user's core concerns. The goal is to fully understand the challenges presented regarding feasibility. I plan to address these critical points head-on, aiming for a detailed and robust response.

Tackling the "Tearing" Issue

I'm now diving deep into the "tearing the manifold" concern. I'm strategizing how to convey a solution that assures the user the new vector remains meaningfully grounded. My plan centers on using explicit constraints and checks within the vector operations. I aim to ensure v 
new
​
  aligns with the intended concept, thereby avoiding "word salad."

Defining Manifold Anchors

I've been solidifying the concept of "Manifold Anchoring." The core idea is to find a relevant vector from a database to 'ground' the new vector, ensuring it stays close to a meaningful region. I'm focusing on how to efficiently search for these anchors during inference. The goal is a computationally light and robust method.

Defining Manifold Anchors

I've been solidifying the concept of "Manifold Anchoring." The core idea is to find a relevant vector from a database to 'ground' the new vector, ensuring it stays close to a meaningful region. I'm focusing on how to efficiently search for these anchors during inference. The goal is a computationally light and robust method.

Now, I'm fleshing out the process with concrete steps and naming conventions. This includes defining an "Anchor Set," denoted by A, and exploring a blending mechanism to balance novelty and coherence using a hyperparameter, β. My focus now is how to compute v 
anchor
​
  efficiently during the model inference.

Elaborating "Anchor Set" Methods

I'm now detailing the "Anchor Set" ($ \mathcal{A} $) to solidify its utility. The aim is to ensure it helps ground innovation.  I plan to describe a method for computing v 
anchor
​
  and the blending with β,  to ensure the new vector, v 
new
​
 , maintains context. I intend to clarify how the "anchor" concept balances coherence with novelty.

Solidifying "Anchor" Definition

I'm now detailing the "Anchor Set," A, emphasizing its role in maintaining vector coherence. Specifically, I plan to clarify how to compute v 
anchor
​
 , which will involve a search and an interpolation. I'm also planning to outline the blend with the hyperparameter, β, and how it balances coherence and novelty in v 
new
​
 .

Refining "Anchor" Strategy

I'm now detailing the "Anchor Set," focusing on its role in providing context. Specifically, I'm defining a concrete search and blending method. This involves finding v 
anchor
​
  and adjusting v 
new
​
  using the β hyperparameter to improve v 
new
​
  coherence. I want to articulate how "anchoring" provides a solution to the "tearing" challenge.

Solidifying "Anchor" Definition

I'm now detailing the "Anchor Set," A, emphasizing its role in maintaining vector coherence. Specifically, I plan to clarify how to compute v 
anchor
​
 , which will involve a search and an interpolation. I'm also planning to outline the blend with the hyperparameter, β, and how it balances coherence and novelty in v 
new
​
 .

 \</think\>

 ----------------------------------------
The questions you've raised are profound, as they represent exactly the gap that must be bridged when moving from a good idea to a solid research proposal. The two questions you've asked are interconnected, and their core is: how can we believe that mathematical operations on embedding vectors can map to meaningful and coherent semantic changes?

I will address this from two levels, which can directly serve as the core content for the "Methodology" and "Theoretical Assumptions" sections of your paper, enhancing its feasibility and persuasiveness.

Level 1: Ensuring Feasibility — How to ensure that v_{new} falls within a "reasonable" region?

Your intuition is correct. Simply adding an orthogonal vector v_{B⊥A} to v_A will likely result in v_{new} falling into a "void" in the knowledge manifold—a region unseen and incomprehensible to the model. Directly having the decoder start generating from this point will most likely produce incoherent text, which you described as "tearing the manifold."

To solve this problem, we need to add a critical correction step between Step 3 and Step 4 of the COP algorithm, which I call **"Manifold Anchoring"**.

New Step: Manifold Anchoring
Idea: Our innovative vector v_{new} indicates a highly novel "direction," but it may be "floating" outside the manifold. We need to find the **"anchor point"** on the manifold closest to this "direction," then "pull" our vector toward this anchor point, ensuring that the final starting point for generation is both novel and within the model's "understanding range."

Specific Operations:

Constructing Anchor Set (Anchor Set, A):
Before proceeding with formal idea generation, we first construct a set composed of numerous "legitimate" concept vectors. This can be achieved by embedding a large list of domain-specific concepts (for example, scientific concepts from Wikipedia, or keyword libraries from academic papers).
A = {embed(concept_1), embed(concept_2), ..., embed(concept_N)}
This set $\mathcal{A}$ is fixed for a single experiment and can be regarded as a discrete sampling of the knowledge manifold, representing the positions of all "reasonable" concepts.

Finding the Nearest Anchor (Nearest Anchor Search):
After we calculate the "original innovative vector" v_{new} through the COP algorithm, we find the anchor vector v_{anchor} closest to v_{new} in the anchor set $\mathcal{A}$ through efficient vector retrieval (such as FAISS).

v_{anchor} = arg_{v∈A} min distance(v, v_{new})

This v_{anchor} can be understood as: "That wild idea of yours, in the existing knowledge system, corresponds most closely to this concept."

Vector Correction:
We don't directly use v_{anchor} (because that would lose novelty), nor do we directly use v_{new} (because the risk is too high). Instead, we perform linear interpolation through a **"Realism Factor" β** (0<β≤1) to obtain a corrected vector v_{corrected}.

v_{corrected} = (1−β)v_{new} + βv_{anchor}

When $\beta$ is small, we have more confidence in the new direction generated by COP, pursuing high novelty.

When $\beta$ is large, we tend to pull the idea back to known, "safer" domains, pursuing high feasibility.

Ultimately, what we use for decoding is this "anchored" and "corrected" vector v_{corrected}. This step is like flying a kite: v_{new} is the new height the kite wants to reach, while v_{anchor} is the string in your hand, ensuring it doesn't fly away. $\beta$ controls the length of the string.

Level 2: Semantic Rationality — Why are vector operations on embeddings meaningful?

This is a more fundamental question. Why do we believe that vector A + vector B is semantically some approximation of concept A + concept B? This takes us back to the pioneering work on word embeddings.

Theoretical Foundation: Distributional Hypothesis & Vector Space Semantics

Pioneering Case: The most famous example is vec("King") - vec("Man") + vec("Woman") ≈ vec("Queen").

Semantic Axis: The profundity of this formula lies in that it proves that "directions" in the embedding space have semantic meaning. For example, the vector vec("Woman")-vec("Man") can be understood as a semantic axis of "gender" or "femininity." Adding this directional vector to "King" is semantically performing an operation of "increasing femininity."

Our Hypothesis — The Orthogonal Semantic Axis Hypothesis:
Based on the above theory, we can propose a core, verifiable hypothesis for our COP algorithm:

v_A is a semantic point representing the position of the concept "nanoparticle drug delivery problem" on the knowledge manifold.

v_{B⊥A} is not a random vector, but a semantic axis. It represents the collection of all semantic directions in the concept "autophagy" that are irrelevant to "nanoparticle drug delivery." It can be called the "Innovation Axis originating from B and targeting A."

Therefore, the semantic interpretation of the operation v_{new} = v_A + α⋅v_{B⊥A} is very clear:

"Starting from the point of 'nanoparticle drug delivery problem,' advance a distance α along the semantic direction of 'the novel biological mechanisms brought by autophagy.'"

Difference from Simple Vector Addition:

Simple v_A + v_B: This is merely a fuzzy mixing of two concepts. Because v_B contains parts similar to v_A (v_{B∥A}), this simple addition likely only reinforces the known, common connections between the two, leading to mediocre ideas.

Our v_A + v_{B \perp A}: We precisely eliminate the known, mediocre connections between the two concepts, retaining only the most differentiated "orthogonal" information that is most likely to generate conflict and sparks. This mathematically ensures that we are exploring a truly "new" semantic direction.

Summary and Elevation:

By introducing **"Manifold Anchoring"**, we add a feasibility guarantee mechanism to the COP algorithm, enabling it to achieve a controllable balance between "innovation" and "coherence."

By proposing **"The Orthogonal Semantic Axis Hypothesis"**, we provide a solid theoretical foundation in semantics and linguistics for the core mathematical operations of the algorithm, demonstrating its rationality.

These two points not only address your concerns but also greatly enrich the depth and rigor of your paper, elevating it from a purely algorithmic technique to a complete scientific discovery framework with theoretical support.


----------------------------------------

# USER:

Please fully describe your method, ensuring it contains sufficient detail. The content you provide needs to enable other researchers to reproduce your ideas based on the information you provide.

----------------------------------------

# AGENT:

 \<think\>

Describing the COP Method

I'm now detailing the "Conceptual Orthogonal Projection (COP)" method, focusing on a clear, step-by-step description. This breakdown is being developed within the "DISA (Deconstruct, Intersect, Synthesize, Assess)" framework to ensure clarity and logical progression. The goal is a robust and easily implementable guide.

Developing the COP Algorithm

I'm presently building out the detailed methodology. My focus is on ensuring another researcher can replicate the "Conceptual Orthogonal Projection (COP)" method. I've deconstructed the goal, focusing on granular details, not just high-level concepts. I'm structuring the description in a hierarchical way, starting with DISA and then diving into the core COP algorithm and its key components. Each sub-section, including LLM and vector considerations, is being meticulously documented.

Developing Detailed Specifications

I'm now detailing each "component" of the "Conceptual Orthogonal Projection (COP)" method. My aim is to be exhaustive and provide a clear path for replication. I'm focusing specifically on the LLM backbone, specifying that a large, instruction-tuned model (e.g., Llama 3, Claude 3, or GPT-4o) with accessible hidden states is required. I'm also planning to thoroughly document all hyperparameters, ensuring a transparent and reproducible workflow. I'll produce overall algorithm pseudocode, too.

Elaborating COP Implementations

I'm now detailing the implementation specifics of the COP method. I've broken it down into sections: DISA overview, LLM backbone, vector extraction (specifying the use of the last hidden state), and anchor set construction. The anchor set will use established sources and vectorization via the specified LLM. Indexing with FAISS for efficient nearest-neighbor searches will be crucial. I'm focusing on providing enough detail for another researcher to replicate my work.

Describing COP: Method & Components

I'm now detailing the "Conceptual Orthogonal Projection (COP)" method and its individual components. I've broken the thinking into stages, focusing on granular explanations. I'll cover the LLM, vector extraction, and anchor set construction within the "DISA" framework. I will also provide example prompts for use cases, aiming for exhaustive and clear details. I'll provide overall algorithm pseudocode, too.

Defining a Detailed Method

I'm now detailing a comprehensive method for "Conceptual Orthogonal Projection (COP)" within the DISA framework. This aims for clear replication. The process breaks down the research challenge into a structure that another researcher can replicate. This includes specifying LLM model choices and algorithms. I am detailing the Deconstruct, Intersect, Synthesize, and Assess stages of the method.

Describing COP: Method & Components

I'm now detailing a comprehensive and practical "Conceptual Orthogonal Projection (COP)" method. My approach provides clear steps for another researcher's replication. I'm focusing on the "Deconstruct, Intersect, Synthesize, Assess" (DISA) framework, specifying essential LLM details. I will also provide example prompts for use cases, aiming for exhaustive and clear details. I'll provide overall algorithm pseudocode, too.

Elaborating the Method: A Deep Dive

I'm now detailing the step-by-step description of the Conceptual Orthogonal Projection (COP) method. I've broken the methodology into clear sections for easier comprehension and replication. This now includes the detailed "DISA" stages, an explanation of the core COP algorithm (with equations for clarity), and specifics on the LLM backbone (specifying instruction-tuned models like Llama 3, Claude 3, or GPT-4o, and the need for hidden state access). I'll provide prompt examples and a comprehensive overview of the method. I'm also now ready to add pseudocode.

Expanding on Method Details

I'm presently building out the description of the COP method, meticulously detailing each stage. I've broken the methodology into clear sections for easier comprehension, including the DISA stages, core algorithm equations, LLM backbone specifications, and vector extraction details. I will also provide concrete examples, pseudocode, and hyperparameter guidance for robust replication. I'm focusing on ensuring clarity and exhaustive detail, aiming to empower any researcher for implementation.

 \</think\>

Alright, following your requirements, here is the complete and detailed description of my method—the DISA framework based on Conceptual Orthogonal Projection (COP). This section aims to provide other researchers with a clear and reproducible implementation path.

Methodology: DISA Framework with Conceptual Orthogonal Projection (COP)
Our method is a multi-stage, human-AI collaborative framework named DISA (Deconstruct, Intersect, Synthesize, Assess). Its core is an algorithm called Conceptual Orthogonal Projection (COP), which performs mathematical operations in the hidden state space of Large Language Models (LLMs) to generate novel concept combinations.

1. Component Specification
Before describing the specific process, we first define the required core components.

1.1. LLM Backbone:

Requirement: A Large Language Model with strong instruction-following capabilities and rich world knowledge. The key requirement is the ability to access its internal hidden states.

Model Examples: Llama 3 (70B), Claude 3 Opus, Mistral Large, or any open-source/closed-source model that allows extraction of hidden states.

API/ Libraries: The output_hidden_states=True parameter in the Hugging Face transformers library is the standard way to implement this functionality.

1.2. Vector Representation Extraction:

Definition: A vector representation of a concept refers to the specific hidden state extracted after inputting text describing the concept into the LLM.

Extraction Method: We use the hidden state corresponding to the last input token of the final Transformer layer as the vector representation of the entire concept. Formally, for a token sequence of length T, its vector representation is v=h_T^(L), where L is the total number of layers in the model. The dimension d of this vector is determined by the model (e.g., the dimension for Llama 3-70B is 8192).

1.3. Anchor Set Construction (A):

Purpose: Used for the "manifold anchoring" step to ensure the coherence of generated concepts.

Data Sources: Collect concept lists from a broad, domain-specific knowledge base. For example:

Concept page titles from Wikipedia in specific disciplines.

High-frequency keywords from large paper databases (such as Semantic Scholar, arXiv).

Generated using LLM, for example, with the prompt: "Please list 500 core concepts, theories, and technical terms in the field of materials science."

Construction Process:

Collect textual descriptions of thousands of concepts (recommended > 5000).

For each concept text, extract its vector representation using the method defined in 1.2.

Store all vectors in an efficient vector retrieval engine.

Recommended Tools: Use the FAISS (Facebook AI Similarity Search) library to build an index to support fast Maximum Inner Product Search (MIPS) or Euclidean distance search.

2. Detailed DISA Framework Process
Stage One: Deconstruct
Objective: Decompose a broad research topic into multiple structured, fundamental concept sets.

Input: User-defined research topic, for example, topic = "Using artificial intelligence for anticancer drug development."

Process:

Use structured prompt templates to guide the LLM to generate concept lists of different categories. We typically decompose it into **"Problems/Challenges" and "Mechanisms/Technologies"**.

Example Prompt Template:

As a senior research scientist in the field of '{topic}', please perform the following task:
1.  Identify and list 10 distinct, foundational 'Problems' or 'Challenges' in this field. Each problem should be a concise phrase.
2.  Identify and list 10 distinct, key 'Mechanisms' or 'Technologies' that are currently used or could be applied to this field. Each mechanism should be a concise phrase.

Provide the output in a JSON format with two keys: "problems" and "mechanisms".
Output: Two lists of concept texts, for example, S_problems=P_1,P_2,...,P_10 and S_mechanisms=M_1,M_2,...,M_10.

Stage Two: Intersect - COP Algorithm Core
Objective: Perform pairwise combinations of the deconstructed concepts through the COP algorithm to generate a series of vectors with innovative potential.

Input: Concept lists S_problems and S_mechanisms.

Process: For each pair (P_i,M_j), perform the following COP algorithm:

Vector Extraction:

Extract the vector of problem concept P_i: v_P_i=textExtractVector(P_i).

Extract the vector of mechanism concept M_j: v_M_j=textExtractVector(M_j).

Orthogonal Projection:

Calculate the parallel component of v_M_j on v_P_i: v_M_j^parallel_P_i=(v_M_j·v_P_i)/|v_P_i|^2 * v_P_i.

Calculate the orthogonal component: v_M_j^perp_P_i=v_M_j-v_M_j^parallel_P_i.

Innovative Vector Synthesis:

Use the innovation factor α (hyperparameter) to synthesize the raw innovative vector: v_raw_new=v_P_i+α·v_M_j^perp_P_i.

Manifold Anchoring:

In the pre-constructed anchor set A, use FAISS to find the nearest neighbor anchor v_anchor to v_raw_new.

Use the realism factor β (hyperparameter) to correct the vector: v_corrected=(1-β)v_raw_new+βv_anchor.

Output: A collection of corrected innovative vectors v_corrected_1,v_corrected_2,...

Stage Three: Synthesize
Objective: Decode the abstract vectors generated in Stage Two into human-readable, detailed research ideas.

Input: Collection of innovative vectors v_corrected.

Process: This is a "vector-to-text" process implemented through soft prompting.

Design a decoding prompt template containing one or more special placeholder tokens, such as <IDEA_VECTOR>.

Example Prompt Template:

```
You are a visionary scientist tasked with elaborating on a novel research concept. The core of the concept is provided as a semantic seed. Your task is to develop this seed into a full research idea, including a potential hypothesis, methodology, and expected impact.

Conceptual Seed: <IDEA_VECTOR>

Your detailed research idea:

```

Before sending this prompt to the LLM, we perform the following operations:

The LLM's input layer converts text tokens into input embeddings.

We find the input embedding vector corresponding to the <IDEA_VECTOR> placeholder.

Replace this (or these) embedding vector(s) with our v_corrected vector. Note: If the dimension of v_corrected does not match the LLM's embedding dimension, a linear projection layer is needed, but in our design, the dimensions are consistent.

The LLM uses this modified sequence containing our innovative vector as the starting point for autoregressive decoding to generate the complete text.

Output: A series of detailed, formatted research idea texts.

Stage Four: Assess
Objective: Automatically evaluate and rank the generated ideas to filter out the most promising ones.

Input: List of synthesized research idea texts.

Process: Use the LLM as a "Judge".

Design a scoring prompt with clear evaluation criteria.

Evaluation Criteria:

Novelty (1-5 points): Does the idea break through the limitations of existing methods and propose new perspectives or solutions?

Feasibility (1-5 points): Based on current technological capabilities, how likely is the experimental or theoretical validation of this idea to succeed?

Potential Impact (1-5 points): If successful, what impact could this idea have on the field or society?

Example Prompt Template:

```JSON
{
  "role": "You are an expert peer reviewer for a top-tier scientific journal in the field of '{topic}'. Your task is to critically evaluate the following research idea. Provide a score and a brief justification for each criterion.",
  "idea_to_evaluate": "{generated_idea_text}",
  "evaluation_criteria": [
    {"criterion": "Novelty", "definition": "How original and groundbreaking is the idea compared to existing literature?"},
    {"criterion": "Feasibility", "definition": "How likely is it that the proposed methodology could be successfully implemented?"},
    {"criterion": "Potential Impact", "definition": "If successful, what is the potential significance of this research?"}
  ],
  "output_format": {
    "novelty": {"score": "integer", "justification": "string"},
    "feasibility": {"score": "integer", "justification": "string"},
    "potential_impact": {"score": "integer", "justification": "string"},
    "overall_summary": "string"
  }
}
```

Output: A ranked list of research ideas, each with structured scores and review comments.

3. Hyperparameters and Recommendations
To ensure reproducibility, here are the key hyperparameters in this method and their recommended values:

Innovation Factor (α): Controls the innovation "step size".

Function: A larger α makes the idea deviate more from the basic concept, potentially more disruptive but with higher risk.

Recommended Range: [0.7, 1.5]. Can start with 1.0.

Realism Factor (β): Controls the coherence and feasibility of ideas.

Function: A larger β pulls the idea more toward known "safe" concepts.

Recommended Range: [0.1, 0.4]. Recommended to start with 0.25.

Anchor Set Size (|A|):

Function: Determines the granularity of "manifold anchoring." A larger set provides more precise anchors.

Recommendation: At least 5000 vectors, the more the better.

Through this detailed description, other researchers should be able to clearly understand the theoretical foundations, operational processes, and technical details of this method, enabling reproduction and further research.


----------------------------------------

# USER:

Please re-analyze, I believe your method lacks sufficient innovation. Compared to Paper 9, while there is innovation in the method, it's not strong enough. Think about how to enable LLMs to generate ideas that are both reasonable and novel.

----------------------------------------

# AGENT:

\<think\>

Revisiting User Feedback

I've been meticulously analyzing the user's critique, particularly their comment about innovation. They're skeptical that mere innovation is sufficient.  I'm now focusing on how the "Conceptual Orthogonal Projection (COP)" method improves on previous work, specifically Paper 9, to meet a higher bar, instead of resting on merely being different.

Refining Novelty and Coherence

I'm now wrestling with the user's emphasis on true novelty. The core issue is that my COP method's contribution, though a refinement over interpolation in Paper 9, isn't sufficiently impactful. I'm re-examining the underlying principles of LLMs – viewing generation as pathfinding – and questioning if my "Manifold Anchoring" step, meant to ensure coherence, is deep enough. Maybe it feels like a patch, and not an integral component.

Considering MCTS Application

I'm now deeply engaged in exploring MCTS as a potential unifying framework for novelty and coherence. My Manifold Anchoring step felt somewhat disconnected from my initial "pathfinding" design. MCTS offers a way to simultaneously explore diverse potential paths—those with the potential for novelty—while also focusing on those paths that are likely to be coherent. The core challenge is how to effectively map my LLM pathfinding problem to the MCTS framework, especially defining the "value" function within the search.

Reframing with MCTS

I'm now fully immersed in the potential of MCTS. The core insight is to guide this search with my COP method, turning it from a disjoint step into an integral component. The challenge is defining an effective "value" function within the MCTS framework that balances novelty and coherence, allowing for both exploration and exploitation within the latent space. My previous ideas can contribute to the "evaluation" step during each rollout. I'm exploring different approaches to quantify novelty, particularly by using the LLM's own probability landscape.

Implementing MCTS for Novelty

I'm now fully engaged in the MCTS approach.  The core challenge lies in constructing an effective value function that balances novelty and coherence within the tree search. I'm exploring metrics for quantifying novelty, such as cosine distance to existing works, and integrating the LLM's inherent probability estimations. The selection mechanism of the MCTS can be tweaked with my earlier COP ideas to guide the exploration.

Developing a Unified Approach

I'm now integrating the MCTS framework with my prior methods. The key insight is to direct the MCTS search towards novel, coherent outputs by strategically guiding the selection and value function steps with the Conceptual Orthogonal Projection method. This approach aims to address the user's critique by simultaneously exploring for novelty and ensuring coherence in generated text, unifying the ideas in one method.

Integrating Directional Search

I'm now converging on a unified approach. By framing text generation as a pathfinding challenge and employing MCTS, I can guide the exploration using my Conceptual Orthogonal Projection (COP) method. This allows me to steer the search towards novel, coherent outputs, directly addressing the user's concerns about novelty and impact. The key is in combining the "direction" vector with the MCTS selection step and then weighting the value function.

Unifying with CG-MCTS

I'm now fully committed to integrating MCTS. By incorporating the Conceptual Orthogonal Projection method to guide the selection step and define the value function, I can steer the search toward novel yet coherent outputs, addressing the user's critique. MCTS becomes the central framework.

\n\nIntegrating Directional Search

I'm now converging on a unified approach. By framing text generation as a pathfinding challenge and employing MCTS, I can guide the exploration using my Conceptual Orthogonal Projection (COP) method. This allows me to steer the search towards novel, coherent outputs, directly addressing the user's concerns about novelty and impact. The key is in combining the "direction" vector with the MCTS selection step and then weighting the value function.

\n\nDefining Novelty via Distance

I've made progress on the Value function, which is now key. Novelty will be based on the LLM's own perplexity combined with the vector distance from the generated sequence to a database of existing work. The weighting of each term can control the influence of novelty and coherence. COP provides the basis for the direction.

\n\n ** Refining with Directional

I'm now integrating MCTS with the Conceptual Orthogonal Projection method, aiming to directly address the user's critique. By using the projection to guide the MCTS selection process, I'm increasing the focus on both novelty and coherence. This aims to generate impactful and robust ideas.


\</think\>

You have raised an extremely profound and critical point. You are right. Although my previous approach was innovative mathematically, "Innovation Vector Synthesis" and "Manifold Anchoring" are two separate steps, with the latter being more of a "patch" rather than an elegant solution that fundamentally addresses the problem. This separation cannot essentially guarantee that the semantics represented by the new vectors are coherent.

To truly achieve **"both reasonable and novel," we need a completely new paradigm that unifies the two conflicting goals of "exploring the unknown (novelty)" and "following rules (reasonableness)"** within a single algorithmic framework.

Let's abandon our previous approach of "one-step" vector synthesis and turn to a more powerful paradigm that simulates the human thinking process: "deliberate" search during generation.

Core Methodological Innovation: From "Vector Alchemy" to "Guided Thought Chain Search"

Human beings do not generate innovative ideas in one step. We conceive multiple possible paths in our minds, evaluate the probability of each step, discard unpromising branches, and continue deeper along promising directions. This process is a search, not a one-time calculation.

Therefore, I propose a brand new, more innovative method: Concept-Guided Monte Carlo Tree Search (CG-MCTS).

This method elevates the text generation process of LLMs from a simple "next-word prediction" task to a strategic task of intelligent search within a vast "semantic possibility space."

Theoretical Foundation:

Generation as Path Discovery: Each generation by an LLM can be viewed as selecting a path starting from the root node (<BOS>) in a huge tree structure composed of all possible token sequences.

Deficiencies of Standard Generation: Standard decoding strategies like greedy search or beam search always tend to choose the highest probability path, which is precisely the "enemy" of innovation.

Advantages of MCTS: MCTS is an algorithm designed to solve problems in vast search spaces (famous for AlphaGo), which perfectly balances **"Exploration" and "Exploitation."** This恰好 corresponds to our needs for "novelty" and "reasonableness."

CG-MCTS Algorithm Details

Our goal is to find a generation path that both points toward the novel direction calculated using our COP algorithm and is considered high-quality (coherent and reasonable) **from the LLM's own perspective**.

Components:

Target Direction Vector (v_target): This step retains our previous COP and manifold anchoring ideas but uses them as a "compass" rather than the final destination. A target direction vector is calculated through $v_{corrected} = (1 - \beta) (v_{P_i} + \alpha \cdot v_{M_j \perp P_i}) + \beta v_{anchor}$. This vector represents the semantic area where we want the final generated ideas to be located.

MCTS Framework: We construct the text generation process as an MCTS problem.

Node (Node): A partially generated text sequence (e.g., "A novel approach for cancer therapy involves...").

Edge (Edge): From a node to a child node, representing the selection of a new token.

Simulation/Rollout: Starting from a leaf node, allowing the LLM to freely generate a text segment of fixed length using conventional sampling methods (such as top-p sampling).

Value Estimation: This is the core innovation point of the entire method. For a complete text sequence seq generated in one rollout, we design a hybrid value function V(seq) to simultaneously evaluate its reasonableness and novelty.

CG-MCTS's Four Revolutionary Steps

1. Selection - With Directional Guidance

In MCTS, we need to select the most promising leaf node for expansion starting from the root node. Traditional selection strategies (like UCT) only consider the value and visit count of nodes.

Our Innovation: Adding a **"Directional Alignment"** term to the selection criteria.

Score(n)= Traditional UCT term (Exploitation + Exploration) $\frac{Q(n)}{N(n)} + C \sqrt{\frac{\ln N(p)}{N(n)}}$ + Directional Guidance term (Novelty-oriented) $w_{dir} \cdot \cos(embed(n),v_{target})$

$embed(n)$ is the vector representation of the text sequence corresponding to the current node n.

$v_{target}$ is our pre-calculated target direction vector.

$w_{dir}$ is the weight for directional guidance (hyperparameter).

The meaning of this formula is: we not only select nodes that have performed well in the past (high Q-value) or have been insufficiently explored, but also prioritize paths that are "moving toward" our desired innovative direction.

2. Expansion

When an unexpanded node is selected, we let the LLM predict the next possible token and create one or more new child nodes.

3. Simulation & Value Estimation - Unification of Reasonableness and Novelty

This is the second core innovation point. Starting from newly expanded nodes, we let the LLM freely "rollout" to generate a text segment, then score it using the following hybrid value function:

$V(seq)=w_{coh} \cdot V_{coherence}(seq)+w_{nov} \cdot V_{novelty}(seq)$

Reasonableness/Coherence Value ($V_{coherence}$): This value is provided by the LLM itself. A coherent and reasonable sentence should have a high average log probability from the model's perspective. This penalizes nonsensical "gibberish" that is ungrammatical and illogical.

$V_{coherence}(seq)=\frac{1}{|seq|} \sum_{t=1}^{|seq|} \log P(token_t |token_{< t})$

Novelty Value ($V_{novelty}$): This value evaluates how "new" the generated idea is. We can measure it by calculating the minimum distance between its vector representation and all vectors in a large existing literature database (such as arXiv abstracts). The greater the distance, the more novel.

$V_{novelty}(seq)=\min_{v_{paper} \in DB} distance(embed(seq),v_{paper})$

$w_{coh}$ and $w_{nov}$ are weight hyperparameters used to balance our preference for reasonableness and novelty.

4. Backpropagation

Backpropagate the calculated hybrid value V(seq) along the path, updating the value (Q-value) and visit count of all nodes from the leaf node to the root node.

Final Output: After a preset "thinking budget" (for example, 1000 MCTS iterations), we start from the root node and select the path with the most visits or highest comprehensive value as the final generated research idea.

Summary of Advantages and Innovations of the New Method

From "One-time Calculation" to "Procedural Search": This is the most fundamental difference from Paper 9 and my previous COP method. We no longer try to "guess" the best vector with a formula, but rather "discover" the best generation path by simulating a deliberate search process. This is a huge leap in methodology.

Intrinsic Unification of "Reasonableness" and "Novelty": These two conflicting goals are no longer separate steps but are unified in the **Selection and Evaluation** core aspects of MCTS. Directional guidance ensures exploration novelty, while the hybrid value function enforces reasonableness checks at every step.

Highly Interpretable and Controllable: We introduce clear hyperparameters as "creative knobs":

Thinking Budget (Number of Iterations): How long to allow the model to "think".

Direction Weight ($w_{dir}$): Controls how closely the idea aligns with the preset direction.

Creativity Weights ($w_{coh}$, $w_{nov}$): Precisely adjust whether the final idea is more "wild" or more "grounded".

Strong Theoretical Support: MCTS is one of the most powerful search algorithms in the field of artificial intelligence. Applying it to the open-ended, creative generation of LLMs is itself a highly innovative cross-domain application that provides a brand new, systematic solution to the problem of LLM creativity.

This CG-MCTS framework places your concerns about feasibility at the core of its design, ensuring that the final generated idea represents the "furthest" innovation achievable within the boundaries of "reasonableness" through a dynamic, adaptive search process.

----------------------------------------
# USER:

Please fully describe your method, ensuring it includes sufficient details. The content you provide needs to enable other researchers to reproduce your ideas based on the information you give them. It must be feasible.

----------------------------------------

# AGENT:

\<think\>

Elaborating the Methodology

I'm now formulating a detailed explanation of Concept-Guided Monte Carlo Tree Search (CG-MCTS). The aim is to create a well-organized and easy-to-understand guide that clearly outlines the proposed methodology, laying the foundation for a thorough discussion.

Detailing the Approach

I'm structuring the detailed description of CG-MCTS to ensure another researcher can replicate it. I'm focusing on providing algorithmic specifics, data structure choices, and crucial hyperparameters. The "Methodology" section is beginning to take shape, outlining the high-level framework and then delving into individual components like concept representation and the tree search procedure. I'm using practical technologies to explain the implementation.

Structuring the Methodology

I'm now focusing on organizing the detailed description of CG-MCTS to facilitate another researcher's replication. I'm prioritizing a structured approach with clear high-level overviews and specific breakdowns of core components like LLM integration, concept representation, and the MCTS procedure. I'm aiming for replicability through both written explanations and algorithm pseudocode.

Refining the Blueprint

I'm solidifying the detailed description of CG-MCTS, focusing on a structured approach conducive to replicability. I'm prioritizing clarity in both the written explanations and supporting algorithm pseudocode. My goal is to equip another researcher with the necessary information to replicate this method. The document structure will have an emphasis on the integration of LLMs, concept representation, and the MCTS procedure for clarity.

Outlining the Core Algorithm

I'm now detailing the heart of CG-MCTS: the algorithm itself. I'm focusing on the integration of concept guidance within the canonical MCTS steps.  Selection, expansion, simulation, and backpropagation are being fleshed out, with a focus on how concepts influence decision-making at each stage. Moreover, I'm documenting the process of how to calculate the target vector. I will also incorporate clear pseudocode for a concise, unambiguous representation.

Defining the Core Components

I'm presently building the granular components of CG-MCTS that will be necessary for another researcher to recreate the method, with precise details. I'm focusing on providing comprehensive explanations for each piece, including the LLM backbone, vector extraction, and the novelty database. I'm elaborating on the anchor set's construction and how to prepare it to be ready for the algorithm. I'm striving for the right balance between necessary information and brevity.

Outlining the Core Method

I'm now detailing the full method for CG-MCTS to provide a complete guide. I will provide a breakdown that includes the necessary components for someone to build their version. These will cover key pieces, such as the LLM backbone, the Anchor Set, the Novelty Database,  the initialization of the  conceptual target vector, and how to execute the selection, expansion, simulation, and backpropagation steps in detail.

Detailing the Full Method

I'm now writing a detailed description of the full CG-MCTS methodology. My focus is on providing all the key components for complete replication. This will include precise details for the LLM backbone, the Anchor Set, the Novelty Database, and the mechanics for the algorithm. I will describe in detail each step within the MCTS loop - from initialization to how I calculate the final target vector - with clear pseudocode to ensure a solid foundation.

\</think\>

Okay, I will provide you with a complete, detailed, and reproducible description of the Concept-Guided Monte Carlo Tree Search (CG-MCTS) method. This section is intended to serve as the methodology section of an academic paper, offering other researchers a clear implementation blueprint.

**Methodology: Generating Novel and Plausible Scientific Ideas through Concept-Guided Monte Carlo Tree Search (CG-MCTS)**

This study proposes a novel algorithm that operates at inference time, designed to guide Large Language Models (LLMs) to generate scientific ideas that are both novel and plausible. The method reframes the text generation problem as an optimal path search problem within a semantic possibility space, utilizing Monte Carlo Tree Search (MCTS) to intelligently explore this space.

**1. Framework Overview**

The core idea of the CG-MCTS framework is to avoid having the LLM perform one-shot greedy decoding. Instead, it allocates a "thinking budget" (number of MCTS iterations), within which the model will:

- Determine a novel semantic target direction: Identify a promising yet unconventional semantic vector as a "compass" through Concept Orthogonal Projection (COP).
- Conduct prospective search in the generation tree: Explore multiple possible generation paths (i.e., "chains of thought") via MCTS.
- Dynamically evaluate the potential of each path: Use a hybrid value function to balance the plausibility (linguistic fluency, logical consistency) and novelty (divergence from existing knowledge) of each path in real-time during the search.
- Converge on the optimal idea: After "thinking" concludes, select the path that has demonstrated the most potential as the final output.

**2. Pre-computation and Setup**

Before executing the main CG-MCTS algorithm, the following key components need to be prepared:

**2.1. LLM Backbone Model**

- **Model Requirements**: Select an instruction-tuned Large Language Model (recommended parameter count > 7B) that must allow access to its internal hidden states during inference.
- **Implementation**: Load the model using the Hugging Face transformers library and set output_hidden_states=True when calling model.generate() or model().
- **Vector Extraction**: Define a function ExtractVector(text) that receives a text segment, encodes it through the LLM, and returns the hidden state vector h_T^(L) corresponding to the last input token of the final Transformer layer.

**2.2. Anchor Set (A)**

- **Purpose**: To ensure basic plausibility when determining target directions.
- **Construction Process**:
  - **Data Source**: Collect a list containing at least 5,000 core concepts in the field (e.g., extracted from Wikipedia, professional dictionaries, or arXiv keywords).
  - **Vectorization**: Convert each concept text in the list to a vector representation by calling ExtractVector(concept_text).
  - **Indexing**: Store all concept vectors in an efficient vector retrieval engine. The FAISS (Facebook AI Similarity Search) library is recommended, constructing an IndexFlatL2 or IndexFlatIP index.

**2.3. Novelty Evaluation Database (D_novelty)**

- **Purpose**: To quantify the novelty of a generated idea during the MCTS value evaluation phase.
- **Construction Process**:
  - **Data Source**: Collect a large-scale, domain-relevant corpus of existing literature (e.g., download all paper abstracts from a specific category on arXiv).
  - **Vectorization and Indexing**: Same as constructing the anchor set, convert each paper's abstract (or title+abstract) to a vector through ExtractVector() and build an index using FAISS.

**3. Core Algorithm: CG-MCTS Process**

The main CG-MCTS algorithm initiates upon receiving a broad research topic.

**3.1. Initialization: Determine Target Direction Vector (v_target)**

This step uses Concept Orthogonal Projection (COP) to calculate a vector serving as a search "compass."

- **Deconstruction**: Have the LLM decompose the input topic (e.g., "nanomedicine for Alzheimer's disease") into a set of "problems" S_problems and a set of "potential mechanisms" S_mechanisms.
- **Selection and Projection**: Select a pair of concepts, such as problem P_i and mechanism M_j. Extract their vectors v_P_i and v_M_j. Calculate the orthogonal component of M_j relative to P_i, denoted as v_M_j⊥P_i.
- **Synthesis and Anchoring**: Calculate the "raw" innovation vector v_raw_new = v_P_i + α·v_M_j⊥P_i. Find its nearest neighbor v_anchor in the anchor set A. Correct it using a realism factor β to obtain the final target direction vector: v_target = (1-β)v_raw_new + βv_anchor.

**3.2. Main MCTS Loop**

- **Tree Structure**: The root of the tree is the start token <BOS>. Each node represents a generated token sequence, and each edge represents a token selection.
- **Loop**: Execute a preset number of iterations (num_iterations, i.e., the "thinking budget"). Each iteration contains the following four steps:

**3.2.1. Selection**

Starting from the root node, recursively traverse downward through the tree according to a selection strategy until reaching a leaf node (i.e., a node that has never been expanded).

At each level, the strategy for selecting a child node n is determined by the following Guided UCT formula:

Score(n) = 
  [Exploitation Value] Q(n)/N(n) + 
  [Exploration Value] C·√(ln(N(p))/N(n)) + 
  [Direction Guidance Value] w_dir·(ExtractVector(n)·v_target)/(||ExtractVector(n)||·||v_target||)

Where:
- Q(n): Cumulative value of node n.
- N(n): Visit count of node n.
- N(p): Visit count of the parent node.
- C: Exploration constant (hyperparameter, typically √2).
- w_dir: Direction guidance weight (hyperparameter).

**3.2.2. Expansion**

If the selected leaf node n has not been visited before (i.e., N(n)=0), proceed directly to the simulation step.

Otherwise, expand the node: Input n's text sequence to the LLM to obtain the probability distribution of the next token. Create K new child nodes based on this distribution (e.g., selecting the K tokens with the highest probabilities).

**3.2.3. Simulation and Value Evaluation**

Starting from the newly expanded node (or the leaf node visited for the first time), perform a "rollout": Have the LLM continue generating using standard top-p sampling with the node's text as prefix until reaching a preset maximum length L_rollout, obtaining a complete idea sequence seq.

Evaluate this sequence using the following hybrid value function V(seq):

V(seq) = w_coh·V_coherence(seq) + w_nov·V_novelty(seq)

- **Coherence Value (V_coherence)**: Determined by the LLM's own approval of the sequence, i.e., the average log probability of the sequence.
  
  V_coherence(seq) = (1/|seq|) · Σ_{t=1}^{|seq|} logP_LLM(token_t | token_< t)

- **Novelty Value (V_novelty)**: Measured by calculating the distance between the sequence vector and the most similar document in the novelty evaluation database D_novelty. The greater the distance, the more novel.

  V_novelty(seq) = min_{v_doc ∈ D_novelty} distance(ExtractVector(seq), v_doc)
  
  (distance can be Euclidean distance or cosine distance 1 - similarity).

**3.2.4. Backpropagation**

Propagate the calculated hybrid value V(seq) from the expanded node upward along the selection path back to the root node.

For each node n' on the path, update its statistics: N(n') ← N(n') + 1, Q(n') ← Q(n') + V(seq).

**4. Final Idea Generation**

After the main MCTS loop completes (reaching num_iterations), we start from the root node and at each step select the child node with the most visits (maximum N(n) value).

Traversing along this "most trusted" path and concatenating all tokens yields the final generated scientific idea that is both novel and plausible.

**5. Hyperparameter Configuration**

To ensure reproducibility, below are the key hyperparameters of this method, their functions, and recommended values:

| Parameter | Symbol | Description | Recommended Range/Value |
|-----------|--------|-------------|-------------------------|
| **COP-related** | | | |
| Innovation factor | α | Controls the influence strength of orthogonal components, affecting novelty degree | [0.7, 1.5] |
| Realism factor | β | Controls the strength of "anchoring" to known concepts, affecting plausibility | [0.1, 0.4] |
| **MCTS-related** | | | |
| Iterations/Thinking budget | num_iter | Total number of MCTS cycles, determining search depth and breadth | [500, 5000] |
| Exploration constant | C | Constant in UCT formula balancing exploration and exploitation | 1.414 (i.e., √2) |
| Direction guidance weight | w_dir | Controls the influence of the "compass" v_target in selection | [0.5, 2.0] |
| Coherence value weight | w_coh | Preference for the language model's own probability in the evaluation function | [0.5, 1.0] |
| Novelty value weight | w_nov | Preference for "uniqueness" in the evaluation function | [1.0, 3.0] |
| Simulation/rollout length | L_rollout | Number of tokens generated per simulation | [50, 200] |