# Scouting for Potential LLMs: A Preliminary Assessment of Domain Adaptability for Supervised Fine-Tuning

This repository contains supplementary materials accompanying the paper:

**Scouting for Potential LLMs: A Preliminary Assessment of Domain Adaptability for Supervised Fine-Tuning**  
---

## Overview

Large Language Models (LLMs) have shown impressive performance across various NLP tasks. However, their domain suitability for supervised fine-tuning (SFT) remains a critical factor in real-world applications.  
This supplementary material provides executable scripts and notebooks that demonstrate the **Potential Scout** framework, which assesses a model’s fine-tuning potential without additional training.

---

## File Structure and Execution Order

Please run the files in the following order:

1. **`1_select_k.py`**
Determines the appropriate resampling length for normalizing Thinking Curves across different model depths.

2. **`2_get_TCM.py`**
Extracts Thinking Curves (TC), resamples them to the determined length, and saves as Thinking Curve Matrix (TCM) in the result/ directory.

3. **`3_experiments.ipynb`**  
   Runs the main experiments, including:
   - Extraction of diagnostic metrics (AGS: Activation Growth Score, LCS: Layer Coverage Score).
   - Evaluation through **In-dataset Scout (IDS)** and **Cross-dataset Scout (CDS)**.  
   - Comparative analysis across multiple models.

---

## Requirements & Usage

- Python 3.10  


Install requirements with:

```bash
pip install -r requirements.txt

python 1_select_k.py
python 2_get_TCM.py

jupyter notebook 3_experiments.ipynb
```