Scores: Accuracy / F1 Score.
| Model Grouping | Model Name | CFA Level 1 | CFA Level 2 | CFA Level 3 | CPA REG |
|---|---|---|---|---|---|
| Financial Models | Model | - | - | - | - |
| Base Models | Llama 3.1 8Bopen_in_new | 13.33 0.133 |
19.48 0.199 |
16.67 0.179 |
31.68 0.317 |
| Llama 3.1 70Bopen_in_new | 42.22 0.418 |
29.87 0.303 |
24.36 0.271 |
41.58 0.426 |
|
| DeepSeek V3open_in_new | 54.44 0.556 |
46.75 0.485 |
47.44 0.496 |
65.35 0.654 |
|
| GPT-4oopen_in_new | 63.33 0.631 |
55.84 0.563 |
51.28 0.517 |
67.33 0.667 |
|
| Gemini 2.0 FLopen_in_new | 55.56 0.556 |
56.67 0.567 |
52.56 0.538 |
63.37 0.638 |
|
| Fine-tuned Models | Llama 3.1 8B LoRA | 86.67 0.867 |
88.31 0.883 |
70.51 0.705 |
80.20 0.802 |
| Llama 3.1 8B QLoRA | 87.78 0.878 |
83.12 0.835 |
66.67 0.675 |
88.12 0.885 |
|
| Llama 3.1 8B DoRA | 87.78 0.878 |
90.91 0.909 |
69.23 0.697 |
90.10 0.901 |
|
| Llama 3.1 8B rsLoRA | 87.78 0.878 |
92.21 0.922 |
79.49 0.795 |
90.10 0.901 |
|
| Gemini 2.0 FL N/A | 52.22 0.530 |
51.11 0.519 |
51.28 0.557 |
51.28 0.557 |