# Introduction
These are the soource codes used for in-context generating tabular data given a benchmark dataset. 


# Benchmark Datasets
The in-context tabular data generation is dependent on benchmark datasets which are available in "data/" folder (downloaded at [ANONYMIZED_DATASET_URL])


# Parameters
The prompts are given  
--seed: the random seed for reproducing the results  
--sim_seed: the random seed used for finding the benchmark datasets, of which the value is from 100 to 109.  
--cm: the name of the causal mechanism, e.g., 'lu' is for linear uniform distributions. Used together with sim_seed for finding the benchmark dataset.  
--bt: how many times the llm is called for generating text.  
--max_table_rows: the number of rows of the benchmark data which are used as in-context learning.  
--max_new_tokens: the number of token for text generation.  
--prow_num: the number of rows that llms are required for generating tabular data.  
--llm: the name of llms.  

e.g., 
```python Qwen/Qwen2-7B-Instruct/tab_eval.py --llm qwen --sim_seed 103 --max_table_rows 100 --bt 10 --max_new_tokens 10000 --prow_num 100 ```

# Evaluation of LLM results

```python eval_llms/eva_causal_graph_adj_cdir.py --cm lu --task_type graph_adj --llm mistral```
```python eval_llms/eva_causal_graph_adj_cdir.py --cm lu --task_type graph_cdir --llm mistral```
```python eval_llms/eva_causal_graph_dsep.py --cm lu --input_type graph --llm mistral```



# Usage
* Generated tabular data: Given the benchmark dataset with causal graph and tabular data in " data/sim_lu/100", the corresponding generated tabular data should be saved in directory: "synthetic/sim_lu/100/qwen_100i.csv".  
* Evaluation results: the evaluated results will be saved in "results/qwen_100i" according to different LLMs.

# Example
* causal skeleton: python card_gt/eva_skeleton_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105  
* d-sepearation: python card_gt/eva_ci_sets_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105  
* causal direction: python card_gt/eva_bcd_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105  
* causal graphs: python card_gt/eva_cdir_lingam_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105 
* interventional inference: python card_gt/eva_intervention_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105
* counterfactural inference: python card_gt/eva_counterfactural_llm.py --cm lu --sz 400 --llm qwen_100i --seed_sim 105
