# Jigsaw3D Project

A 3D style generation pipeline that combines multi-view image generation with texture back-projection.

To set up and run the Jigsaw3D project, you'll need to follow a series of steps to configure the environment, download necessary model checkpoints, and prepare the input data. You also need a GPU with at least 22GB GPU for inference.

## Environment Setup

### 1. Create Conda Environment
```bash
conda create -n jigsaw3D python=3.10
conda activate jigsaw3D
```

### 2. Install PyTorch
Install the appropriate version for your CUDA setup (example for CUDA 11.8):
```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
```

### 3. Install Project Dependencies
```bash
pip install -r requirements.txt
```

### 4. Install CV-CUDA
Follow the official installation guide for CV-CUDA:
- [CV-CUDA Installation Guide](https://github.com/CVCUDA/CV-CUDA?tab=readme-ov-file#installation)

## Model Checkpoints Setup

Before running inference, download and place the following model checkpoints:

### Required Checkpoints:

1. **Jigsaw3D Model**
   - Download from [Google Drive](https://drive.google.com/file/d/1T7d0Ttb5_tpDBmkFgyc8-9sI4-R2LJx-/view?usp=sharing) and place in: `../jigsaw3D_ckpt/image_shuffle_patch_size_64/mp_rank_00_model_states.pt`

2. **SDXL Base Model**
   - Source: [HuggingFace - stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
   - Save to: `../style-pretrained-ckpts/stable-diffusion-xl-base-1.0/`

3. **SDXL VAE Model**
   - Source: [HuggingFace - madebyollin/sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix)
   - Save to: `../style-pretrained-ckpts/sdxl-vae-fp16-fix/`

4. **Real-ESRGAN Upscaler**
   - Source: [Google Drive - RealESRGAN_x2plus.pth](https://drive.google.com/file/d/1N8fkenrm5XgA-MGhWMOrbZRktMTcrqrt/view?usp=sharing)
   - Save to: `../style-pretrained-ckpts/RealESRGAN_x2plus.pth`

5. **LaMa Inpainting Model**
   - Source: [Google Drive - big-lama.pt](https://drive.google.com/file/d/1DMzAiYxEPcQNraNCuvjOFXahTh17r9kv/view?usp=sharing)
   - Save to: `../style-pretrained-ckpts/big-lama.pt`

### Test Data:
Download test data from:
- [Google Drive - Test Dataset](https://drive.google.com/drive/folders/1-p0mBDeV8O1pA54VZDMRXhbidUb1e0c4?usp=drive_link)
- Extract to: `./input/test/`

## Quick Start

### Run Texture Generation
```bash
bash 3Dstyle_generation.sh
```

## Configuration Options

### Background Processing Strategies:
- `bg_padding`: Recommended for landscape/scene images (preserves background context)
- `bg_remove`: Recommended for object-focused images (isolates main object)

### Key Parameters:
- `--img_patch_size`: Size of image patches for processing (default: 128)
- `--is_image_shuffle`: Enable image patch shuffling during inference
- `--skip_existing`: Skip processing if output files already exist
- `--exclude_str`: Exclude mesh files containing specified string
- `--jigsaw3D_path`: Path to Jigsaw3D model checkpoint
- `--upscaler_ckpt_path`: Path to Real-ESRGAN upscaler model
- `--inpaint_ckpt_path`: Path to LaMa inpainting model


## Project Structure
```
./
├── scripts/
│   └── texture_i2tex.py      # Main texture generation script
├── input/
│   └── test/                 # Test data directory
│       ├── mesh/             # 3D mesh files (.glb, .obj)
│       └── reference/        # Reference image files (.jpg, .jpeg, .png)
├── outputs_glb/
│   └── test/                 # Output directory for generated models
├── ../jigsaw3D_ckpt/         # Jigsaw3D model checkpoints
└── ../style-pretrained-ckpts/ # Pretrained model checkpoints
```

Thanks to [MV-Adapter](https://github.com/huanngzh/MV-Adapter). Our code is built on their work.

这段话加上推理至少需要22G 的GPU