# Install Falsh Attention

To install:

1. Make sure that PyTorch is installed.
2. Make sure that packaging is installed (`pip install packaging`)
3. Make sure that ninja is installed and that it works correctly (e.g. `ninja --version` then `echo $?` should return exit code 0). If not (sometimes `ninja --version` then `echo $?` returns a nonzero exit code), uninstall then reinstall `ninja` (`pip uninstall -y ninja && pip install ninja`). Without `ninja`, compiling can take a very long time (2h) since it does not use multiple CPU cores. With ninja compiling takes 3-5 minutes on a 64-core machine.

Then:
```bash
pip install flash-attn --no-build-isolation
```