windows下装unsloth,deepspeed==0.15.1
https://github.com/leejohannes/unsloth_Windows_conda
思考过程很混乱,请参考上面发在github上的安装过程。
目前按照
https://github.com/unslothai/unsloth
提示的
xfrom安装不到合适的版本
pip install torch torchvision torchaudio triton
然后triton也没有合适的windows版本而且从2.0版本就开始这样了
https://blog.csdn.net/ddrfan/article/details/130127401
https://hf-mirror.com/madbuda/triton-windows-builds
也有人放了所谓的3.0.0,我没试。。。
https://github.com/triton-lang/triton
发现也是不支持windows但是脚本中又是有ps1之类的windows下的环境设置文件
git clone https://github.com/triton-lang/triton.git; cd triton; python -m venv .venv --prompt triton; source .venv/bin/activate; pip install ninja cmake wheel pybind11; # build-time dependencies pip install -e python
因为是windows就不可能source source .venv/bin/activate
而且实际git下来的包.\.venv\Scripts\Activate.ps1这个位置才对。。。
然后发现运行的时候要找linux的cuda。。。
https://anaconda.org/nvidia/cuda-nvcc/
conda install nvidia::cuda-nvcc
linux上需求的是12.4.99
先在conda下装cuda
https://anaconda.org/nvidia/cuda-nvcc/files
conda上还真有12.4.99版本的
所以改变python/setup.py中download_and_copy中的连接设置
495行:
但是还有路径的反斜杠的错误
490行:
bin/ptxas
改成r"bin\ptxas"因为一个反斜杠在python中是要输出特殊字符。。。
DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
呃。。。3.12解压又出问题。。。
https://docs.pythonlang.cn/3/library/tarfile.html
在download_and_copy的解压中加入,filter='data'用来避免警告
还是复制不道文件,解压了。更新新版本中文件路径名
调整相对路径。。。
然后就报LLVM不支持windows的问题了。。。继续处理中
https://github.com/llvm/llvm-project/releases/download/llvmorg-18.1.8/LLVM-18.1.8-win64.exe
但这个是安装文件。。。
https://github.com/vovkos/llvm-package-windows
最后参考这个个在conda 3.11下安装了。。。
从这里从新开始把
管理员方式运行Anaconda Powershell Prompt
其实不用管理员应该也可以,但是我的conda是安装给全部用户的担心没有管理员会遇到读写问题。。。
conda create -p 'unsloth目录' `
python=3.11 `
pytorch-cuda=12.1 `
pytorch cudatoolkit -c pytorch -c nvidia `
-y
这是根据官网的
conda create --name unsloth_env \ python=3.11 \ pytorch-cuda=12.1 \ pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers \ -y
改的,目录可以是你自己要安装的位置,如果你C盘够用就用官网的-n方式
conda conda 24.7.1中提示安装的是这些包:
blas pkgs/main/win-64::blas-1.0-mkl
bzip2 pkgs/main/win-64::bzip2-1.0.8-h2bbff1b_6
ca-certificates pkgs/main/win-64::ca-certificates-2024.7.2-haa95532_0
cuda-cccl nvidia/win-64::cuda-cccl-12.4.127-0
cuda-cudart nvidia/win-64::cuda-cudart-12.1.105-0
cuda-cudart-dev nvidia/win-64::cuda-cudart-dev-12.1.105-0
cuda-cupti nvidia/win-64::cuda-cupti-12.1.105-0
cuda-libraries nvidia/win-64::cuda-libraries-12.1.0-0
cuda-libraries-dev nvidia/win-64::cuda-libraries-dev-12.1.0-0
cuda-nvrtc nvidia/win-64::cuda-nvrtc-12.1.105-0
cuda-nvrtc-dev nvidia/win-64::cuda-nvrtc-dev-12.1.105-0
cuda-nvtx nvidia/win-64::cuda-nvtx-12.1.105-0
cuda-opencl nvidia/win-64::cuda-opencl-12.4.127-0
cuda-opencl-dev nvidia/win-64::cuda-opencl-dev-12.4.127-0
cuda-profiler-api nvidia/win-64::cuda-profiler-api-12.4.127-0
cuda-runtime nvidia/win-64::cuda-runtime-12.1.0-0
cudatoolkit nvidia/win-64::cudatoolkit-11.5.0-hfde6d1b_9
filelock pkgs/main/win-64::filelock-3.13.1-py311haa95532_0
gmpy2 pkgs/main/win-64::gmpy2-2.1.2-py311h7f96b67_0
intel-openmp pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46320
jinja2 pkgs/main/win-64::jinja2-3.1.4-py311haa95532_0
libcublas nvidia/win-64::libcublas-12.1.0.26-0
libcublas-dev nvidia/win-64::libcublas-dev-12.1.0.26-0
libcufft nvidia/win-64::libcufft-11.0.2.4-0
libcufft-dev nvidia/win-64::libcufft-dev-11.0.2.4-0
libcurand nvidia/win-64::libcurand-10.3.5.147-0
libcurand-dev nvidia/win-64::libcurand-dev-10.3.5.147-0
libcusolver nvidia/win-64::libcusolver-11.4.4.55-0
libcusolver-dev nvidia/win-64::libcusolver-dev-11.4.4.55-0
libcusparse nvidia/win-64::libcusparse-12.0.2.55-0
libcusparse-dev nvidia/win-64::libcusparse-dev-12.0.2.55-0
libffi pkgs/main/win-64::libffi-3.4.4-hd77b12b_1
libnpp nvidia/win-64::libnpp-12.0.2.50-0
libnpp-dev nvidia/win-64::libnpp-dev-12.0.2.50-0
libnvjitlink nvidia/win-64::libnvjitlink-12.1.105-0
libnvjitlink-dev nvidia/win-64::libnvjitlink-dev-12.1.105-0
libnvjpeg nvidia/win-64::libnvjpeg-12.1.1.14-0
libnvjpeg-dev nvidia/win-64::libnvjpeg-dev-12.1.1.14-0
libuv pkgs/main/win-64::libuv-1.48.0-h827c3e9_0
markupsafe pkgs/main/win-64::markupsafe-2.1.3-py311h2bbff1b_0
mkl pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46358
mpc pkgs/main/win-64::mpc-1.1.0-h7edee0f_1
mpfr pkgs/main/win-64::mpfr-4.0.2-h62dcd97_1
mpir pkgs/main/win-64::mpir-3.0.0-hec2e145_1
mpmath pkgs/main/win-64::mpmath-1.3.0-py311haa95532_0
networkx pkgs/main/win-64::networkx-3.3-py311haa95532_0
openssl pkgs/main/win-64::openssl-3.0.15-h827c3e9_0
pip pkgs/main/win-64::pip-24.2-py311haa95532_0
python pkgs/main/win-64::python-3.11.9-he1021f5_0
pytorch pytorch/win-64::pytorch-2.4.1-py3.11_cuda12.1_cudnn9_0
pytorch-cuda pytorch/win-64::pytorch-cuda-12.1-hde6ce7c_5
pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cuda
pyyaml pkgs/main/win-64::pyyaml-6.0.1-py311h2bbff1b_0
setuptools pkgs/main/win-64::setuptools-72.1.0-py311haa95532_0
sqlite pkgs/main/win-64::sqlite-3.45.3-h2bbff1b_0
sympy pkgs/main/win-64::sympy-1.13.2-py311haa95532_0
tbb pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0
tk pkgs/main/win-64::tk-8.6.14-h0416ee5_0
typing_extensions pkgs/main/win-64::typing_extensions-4.11.0-py311haa95532_0
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
vc pkgs/main/win-64::vc-14.40-h2eaa2aa_1
vs2015_runtime pkgs/main/win-64::vs2015_runtime-14.40.33807-h98bb1dd_1
wheel pkgs/main/win-64::wheel-0.43.0-py311haa95532_0
xz pkgs/main/win-64::xz-5.4.6-h8cc25b3_1
yaml pkgs/main/win-64::yaml-0.2.5-he774522_0
zlib pkgs/main/win-64::zlib-1.2.13-h8cc25b3_1
不加xformers因为windowsconda没有单独的xformer频道。。。
之后
conda activate 'unsloth目录'
pip install xformers
xformers-0.0.27.post2-cp311-cp311-win_amd64.whl
torch-2.4.0-cp311-cp311-win_amd64.whl
numpy-2.1.1-cp311-cp311-win_amd64.whl
涉及到这些包,torch要降级成2.4.0,当然目前都是自动的
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
官网给的是这个,需要把git的bin目录加到path里面不然找不到git
除了unsloth还安装了hf-transfer-0.1.8 protobuf-3.20.3 sentencepiece-0.2.0 unsloth-2024.8
pip install --no-deps trl peft accelerate bitsandbytes
安装了这些包:
accelerate-0.34.2 aiohappyeyeballs-2.4.0 aiohttp-3.10.5 aiosignal-1.3.1 attrs-24.2.0 bitsandbytes-0.43.3 certifi-2024.8.30 charset-normalizer-3.3.2 colorama-0.4.6 datasets-3.0.0 dill-0.3.8 docstring-parser-0.16 frozenlist-1.4.1 fsspec-2024.6.1 huggingface-hub-0.24.6 idna-3.8 markdown-it-py-3.0.0 mdurl-0.1.2 multidict-6.1.0 multiprocess-0.70.16 packaging-24.1 pandas-2.2.2 peft-0.12.0 psutil-6.0.0 pyarrow-17.0.0 pygments-2.18.0 python-dateutil-2.9.0.post0 pytz-2024.2 regex-2024.9.11 requests-2.32.3 rich-13.8.1 safetensors-0.4.5 shtab-1.7.1 six-1.16.0 tokenizers-0.19.1 tqdm-4.66.5 transformers-4.44.2 trl-0.10.1 tyro-0.8.10 tzdata-2024.1 urllib3-2.2.2 xxhash-3.5.0 yarl-1.11.1
找到下载的trion https://hf-mirror.com/madbuda/triton-windows-builds
然后
pip install triton-2.1.0-cp311-cp311-win_amd64.whl
其实安装顺序没什么区别。。。
deepspeed其实跟unsloth功能类似,只是安装unsloth没必要安装deepspeed,主要解决模型过大显存不够的问题
ZeRO3的情况下,可以将内存“虚拟化”为显存,参考自https://www.cnblogs.com/alphainf/p/18003689
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install deepspeed
还是出library_dir += [os.path.join(os.environ.get("CUDA_PATH"), "lib", "x64")]
找不到cuda
python -c "import torch; print('Using device: ', torch.device('cuda' if torch.cuda.is_available() else 'cpu'))";
用上面的测试了下显示Using device: cuda
然后装了cuda驱动重启重新安装。。。
Installed CUDA version 12.6 does not match the version torch was compiled with 12.1, unable to compile cuda/cpp extensions without a matching cuda version.
嗯。。。需要12.1的。。。12.6还装不上去。。。
所以是xformers需要2.4.0的torch所以需要cuda的版本是121。。。
https://developer.nvidia.com/cuda-12-1-0-download-archive?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local
装完要重启的
只装runtime中的libaraies和development就可以了
$DS_BUILD_OPS=0
还有个错误是这个。。。
conda install nvidia/label/cuda-12.1.0::cuda-nvcc
在conda环境下安装cuda12.1。。。。
pip install deepspeed --no-clean
去提示错误的setup.py改掉
$Env:DS_BUILD_OPS = 0
环境变量得这么设置。。。在powershell中
又有新的错误了。。。。
bin\\deepspeed.bat
这个找不到。。。
去里面下载 https://github.com/microsoft/DeepSpeed/tree/master/bin
pip install deepspeed --no-clean
去把之前的没装完按照错误提示的temp路径复制进去
然后再在conda powershell中cd过去
运行
python .\setup.py install
推荐阅读: