Bella's Blog
0%

Windows+anaconda+4050 6G+chatglm本地部署一

发表于 更新于 分类于 Valine：
本文字数： 2.3k 阅读时长 ≈ 2 分钟

本系列主要介绍如何在本地单卡运行chatglm大模型，并实现一些有趣的应用，实践过程中出现的问题解决方法等都进行记录和总结。

1. 环境创建

1
2
3
4
5
git clone https://github.com/THUDM/ChatGLM-6B.git
cd ChatGLM-6B
conda create --name chatglm python=3.10 #3.12会出现奇奇怪怪的问题，这里建议还是用3.10
conda activate chatglm
pip install -r requirements.txt  -i https://pypi.tuna.tsinghua.edu.cn/simple

这里注意可能会遇到ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
      running build_ext
      running build_rust
      error: can't find Rust compiler

      If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

      To update pip, run:

          pip install --upgrade pip

      and then retry package installation.

      If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

原因是需要安装rust.

安装链接参考https://blog.csdn.net/md521/article/details/108110676

在添加环境变量时注意默认安装路径是

1
C:\Users\XXXX\.cargo\bin

添加环境变量后新开promt检查rust是否安装成功

2. chatglm-6b-int4

在huggingface下载好模型文件存放在文件夹

跑着试一下

1
2
3
4
5
6
7
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("chatglm-6b-int4", trust_remote_code=True)
model = AutoModel.from_pretrained("chatglm-6b-int4", trust_remote_code=True).quantize(4).half().cuda()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)
response, history = model.chat(tokenizer, "晚上睡不着应该怎么办", history=history)
print(response)

image-20240315162242918

3. 结果展示

可以换着花样把输入给模型，对比输出的变化

屏幕截图 2024-03-15 164619

屏幕截图 2024-03-15 164613

参考链接

  1. https://github.com/IronSpiderMan/MachineLearningPractice/blob/main/chatglm_qa%2Fchatglm_document_qa_READM.md
  2. https://blog.csdn.net/qq_40968179/article/details/128996692