Vicuna

Vicuna，Github - FastChat。

运行

安装依赖

$ pip3 install torch transformers fschat
# 安装 lfs
## ubuntu
sudo apt install git-lfs
# 验证
$ git lfs install

下载

# llama 原始模型
$ git clone https://huggingface.co/decapoda-research/llama-13b-hf
# vicuna 模型
$ git clone https://huggingface.co/lmsys/vicuna-13b-delta-v1.1

更新参数

$ python3 -m fastchat.model.apply_delta \
    --base-model-path /data/llama-13b-hf \
    --target-model-path /data/vicuna-13b \
    --delta-path /data/vicuna-13b-delta-v1.1

Serving

nohup python3 -m fastchat.serve.controller &
nohup python3 -m fastchat.serve.model_worker --model-path /data/vicuna-13b --device cpu &
nohup python3 -m fastchat.serve.gradio_web_server &