Ollama安装
先装个Ollama
1
|
curl -fsSL https://ollama.com/install.sh | sh
|
还是先下载下来吧
1
2
3
4
5
|
cd /opt
proxychains curl -fsSL https://ollama.com/install.sh -o ollama_install.sh
#这里一定要加sudo，不然useradd会未找到
#这里需要代理，因为下载Linux bundle太慢了
sudo proxychains bash ollama_install.sh
|
坐等
提示出
1
2
|
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
|
就ok了
注意，下面几个连续操作都是改变ollama.service，你可以一次配完
开放一下api
1
2
3
4
5
6
7
8
|
nano /etc/systemd/system/ollama.service
#在[Service]里面加
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"
#重启一下服务
systemctl daemon-reload
systemctl restart ollama
systemctl status ollama
|
修改模型路径
1
2
3
4
5
|
chown -R ollama:ollama /mnt/model
chmod -R 775 /mnt/model
#在[Service]里面加
Environment="OLLAMA_MODELS=/mnt/model"
|
添加一下代理
1
2
|
#在[Service]里面加
Environment="https_proxy=192.168.40.1:7890"
|
保持加载
1
2
|
#在[Service]里面加
Environment="OLLAMA_KEEP_ALIVE=-1"
|
上面的方法保持效果不好，用下面的
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
nano /etc/systemd/system/ollama-run.service
[Unit]
Description=Ollama Run Service
After=network.target
[Service]
ExecStart=/usr/local/bin/ollama run qwq
Restart=always
User=root
WorkingDirectory=/root
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable --now ollama-run
|
安装模型
1
|
curl http://localhost:11434/api/generate -d '{"model": "qwq", "keep_alive": -1}'
|
ok，pull下来的话就可以使用了
调整layer
1
2
3
|
#在[Service]里面加
Environment="OLLAMA_GPU_LAYER=cuda"
Environment="CUDA_VISIBLE_DEVICES=GPU-6b53d422-019c-11f0-b3ec-82c4d041ae01"
|