doc/README: initial commit

This commit is contained in:
mappu 2023-04-08 15:30:37 +12:00
parent fa8db95cc6
commit a7dd9580a5
2 changed files with 32 additions and 0 deletions

32
README.md Normal file
View File

@ -0,0 +1,32 @@
# llamacpphtmld
A web interface and API for the LLaMA large language AI model, based on the [llama.cpp](https://github.com/ggerganov/llama.cpp) runtime.
## Features
- Live streaming responses
- Continuation-based UI, supporting interrupt, modify, and resume
- Configure the maximum number of simultaneous users
- Works with any LLaMA model including [Vicuna](https://huggingface.co/eachadea/ggml-vicuna-13b-4bit)
- Bundled copy of llama.cpp, no separate compilation required
## Usage
All configuration should be supplied as environment variables:
```
LCH_MODEL_PATH=/srv/llama/ggml-vicuna-13b-4bit-rev1.bin \
LCH_NET_BIND=:8090 \
LCH_SIMULTANEOUS_REQUESTS=1 \
./llamacpphtmld
```
## API usage
```
curl -v -d '{"ConversationID": "", "APIKey": "", "Content": "The quick brown fox"}' -X 'http://localhost:8090/api/v1/generate'
```
## License
MIT

BIN
doc/screenshot.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB