diff --git a/README.md b/README.md new file mode 100644 index 0000000..cd0b502 --- /dev/null +++ b/README.md @@ -0,0 +1,32 @@ +# llamacpphtmld + +A web interface and API for the LLaMA large language AI model, based on the [llama.cpp](https://github.com/ggerganov/llama.cpp) runtime. + +## Features + +- Live streaming responses +- Continuation-based UI, supporting interrupt, modify, and resume +- Configure the maximum number of simultaneous users +- Works with any LLaMA model including [Vicuna](https://huggingface.co/eachadea/ggml-vicuna-13b-4bit) +- Bundled copy of llama.cpp, no separate compilation required + +## Usage + +All configuration should be supplied as environment variables: + +``` +LCH_MODEL_PATH=/srv/llama/ggml-vicuna-13b-4bit-rev1.bin \ + LCH_NET_BIND=:8090 \ + LCH_SIMULTANEOUS_REQUESTS=1 \ + ./llamacpphtmld +``` + +## API usage + +``` +curl -v -d '{"ConversationID": "", "APIKey": "", "Content": "The quick brown fox"}' -X 'http://localhost:8090/api/v1/generate' +``` + +## License + +MIT diff --git a/doc/screenshot.png b/doc/screenshot.png new file mode 100644 index 0000000..eb54732 Binary files /dev/null and b/doc/screenshot.png differ