Web Interface
llama serve -hf deepreinforce-ai/Ornith-1.0-35B-GGUF:Q4_K_M --no-mmap --tools all -c 0
The -c 0 parameter passes the model’s default context length to Llama.cpp. Otherwise you may end up with something small like 4096 tokens.
llama serve -hf deepreinforce-ai/Ornith-1.0-35B-GGUF:Q4_K_M --no-mmap --tools all -c 0
The -c 0 parameter passes the model’s default context length to Llama.cpp. Otherwise you may end up with something small like 4096 tokens.