Metadata-Version: 2.1
Name: langdash
Version: 1.20.0
Summary: A simple library for interfacing with language models.
Home-page: https://git.mysymphony.jp.net/nana/langdash
Author: Nana Mochizuki
Author-email: nana@mysymphony.jp.net
Project-URL: Source, https://git.mysymphony.jp.net/nana/langdash
Project-URL: Documentation, https://langdash.readthedocs.io/en/latest/
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Provides-Extra: embeddings
Provides-Extra: rwkv_cpp
Provides-Extra: llama_cpp
Provides-Extra: transformers
Provides-Extra: ctransformers
Provides-Extra: sentence_transformers
License-File: LICENSE.txt

# langdash

A simple library for interfacing with language models.

**Currently in beta!**

**Features:**

  * Support for guided text generation, text classification (through prompting) and vector-based document searching.
  * Lightweight, build-it-yourself-style prompt wrappers in pure Python, with no domain-specific language involved.
  * Token healing and transformers/RNN state reuse for fast inference, like in [Microsoft's guidance](https://github.com/microsoft/guidance).
  * First-class support for ggml backends.

**Documentation:** [Read on readthedocs.io](https://langdash.readthedocs.io/en/latest/)

**Repository:** [main](https://git.mysymphony.jp.net/nana/langdash/) / [Gitlab mirror](https://gitlab.com/nanamochizuki77/langdash)

## Installation

Use [pip](https://pip.pypa.io/en/stable/) to install. By default, langdash does not come preinstalled with any additional modules. You will have to specify what you need like in the following command:

```
pip install --user langdash[embeddings,sentence_transformers]
```

List of modules:
  
  * **core:**
    * *embeddings:* required for running searching through embeddings
  * **backends:**
    * Generation backends: *rwkv_cpp*, *llama_cpp*, *ctransformers*, *transformers*
    * Embedding backends: *sentence_transformers*

**Note:** If running from source, initialize the git submodules in the `langdash/extern` folder to compile foreign backends.
    
## Usage

Examples:

  * [Text generation](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/text-generation.md)
  * [Generating TV shows](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/generating-tv-shows.md)
  * [Embedding search](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/embedding-search.md)

See [examples folder](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/examples) for full examples.

## Running the Examples

All examples can be ran with the following command:

```
python examples/instruct.py [model type] [model name or path]
```

You can specify additional model parameters using the `-ae` CLI argument, and passing a valid Python literal. For example, to run the chat example using the WizardLM model with context length of 4096, do:

```
python examples/chat.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096
```

Some examples require you to specify the prompt format. Formats include: `wizardlm` (shortened Alpaca format without the first prompt line and `# Instruction:`), and `alpaca` (the full format). You will need to specify it for most of the examples:

```
python examples/instruct.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096 --prompt-format wizardlm
```

For a full list, see the `examples/_instruct_format.py` file.


## License

Apache 2.0
