Controlled language model generation using regex
Project description
regex llm
Constrained language model generation with regex.
Installation
pip install regex-llm
Usage
import regex
from transformers import (
AutoTokenizer,
GenerationMixin,
PreTrainedTokenizer,
AutoModelForCausalLM,
)
from regex_llm import RegexConstraint
model: GenerationMixin = AutoModelForCausalLM.from_pretrained("your model")
tokenizer: PreTrainedTokenizer = AutoTokenizer.from_pretrained("your model")
controller = RegexConstraint(tokenizer=tokenizer, model=model)
prompt = '帮我生成3种不同的水果\n输出json格式如下:\n[{"name":"xxx","description":"xxx"}]'
pattern = r"\[(,?\s*\{\"name\":\s*\"[^\"]+\",\s*\"description\":\s*\"[^\"]+\"\\s*})+\s*\]"
output = controller.generate(
prompt=prompt,
pattern=pattern,
do_sample=True,
num_beams=1,
top_p=0.7,
temperature=0.95,
)
print(output)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
regex_llm-0.1.0.tar.gz
(2.8 kB
view hashes)
Built Distribution
Close
Hashes for regex_llm-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f5d0b1ebbc57fd14ef7ddb43d8a0c1baf791aab4d65e21aa097651ebc082b52d |
|
MD5 | 691484e081bec8dd0f4e799c66e2830b |
|
BLAKE2b-256 | fcb1a21f033c6c471befccb2e55b53320d86095d12f98c49b05fc1672409c1c2 |