Huggingface bolts for geniusrise
Project description
Huggingface Bolts
This is a collection of generic streaming and (micro) batch bolts interfacing with the huggingface ecosystem.
Table of Contents
Includes:
No. | Name | Description | Input Type | Output Type |
---|---|---|---|---|
1 | Text Classification | Fine-tuning for text classification tasks | Batch | Batch |
2 | Instruction Tuning | Fine-tuning for instruction tuning tasks | Batch | Batch |
3 | Commonsense Reasoning | Fine-tuning for commonsense reasoning tasks | Batch | Batch |
4 | Language Modeling | Fine-tuning for language modeling tasks | Batch | Batch |
5 | Named Entity Recognition | Fine-tuning for named entity recognition tasks | Batch | Batch |
6 | Question Answering | Fine-tuning for question answering tasks | Batch | Batch |
7 | Sentiment Analysis | Fine-tuning for sentiment analysis tasks | Batch | Batch |
8 | Summarization | Fine-tuning for summarization tasks | Batch | Batch |
9 | Translation | Fine-tuning for translation tasks | Batch | Batch |
Usage
To test, first bring up all related services via the supplied docker-compose:
docker compose up -d
docker compose logs -f
These management consoles will be available:
Console | Link |
---|---|
Kafka UI | http://localhost:8088/ |
Postgres can be accessed with:
docker exec -it geniusrise-postgres-1 psql -U postgres
Usage
Text Classification
To fine-tune a model for text classification tasks, you can use the following command:
genius HuggingFaceClassificationFineTuner rise \
batch \
--input_folder my_dataset \
streaming \
--output_kafka_topic my_topic \
--output_kafka_cluster_connection_string localhost:9094 \
postgres \
--postgres_host 127.0.0.1 \
--postgres_port 5432 \
--postgres_user postgres \
--postgres_password postgres \
--postgres_database geniusrise \
--postgres_table state \
load_dataset \
--args
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for geniusrise-huggingface-0.4.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e82e5d0113b8c3b2ab21273b62f915b69ba6dc47ebb3789ead6f07839c7d58c |
|
MD5 | 58580516f5a9d0d5a77393f8e9d6faf6 |
|
BLAKE2b-256 | f699eec49503e95287c2efc3ffad1d2db9417bce98dbf5bb49c6606477a8be99 |
Close
Hashes for geniusrise_huggingface-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75a9d916d93fb198968bfef9d07d2241533df55a0e3f8768a4de0f25316ebc62 |
|
MD5 | 923b4007fe791bb7c9dd1b12532fcdab |
|
BLAKE2b-256 | 316ae402d03367d0877c0f7d21bee8ac0b22c4d4a0c2b8f105f7b2f3b9ed3d6c |