Few-Shot Named Entity Recognition using Span Markers
Project description
SpanMarker for Named Entity Recognition
SpanMarker is a framework for training powerful Named Entity Recognition models using familiar encoders such as BERT, RoBERTa and DeBERTa. Tightly implemented on top of the 🤗 Transformers library, SpanMarker can take advantage of its valuable functionality.
Based on the PL-Marker paper, SpanMarker breaks the mold through its accessibility and ease of use. Crucially, SpanMarker works out of the box with many common encoders such as bert-base-cased
and roberta-large
, and automatically works with datasets using the IOB
, IOB2
, BIOES
, BILOU
or no label annotation scheme.
Documentation
Feel free to have a look at the documentation.
Installation
You may install the span_marker
Python module via pip
like so:
pip install span_marker
Quick Start
Please have a look at our Getting Started notebook for details on how SpanMarker is commonly used. It explains the following snippet in more detail.
Colab | Kaggle | Gradient | Studio Lab |
---|---|---|---|
from datasets import load_dataset
from span_marker import SpanMarkerModel, Trainer
from transformers import TrainingArguments
def main():
dataset = load_dataset("DFKI-SLT/few-nerd", "supervised")
labels = dataset["train"].features["ner_tags"].feature.names
model_name = "bert-base-cased"
model = SpanMarkerModel.from_pretrained(model_name, labels=labels)
args = TrainingArguments(
output_dir="my_span_marker_model",
learning_rate=5e-5,
gradient_accumulation_steps=2,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
num_train_epochs=1,
save_strategy="steps",
eval_steps=200,
logging_steps=50,
fp16=True,
warmup_ratio=0.1,
dataloader_num_workers=2,
)
trainer = Trainer(
model=model,
args=args,
train_dataset=dataset["train"].select(range(8000)),
eval_dataset=dataset["validation"].select(range(2000)),
)
trainer.train()
trainer.save_model("my_span_marker_model/checkpoint-final")
metrics = trainer.evaluate()
print(metrics)
if __name__ == "__main__":
main()
Pretrained Models
-
tomaarsen/span-marker-bert-base-fewnerd-fine-super
is a model that I have trained in 2 hours on the finegrained, supervised Few-NERD dataset. It reached a 0.7053 Test F1, competitive in the all-time Few-NERD leaderboard usingbert-base
. My training script resembles the one that you can see above.- Try the model out online using this 🤗 Space.
-
tomaarsen/span-marker-roberta-large-fewnerd-fine-super
was trained in 6 hours on the finegrained, supervised Few-NERD dataset usingroberta-large
. It reached a 0.7103 Test F1, very competitive in the all-time Few-NERD leaderboard.
Changelog
See CHANGELOG.md for news on all SpanMarker versions.
License
See LICENSE for the current license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for span_marker-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96eb230ae8787df5986d5f3ccec4f10d595bf4caf64bfd71fd314c49375f9af1 |
|
MD5 | ecee1b00e82408d2d8ecc84db3fc7d54 |
|
BLAKE2b-256 | a505e61938db70f86c4a5a5537aec1faec006c914d889a1a84b537a77fa03614 |