Skip to main content

Topic modelling over short texts

Project description

tweetopic


Blazing Fast Topic modelling for Short Texts

PyPI version pip downloads python version Code style: black
NumPy SciPy scikit-learn

:zap: Blazing Fast topic modelling over short texts utilizing the power of :1234: Numpy and :snake: Numba.

Features

  • Fast :zap:
  • Scalable :collision:
  • High consistency and coherence :dart:
  • High quality topics :fire:
  • Easy visualization and inspection :eyes:
  • Full scikit-learn compatibility :nut_and_bolt:

🛠 Installation

Install from PyPI:

pip install tweetopic

👩‍💻 Usage (documentation)

Train your a topic model on a corpus of short texts:

from tweetopic import DMM
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.pipeline import Pipeline

# Creating a vectorizer for extracting document-term matrix from the
# text corpus.
vectorizer = CountVectorizer(min_df=15, max_df=0.1)

# Creating a Dirichlet Multinomial Mixture Model with 30 components
dmm = DMM(n_components=30, n_iterations=100, alpha=0.1, beta=0.1)

# Creating topic pipeline
pipeline = Pipeline([
    ("vectorizer", vectorizer),
    ("dmm", dmm),
])

You may fit the model with a stream of short texts:

pipeline.fit(texts)

To investigate internal structure of topics and their relations to words and indicidual documents we recommend using topicwizard.

Install it from PyPI:

pip install topic-wizard

Then visualize your topic model:

import topicwizard

topicwizard.visualize(pipeline=pipeline, corpus=texts)

topicwizard visualization

🎓 References

  • Yin, J., & Wang, J. (2014). A Dirichlet Multinomial Mixture Model-Based Approach for Short Text Clustering. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 233–242). Association for Computing Machinery.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tweetopic-0.3.0.tar.gz (12.8 kB view hashes)

Uploaded Source

Built Distribution

tweetopic-0.3.0-py3-none-any.whl (15.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page