A collection of measures for Approximate Functional Dependencies in relational data.
Project description
AFD comparative study
This repository contains all artifacts to "Approximately Measuring Functional Dependencies: a Comparative Study".
Overview
code
: this directory holds the code used to generate the results in the paperafd_measures
: all Python source code relating to the implemented AFD measuresexperiments
: Jupyter notebooks containing the processing steps to generate the results, figures or tables in the papersynthetic_data
: all Python source code relating to the synthetic data generation process
data
: the datasets used in the paperrwd
: manually annotated dataset of files found on the web (seedata/ground_truth.csv
)rwd_e
: datasets fromrwd
with errors introduced into them. Generated by the notebookcode/experiments/create_rwd_e_dataset.ipynb
.syn_e
: synthetic dataset generated focussing on errors. Generated by the notebookcode/experiments/create_syn_e.ipynb
syn_u
: synthetic dataset generated focussing on left-hand side uniqueness. Generated by the notebookcode/experiments/create_syn_u.ipynb
syn_s
: synthetic dataset generated focussing on right-hand side skewness. Generated by the notebookcode/experiments/create_syn_s.ipynb
paper
: A full version of the paper including all proofs.results
: results of applying the AFD measures to the datasets.
Installation
Use the code in this repository with Poetry or Conda.
Poetry
Install all dependencies via Poetry and start Jupyter lab to investigate the code.
$ poetry install
$ jupyter lab
Conda
Create a new environment from the conda_environment.yaml
file, activate it and run Jupyter lab to investigate the code.
$ conda create -f conda_environment.yaml
$ jupyter lab
Dataset References
In addition to this repository, we made our benchmark also available on Zenodo: find it here
adult.csv
: Dua, D. and Graff, C. (2019). UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science.claims.csv
: TSA Claims Data 2002 to 2006, published by the U.S. Department of Homeland Security.dblp10k.csv
: Frequency-aware Similarity Measures. Lange, Dustin; Naumann, Felix (2011). 243–248. Made available as DBLP Dataset 2.hospital.csv
: Hospital dataset used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824). Made available as part the dataset collection to that paper.t_biocase_...
files: t_bioc_... files used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824). Made available as part the dataset collection to that paper.tax.csv
: Tax dataset used in Johann Birnick, Thomas Bläsius, Tobias Friedrich, Felix Naumann, Thorsten Papenbrock, and Martin Schirneck. 2020. Hitting set enumeration with partial information for unique column combination discovery. Proc. VLDB Endow. 13, 12 (August 2020), 2270–2283. https://doi.org/10.14778/3407790.3407824). Made available as part the dataset collection to that paper.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
afd_measures-0.9.2.tar.gz
(7.6 kB
view hashes)
Built Distribution
Close
Hashes for afd_measures-0.9.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8df60b8e40110cf27324cd0a913942ebe36b7ff2981542f4ed92e3bfd381c8af |
|
MD5 | 1ab063f3c2ccfcc3a098f950a25b406e |
|
BLAKE2b-256 | fb99ea1c0620be796bd25de1a81471c7a01a2d09329ffe00919ceb5a8929a50a |