arv · PyPI

A fast 23andMe raw genome file parser

These details have not been verified by PyPI

Project links

Homepage

Project description

arv — a fast 23andMe parser for Python
======================================
|travis-status| |versions| |license| |pypi|

Arv (Norwegian; "heritage" or "inheritance") is a Python module for parsing raw
23andMe genome files. It lets you lookup SNPs from RSIDs.

.. code:: python

from arv import load, unphased_match as match

genome = load("genome.txt")

print("You are a {gender} with {color} eyes and {complexion} skin.".format(
gender = "man" if genome.y_chromosome else "woman",
complexion = "light" if genome["rs1426654"] == "AA" else "dark",
color = match(genome["rs12913832"], {"AA": "brown",
"AG": "brown or green",
"GG": "blue"})))

For my genome, this little program produces::

You are a man with blue eyes and light skin.

The parser is insanely fast, having been written in finely tuned C++, exposed
via Cython. A 2013 Xeon machine I've tested on parses a 24 Mb file into a hash
table in about 78 ms. The newer 23andMe files are smaller, and parses in a mere
62 ms!

Works with Python 2.7+ and 3+. Installable with pip!

.. code:: bash

p i p i n s t a l l - - u p g r a d e a r v S e e b e l o w f o r s o f t w a r e r e q u i r e m e n t s . I m p o r t a n t d i s c l a i m e r ==================== I t^{'} s v e r y i m p o r t a n t t o t e l l y o u t h a t I, t h e a u t h o r o f a r v, a m m e r e l y a * h o b b y i s t *! I * a m * a p r o f e s s i o n a l s o f t w a r e d e v e l o p e r, b u t * n o t * a g e n e t i c i s t, b i o l o g i s t, m e d i c a l d o c t o r o r a n y t h i n g l i k e t h a t . B e c a u s e o f t h a t, t h i s s o f t w a r e m a y n o t o n l y l o o k w e i r d t o p e o p l e i n t h e f i e l d, i t m a y a l s o c o n t a i n s e r i o u s e r r o r s . I f y o u f i n d a n y p r o b l e m w h a t s o e v e r, p l e a s e s u b m i t a G i t H u b i s s u e . T h i s a s l i g h t l y m o d i f i e d v e r s i o n o f w h a t I w r o t e f o r t h e o r i g i n a l s o f t w a r e c a l l e d " d n a - t r a i t s ", a n d t h e s a m e g o e s f o r t h i s s o f t w a r e : I n a d d i t i o n t o t h e G P L v 3 l i c e n s i n g t e r m s, a n d g i v e n t h a t t h i s c o d e d e a l s w i t h h e a l t h - r e l a t e d i s s u e s, I w a n t t o s t r e s s t h a t t h e p r o v i d e d c o d e m o s t l i k e l y c o n t a i n s e r r o r s, o r i n v a l i d g e n o m e r e p o r t s . R e s u l t s f r o m t h i s c o d e m u s t b e i n t e r p r e t e d a s H I G H L Y S P E C U L A T I V E a n d m a y e v e n b e d o w n r i g h t I N C O R R E C T . A l w a y s c o n s u l t a n e x p e r t (m e d i c a l d o c t o r, g e n e t i c i s t, e t c .) f o r g u i d a n c e . I t a k e N O R E S P O N S I B I L I T Y w h a t s o e v e r f o r a n y c o n s e q u e n c e s o f u s i n g t h i s c o d e, i n c l u d i n g b u t n o t l i m i t e d t o l o s s o f l i f e, m o n e y, s p o u s e s, s e l f - e s t e e m a n d s o o n . U s e a t Y O U R O W N R I S K . T h e i n d e n d e d u s e i s f o r c a s u a l, e d u c a t i o n a l p u r p o s e s . I f t h i s c o d e i s u s e d f o r r e s e a r c h p u r p o s e s, p l e a s e c r o s s - c h e c k k e y r e s u l t s w i t h o t h e r s o f t w a r e : T h e p a r s e r c o d e m a y c o n t a i n s e r i o u s e r r o r s, f o r e x a m p l e . A n i n t e r e s t i n g s t o r y a b o u t t h e r e s e a r c h p a r t : I o n c e r e l e a s e d a p r e t t y g o o d M e r s e n n e T w i s t e r P R N G f o r C + + t h a t e n d e d u p b e i n g u s e d i n r e s e a r c h . T u r n e d o u t t h e e n g i n e h a d b u g s, a n d b y t h e t i m e I h a d f i x e d t h e m, a p o o r r e s e a r c h e r h a d a l r e a d y p r o d u c e d r e s u l t s w i t h i t (h o p e f u l l y n o t p u b l i s h e d; I d o n^{'} t k n o w) . T h e g u y h a d t o g o b a c k a n d f i x h i s s t u f f, a n d I f e l t t e r r i b l y b a d a b o u t i t . S o b e w a r e! I n s t a l l a t i o n ============ T h e r e c o m m e n d e d w a y i s t o i n s t a l l f r o m P y P i . . . c o d e :: b a s h

pip install arv

This will most likely build Arv from source. The package will automatically
install Cython, but it doesn't check if you have a C++11 compiler. Furthermore,
it passes some additional compilation flags that are specific to clang/gcc.

If you have problems running ``pip install arv``, please open an issue on
GitHub with as much detail as possible (``g++/clang++ --version``, ``uname
-a``, ``python --version`` and so on).

If you set the environment variable ``ARV_DEBUG``, it will build with full
warnings and debug symbols.

You can also install it locally through ``setup.py``. The following builds and
tests, but does not install, arv:

.. code:: bash

p y t h o n s e t u p . p y t e s t I f y o u s e t t h e e n v i r o n m e n t v a r i a b l e ‘ ‘ A R V_{B} E N C H M A R K ‘ ‘ t o a g e n o m e f i l e n a m e a n d r u n t h e t e s t s, i t w i l l p e r f o r m a s h o r t b e n c h m a r k, r e p o r t i n g t h e b e s t p a r s i n g t i m e o n i t . Y o u c a n a l s o s e t ‘ ‘ A R V_{B} E N C H M A R K_{C} O U N T =< n u m b e r > ‘ ‘ t o c h a n g e h o w m a n y t i m e s i t s h o u l d p a r s e t h e g i v e n f i l e . U s a g e ===== F i r s t y o u n e e d t o d u m p t h e r a w g e n o m e f i l e f r o m 23 a n d M e . Y o u^{'} l l f i n d i t u n d e r t h e r a w g e n o m e b r o w s e r, a n d d o w n l o a d t h e f i l e . Y o u m a y h a v e t o u n z i p i t f i r s t : T h e p a r s e r w o r k s o n t h e p u r e t e x t f i l e s . T h e n y o u l o a d t h e g e n o m e i n P y t h o n w i t h . . c o d e :: p y t h o n >>> g e n o m e = a r v . l o a d (" f i l e n a m e . t x t ") >>> g e n o m e < G e n o m e : S N P s = 960613, n a m e =^{'} f i l e n a m e . t x t^{'} > T o s e e i f t h e r e a r e a n y Y - c h r o m o s o m e s p r e s e n t i n t h e g e n o m e, . . c o d e :: p y t h o n >>> g e n o m e . y_{c} h r o m o s o m e T r u e T h e g e n o m e p r o v i d e s a ‘ ‘ d i c t ‘ ‘ - l i k e i n t e r f a c e . T o g e t a g i v e n S N P, j u s t e n t e r t h e R S I D . . . c o d e :: p y t h o n >>> g e n o m e [" r s 123 "] >>> s n p < S N P : c h r o m o s o m e = 7 p o s i t i o n = 24966446 g e n o t y p e =^{'} A A^{'} >>>> s n p . c h r o m o s o m e 7 >>> s n p . p o s i t i o n 24966446 >>> s n p . g e n o t y p e < G e n o t y p e^{'} A A^{'} > T h e ‘ ‘ G e n o t y p e ‘ ‘ o b j e c t c a n b e c o n v e r t e d t o a s t r i n g w i t h ‘ ‘ s t r ‘ ‘, b u t i t a l s o a l l o w s r i c h c o m p a r i s o n s w i t h s t r i n g s d i r e c t l y : . . c o d e :: p y t h o n >>> s n p . g e n o t y p e == " A A " T r u e y o u c a n g e t i t s c o m p l e m e n t w i t h t h e ‘ ‘ ‘ ‘ - o p e r a t o r . . . c o d e :: p y t h o n >>> t y p e (s n p . g e n o t y p e) < c l a s s_{a}^{'} r v . G e n o t y p e^{'} >>>> s n p . g e n o t y p e < G e n o t y p e^{'} T T^{'} > T h e c o m p l e m e n t i s i m p o r t a n t d u e t o e a h S N P s o r i e n t a t i o n . A l l o f 23 a n d M e S N P s a r e o r i e n t e d t o w a r d s t h e p o s i t i v e (" p l u s ") s t r a n d, b a s e d o n t h e ‘ G R C h 37 < h t t p s : / / w w w . n c b i . n l m . n i h . g o v / g r c / h u m a n > ‘_{r} e f e r e n c e h u m a n g e n o m e a s s e m b l y b u i l d . B u t s o m e S N P s o n S N P e d i a a r e g i v e n w i t h t h e ‘ m i n u s o r i e n t a t i o n < h t t p : / / s n p e d i a . c o m / i n d e x . p h p / O r i e n t a t i o n > ‘_{.} F o r e x a m p l e, t o d e t e r m i n e i f t h e h u m a n i n q u e s t i o n i s l i k e l y l a c t o s e t o l e r a n t o r n o t, w e c a n l o o k a t ‘ r s 4988235 < h t t p : / / s n p e d i a . c o m / i n d e x . p h p / R s 4988235 > ‘_{.} S N P e d i a r e p o r t s i t s * S t a b i l i z e d * o r i e n t a t i o n t o b e m i n u s, s o w e n e e d t o u s e t h e c o m p l e m e n t : . . c o d e :: p y t h o n >>> g e n o m e [" r s 4988235 "] . g e n o t y p e < G e n o t y p e^{'} A A^{'} >>>> g e n o m e [" r s 4988235 "] . g e n o t y p e < G e n o t y p e^{'} T T^{'} > B y r e a d i n g a f e w ‘ G W A S < h t t p s : / / e n . w i k i p e d i a . o r g / w i k i / G e n o m e - w i d e_{a} s s o c i a t i o n_{s} t u d y > ‘_{r} e s e a r c h p a p e r s, w e c a n b u i l d a r u l e t o d e t e r m i n e a h u m a n^{'} s l i k e l i h o o d f o r l a c t o s e t o l e r a n c e : . . c o d e :: p y t h o n >>> a r v . u n p h a s e d_{m} a t c h (g e n o m e [" r s 4988235 "] . g e n o t y p e, " T T " : " L i k e l y l a c t o s e t o l e r a n t ", " T C " : " L i k e l y l a c t o s e t o l e r a n t ", " C C " : " L i k e l y l a c t o s e i n t o l e r a n t ", N o n e : " U n a b l e t o d e t e r m i n e (g e n o t y p e n o t p r e s e n t) ")^{'} L i k e l y l a c t o s e t o l e r a n t^{'} N o t e t h a t r e a d i n g G W A S p a p e r s f o r h o b b y i s t s c a n b e a b i t t r i c k y . I f y o u a r e a h o b b y i s t, b e s u r e t o s p e n d s o m e t i m e r e a d i n g t h e p a p e r c l o s e l y, c h e c k i n g u p S N P s o n p l a c e s l i k e ‘ S N P e d i a < h t t p : / / s n p e d i a . c o m > ‘_{,} ‘ d n S N P < h t t p s : / / w w w . n c b i . n l m . n i h . g o v / p r o j e c t s / S N P / > ‘_{a} n d ‘ O p e n S N P < h t t p s : / / o p e n s n p . o r g / g e n o t y p e s > ‘_{.} F i n a l l y, h a v e f u n, b u t b e e x t r e m e l y c a r e f u l a b o u t d r a w i n g c o n c l u s i o n s f r o m y o u r r e s u l t s . C o m m a n d l i n e i n t e r f a c e ====================== Y o u c a n a l s o i n v o k e ‘ ‘ a r v ‘ ‘ f r o m t h e c o m m a n d l i n e : . . c o d e :: b a s h

python -m arv --help

For example, you can drop into a Python REPL like so:

.. code:: bash

p y t h o n - m a r v - - r e p l g e n o m e . t x t g e n o m e . t x t . . . 960614 S N P s, m a l e T y p e ‘ g e n o m e ‘ t o s e e t h e p a r s e d 23 a n d M e r a w g e n o m e f i l e >>> g e n o m e < G e n o m e : S N P s = 960614, n a m e =^{'} g e n o m e . t x t^{'} >>>> g e n o m e [" r s 123 "] < S N P : c h r o m o s o m e = 7 p o s i t i o n = 24966446 g e n o t y p e =< G e n o t y p e^{'} A A^{'} >> I f y o u s p e c i f y s e v e r a l f i l e s, y o u c a n a c c e s s t h e m t h r o u g h t h e v a r i a b l e ‘ ‘ g e n o m e s ‘ ‘ . T h e e x a m p l e a t t h e t o p o f t h i s d o c u m e n t c a n b e r u n w i t h ‘ ‘ - - e x a m p l e ‘ ‘ : . . c o d e :: b a s h

python -m arv --example genome.txt
genome.txt ... 960614 SNPs, male

genome.txt ... A man with blue eyes and light skin

License
=======

Copyright 2017 Christian Stigen Larsen

Distributed under the GNU GPL v3 or later. See the file COPYING for the full
license text. This software makes use of open source software; see LICENSES for
details.

.. |travis-status| image:: https://travis-ci.org/cslarsen/arv.svg?branch=master
:alt: Travis build status
:scale: 100%
:target: https://travis-ci.org/cslarsen/arv

.. |license| image:: https://img.shields.io/badge/license-GPL%20v3%2B-blue.svg
:target: http://www.gnu.org/licenses/old-licenses/gpl-3.en.html
:alt: Project License

.. |versions| image:: https://img.shields.io/badge/python-2%2B%2C%203%2B-blue.svg
:target: https://pypi.python.org/pypi/arv/
:alt: Supported Python versions

.. |pypi| image:: https://badge.fury.io/py/arv.svg
:target: https://badge.fury.io/py/arv

Algorithm	Hash digest
SHA256	`7cda7fbc1175e0e8bd19b3d4be4208cbfb72f7c554c51ac41f46ce7b51ee40f9`
MD5	`4fc097509e53e4cf5928891cf159a611`
BLAKE2b-256	`c26a7198d7adcb9e5f90080c5f2e0bd9dced045f174dd809267b626278995b67`

Algorithm	Hash digest
SHA256	`c44aaef6d06584a378d353b6408c10adba17331c7eb26e9790c3c37bceb4f041`
MD5	`a2c98152aa06f6900c72f8f29c807668`
BLAKE2b-256	`bc09e5e0750e2c608003861780d4807726937bdfca6bc5ab1ea04532cbef8dd8`

arv 0.9.2

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distribution

File details

File metadata

File hashes

File details

File metadata

File hashes