grasp 0.3.0

====================
Grasp
====================

A set of python functions to help with interactive object inspection
and discovery.

These help one grok, grasp, or get the gist of running code. They're
most useful in the context of an interactive IPython session, but can
be used in any Python shell. They're also helpful in the debugger.
They produce output that can be parsed programmatically if you find
that useful. I'll start with examples using the provided IPython
magic commands, since that's how I use them. Then I'll show how to
use them in other contexts.

There are basically three functions provided:
* gist -- object inspection
* rtype -- recursive type, deep object inspection
* apropos -- deep search for things with a given name, value, etc.

You can find code and dowloads at the Launchpad page or the PyPI page

http://launchpad.net/grasp
http://pypi.python.org/pypi/grasp/

Written and maintained by Greg Novak <greg.novak@gmail.com>

Introduction
============

gist
----

Suppose you are confronted with an unfamiliar object. What are
its characteristics? What kinds of things can it do? What kind
information does it contain?

gist returns all the attributes of the object organized by type:

In [1]: foo = 5
In [2]: %gist foo
Out[2]: {builtin_function_or_method: [bit_length, conjugate],
int: [denominator, imag, numerator, real]}

The return value is a dict with one key for each type. The value of
each key is a list of strings giving the names of all the attributes
of the given type. So integer objects have four integer attributes,
named numerator, denomintor, real, and imag, (accessed via foo.real,
etc.) as well as two function attributes (accessed via
foo.bit_length(), etc) named bit_length and conjugate.

By default, attributes with leading underscores are omitted. You can
include them by asking for verbose output with -v (the output below is
trimmed for brevity):

In [3]: %gist -v foo
Out[3]: {method-wrapper: [__abs__, __add__],
int: [denominator, imag, numerator, real],
builtin_function_or_method: [__format__, __getnewargs__,
bit_length, conjugate],
str: [__doc__],
type: [__class__]}


You can pass python code to the magic command, which is evalated.
(again, output trimmed):

In [4]: %gist numpy.array([1,2,3])
Out[4]: {buffer: [data],
int: [itemsize, nbytes, ndim, size],
builtin_function_or_method: [all, any, argmax]
tuple: [shape, strides],
ndarray: [T, imag, real]}

rtype
-----

Suppose you are confronted with a list that contains a deeply nested
structure of tuples, lists, and so on. What is this object all about?
Are there regularities in the structure? You want a function like
type(), but you want it to be recursive so that it summarizes the
type structure of the object as much as possible.

Start with a trivial case:

In [5]: %rtype 1
Out[5]: 'int'

If the object is a tuple of objects, all of the same type, say so.

In [6]: %rtype [1, 2, 3]
Out[6]: 'list of 3 int'

What if it's a tuple of heterogeneous types? List them all. Note
that the return value is now a list of strings.

In [7]: %rtype [1, 1.1, 2]
Out[7]: ['list of', 'int', 'float', 'int']

The rtype function is recursive, so this gets interesting when you add
another layer of container objects:

In [8]: %rtype [(1,2), (3,4), (5,6)]
Out[8]: ['list of 3', 'tuple of 2 int']

The rtype function knows about numpy arrays and classifies them
according to shape and type.

In [9]: %rtype [numpy.array([1,2]), numpy.array([3,4]), numpy.array([5,6])]
Out[9]: ['list of 3', 'ndarray of (2,) int64']

apropos
-------

Suppose I know that matplotlib (a Python plotting library) defines a
bunch of colormaps, but I don't have any idea where to find them
within the module. I can search recursively through the whole module
namespace, returning all of the ways to 'reach' objects with names
having to do with colormaps:

In [10]: import matplotlib
In [11]: %apropos cmap matplotlib
Out[11]: ['matplotlib.cm.cmapname',
'matplotlib.cm.get_cmap']

Note the many layers of indirection that apropos digs through to
arrive at the results. Apropos is similar to the standard %psearch
magic command that's included in IPython. The difference is that that
psearch only handles one level at a time (although it can search
intermediate modules, as long as you know how many dots separate the
target from the module).

In [12]: %psearch matplotlib.cmap*
In [13]: %psearch matplotlib.*.cmap*

Using apropos, you can also search for objects whose string
representation contains a given string. If no object to search is
given, search the entire namespace given by globals()

In [14]: %apvalue blue

You can search for objects whose docstring contains a given string.
Use quotes if the search string contains a space (this works for any
of the aporpos commands).

In [15]: %apdoc "colormap instance" matplotlib

There are versions of each of the above that accept regular
expressions.

In [16]: %apname_regex [Cc]olors

In [17]: %apvalue_regex [Cc]olors

In [18]: %adoc_regex [Cc]olors

You can also pass python code as the object in which to search and it
will be evaluated, should you find that useful. The apropos commands
assume that the first argument is the search string and everything
else is the object in which to search, so the second argument doesn't
need to be quoted if it contains spaces.

In [18]: %apdoc "colormap instance" dict(a=matplotlib, b=numpy)

You can search for python objects (rather than strings) using %apobj.
This gives the name of any object equal to the tuple (1,3,5) in the
numpy module.

In [19]: %apobj (1,3,5) numpy

If the search object contains spaces, it must be quoted

In [20]: %apobj "(1, 3, 5)" numpy

You can refer to variables in the user's namespace

In [21]: foo = numpy.array([1,2,3])
In [22]: %apobj foo numpy
In [23]: %apobj [foo,37] numpy

With the %apropos and %aobj commands, you can provide your own
function that returns True if the object should be considered a match
and false otherwise. This can be a named function on an anonymous
function (probably requiring quotation marks). In the latter case the
code will be evaluated. See the docstrings for %apropos and %apobj
for details.

In [24]: def my_search_fn(needle, name, obj): return name and needle in name

In [25]: %apropos -s my_search_fn Colors
In [26]: %apropos -s "lambda needle, name, obj: name and needle in name" Colors

For examples, see the search functions in the grasp module:

In [27]: %apname search grasp
Out[27]: ['grasp.search_doc',
'grasp.search_doc_regexp',
'grasp.search_equal',
'grasp.search_name',
'grasp.search_name_regexp',
'grasp.search_value',
'grasp.search_value_regexp']
In [28]: grasp.search_doc?

A final note is that apropos is meant to be exhaustive, so it tends to
return more than you need. You generally have to pick through the
results a little to find what you want.

Installation
============

Installation is the standard distutils routine:

python setup.py install

To install in the user-specific Python directory:

python setup.py install --user

To install somewhere under the user's home directory:

python setup.py install --home=~/some/place

To use grasp just do
import grasp

If IPython is running, grasp will register the magic commands and
you'll see them when you type
%lsmagic

There are two ways to load grasp automatically when IPython is
started.

1. Put a file containing "import grasp" in the directory
~/.ipython/profile_default/startup/

2. Open or create the file
~/.ipython/profile_default/ipython_config.py
and add this line to it:
c.InteractiveShellApp.exec_lines = ['import grasp']

grasp automatically tries to register the magic commands with IPython,
although it does not complain if IPython is not found. You should see
the magic commands listed when you type %lsmagic.

I have used this package on Gnu/Linux and OS X. I have no idea how/if
it works on Windows.

Non-IPython Environments
========================

If you don't use IPython, you just call the python functions upon
which the magic functions are based directly. The translation between
the magic command arguments given above and the arguments of the
Python functions should be straightforward. The magic commands
sometimes have shorter, more cryptic names with a view toward saving
keystrokes during heavy interactive use. It will look something like
this:

>>> import grasp
>>> grasp.gist([1,2,3], verbose=True)
>>> grasp.apropos_name('foo', grasp)
>>> grasp.recursive_type([1,2,3])

IPython has nice pretty-printing facilities, and I took advantage of
those in deciding how grasp presents the information it finds. Thus
gist() returns a dict and relies on IPython to format it in a readable
way for interactive use. This means that it's possible to use the
output programatically. One could, for example, do this to set
every attribute of a class with integer type to 42:

atts = grasp.gist(object)
for att in atts['int']:
setattr(object, att, 42)

Acknowledgements
================

Grasp was written in 2006 by Greg Novak <greg.novak@gmail.com> and
cleaned up for public consumption by the same in 2013.

This was written for my own use when developing Python code to produce
and analyze simulation output using the excellent IPython, Numpy,
Scipy, and matplotlib packages. I have benefited enormously from the
work of the authors of those packages over the years.