Skip to main content

Ctypes Register Bitfields

Project description

The ctypes-bitfield library consists of two modules, bitfield and remotestruct.

bitfield provides a mechanism for creating ctypes compatible implementations of registers made up of bitfields.

remotestruct allows for ctypes derived classes, such as Structure and Bitfield, to be accessed over a remote interface, such as TCP/IP or a VME indirection scheme.

bitfield

bitfield provides a mechanism for creating ctypes compatible implementations of registers made up of bitfields. The base ctypes library already provides much of this functionality, but the bitfield builder implementation wraps it up for simpler usage and avoids some of the quirky behaviors.

Normally the underlying register type would be a fixed size integer, a c_uint16 or c_uint64 or the like. However, a somewhat strange example usage would look something like this:

>>> from bitfield import *
>>> IEEE754 = make_bf('IEEE754', [
...     ('mantissa', c_uint, 23),
...     ('exponent', c_uint, 8),
...     ('sign', c_uint, 1)
... ], basetype=c_float, doc='Bitfields of an IEEE754 single precision float.')
>>> x = IEEE754()
>>> x.keys()
['mantissa', 'exponent', 'sign']
>>> x.base = 5.0
>>> list(x.items()) #doctest: +ELLIPSIS
[('mantissa', 2097152...), ('exponent', 129...), ('sign', 0...)]
>>> x.sign = 1
>>> x.base
-5.0
>>> x.exponent -= 2
>>> x.base
-1.25
>>> x.update(sign = 0, mantissa = 0)
>>> x.base
1.0

Bitfield objects are derived from ctypes.Union. Because of this derivation, these classes can be stacked into ctypes.Structures, which means they can work directly on memory mapped data. If the memory-mapped data is volatile, such as hardware registers, then the fact that the update() method operates on the entire register in one write, rather than one write per field, may be of use.

remotestruct

remotestruct allows for ctypes derived classes, such as Structure and Bitfield, to be accessed over a remote interface, such as TCP/IP or a VME indirection scheme. Effectively, this means turning requests for elements of the structure into requests for arbitrary byte sequences, fetching them, and managing the translation.

If, for instance, you had a mechanism whereby, over an Ethernet socket, you could write 8 bytes to address 0x100 as W 0x100 55 45 10 18 26 28 33 47, then read back four of those bytes by sending R 0x100 4 and getting back 55 45 10 18 (all of that newline delimited), then you would write a protocol handler:

>>> class SerialHandler(object):
...    def __init__(self, sock):
...        self.sock = sock
...
...    def writeBytes(self, addr, data):
...        msg = "W " + hex(addr) + ' '.join(str(d) for d in data)
...        self.sock.sendall(msg.encode('ascii'))
...
...    def readBytes(self, addr, size):
...        msg = "R 0x{0:X} {1}".format(addr, size)
...        self.sock.sendall(msg.encode('ascii'))
...
...        received = []
...        while True:
...            x = self.sock.recv(4096)
...            received.append(x)
...            if b'\n' in x:
...                break
...
...        msg = b''.join(received)
...        data = bytes(int(b) for b in msg.split(b' '))
...        return data

>>> class DataStructure(Structure):
...     _fields_ = [
...         ('flags', c_uint32),
...         ('_dummy1', c_uint32),
...         ('offset', c_int32),
...         ('slope', c_float)
...     ]

>>> sock = socket.create_connection(('1.2.3.4', 80))
>>> handler = SerialHandler(sock)
>>> rs = remotestruct.Remote(DataStructure, handler)
>>> rs.flags
5
>>> rs.flags = 183
>>> rs.flags
183

CachedHandler

RemoteStructs can suffer from performance issues over slow transports; fetching data that you know hasn’t changed over and over again just because it was easier to write the code that way. The solution to this is to wrap the handler in a CachedHandler, which provides a flexible caching mechanism to prevent pulling known data. The CachedHandler will, rather than pulling only the data requested, prefetch an entire aligned “cache line” of data based on the presumption that the next data you’ll need is likely to be physically near to the data you’re currently asking for. So with the default 32 byte cacheline, a request for the 2 bytes at address 40-41 will request all bytes 32-63, and store them in one of the cache sets (default 8). This data will remain cached until it either times out or the cache set is overwritten by a new cacheline.

Repeating the previous example with a CachedHandler would add:

>>> sock = socket.create_connection(('1.2.3.4', 80))
>>> basehandler = SerialHandler(sock)
>>> cachedhandler = CachedHandler(
    ...     handler=basehandler,
    ...     timeout=2.5
    ... )
>>> rs = remotestruct.Remote(DataStructure, cachedhandler)
>>> rs.flags
5
>>> rs.flags = 183
>>> rs.flags
183

The CachedHandler is most useful with a timeout, which dictates how old data in the cache can be before it expires; in the example above the timeout is set to 2.5 seconds. A timeout of None means that data will never expire; a timeout of 0 means that data is always expired, effectively disabling the cache.

The CachedHandler has many options to control the number of cache sets and the length of cache lines which you can easily spend your life tuning to try to get the “perfect” cache settings. Don’t do this. The CachedHandler can be initialized with stats=True, which will make the cache keep statistics on hits, misses and timeouts. If the cache is getting too many timeouts then you’re grabbing more data than you can use and should turn the cache line length down. If you’re getting too many misses then more cache sets or longer cache lines will be your solution, depending on your data access patterns.

The CachedHandler also has nocache and noprefetch options to fine-tune control performance; this can be essential to prevent destructive register accesses.

Changelog

0.3.1

Fixed some packaging problems.

0.3.0

Turned the .items iterator into a list. It’s never going to be so long that the overhead is a problem, and it makes interactive use from the command line so much easier.

Added the CachedHandler, and moved bitfield and remotestruct from being single modules to being full packages.

Works under Python 2.7+ and 3.2+

author:

Rob Gaddi, Highland Technology, Inc.

date:

24-Aug-2015

version:

0.3.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ctypes-bitfield-0.3.2.tar.gz (30.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page