slicerator: A Lazy-Loading, Fancy-Sliceable Iterable

Think of it like a generator that is “reusable” and has a length.

For installation instructions and to see a brief example, visit the GitHub page.

API Reference

class slicerator.Pipeline(proc_func, *ancestors, **kwargs)
class slicerator.SliceableAttribute(slicerator, attribute)

This class enables index-taking methods that are linked to a Slicerator object to remap their indices according to the Slicerator indices.

It also enables fancy indexing, exactly like the Slicerator itself. The new attribute supports both calling and indexing to give identical results.

Attributes:
indices

Methods

__call__  
indices
class slicerator.Slicerator(ancestor, indices=None, length=None, propagate_attrs=None)
Attributes:
indices

Methods

from_class(some_class[, propagate_attrs]) Make an existing class support fancy indexing via Slicerator objects.
from_func(func, length[, propagate_attrs]) Make a Slicerator from a function that accepts an integer index
classmethod from_class(some_class, propagate_attrs=None)

Make an existing class support fancy indexing via Slicerator objects.

When sliced using any iterable with a known length, it returns a Slicerator. When sliced with an integer, it returns the data payload.

Also, the attributes of the parent object can be propagated, exposed through the child Slicerators. By default, no attributes are propagated. Attributes can be white_listed in the following ways:

  1. using the optional parameter propagate_attrs; the contents of this list will overwrite any other list of propagated attributes
  2. using the @propagate_attr decorator inside the class definition
  3. using a propagate_attrs class attribute inside the class definition

The difference between options 2 and 3 appears when subclassing. As option 2 is bound to the method, the method will always be propagated. On the contrary, option 3 is bound to the class, so this can be overwritten by the subclass.

Methods taking an index will be remapped if they are decorated with index_attr. This decorator does not ensure that the method is propagated.

The existing class should support indexing (method __getitem__) and it should define a length (method __len__).

The result will look exactly like the existing class (__name__, __doc__, __module__, __repr__ will be propagated), but the __getitem__ will be renamed to _get and __getitem__ will produce a Slicerator object when sliced.

Parameters:
some_class : class
propagated_attrs : list, optional

list of attributes to be propagated into Slicerator this will overwrite any other propagation list

classmethod from_func(func, length, propagate_attrs=None)

Make a Slicerator from a function that accepts an integer index

Parameters:
func : callback

callable that accepts an integer as its argument

length : int

number of elements; used to supposed revserse slicing like [-1]

propagate_attrs : list, optional

list of attributes to be propagated into Slicerator

indices
slicerator.index_attr(func)
slicerator.key_to_indices(key, length)

Converts a fancy key into a list of indices.

Parameters:
key : slice, iterable of numbers, or boolean mask
length : integer

length of object that will be indexed

Returns:
indices, new_length
slicerator.pipeline(func=None, **kwargs)

Decorator to enable lazy evaluation of a function.

When the function is applied to a Slicerator or Pipeline object, it returns another lazily-evaluated, Pipeline object.

When the function is applied to any other object, it falls back on its normal behavior.

Parameters:
func : callable or type

Function or class type for lazy evaluation

retain_doc : bool, optional

If True, don’t modify func’s doc string to say that it has been made lazy. Defaults to False

ancestor_count : int or ‘all’, optional

Number of inputs to the pipeline. For instance, a function taking three parameters that adds up the elements of two Slicerators and a constant offset would have ancestor_count=2. If ‘all’, all the function’s arguments are used for the pipeline. Defaults to 1.

Returns:
Pipeline

Lazy function evaluation Pipeline for func.

See also

Pipeline

Examples

Apply the pipeline decorator to your image processing function.

>>> @pipeline
...  def color_channel(image, channel):
...      return image[channel, :, :]
...

In order to preserve the original function’s doc string (i. e. do not add a note saying that it was made lazy), use the decorator like so:

>>> @pipeline(retain_doc=True)
... def color_channel(image, channel):
...     '''This doc string will not be changed'''
...     return image[channel, :, :]

Passing a Slicerator the function returns a Pipeline that “lazily” applies the function when the images come out. Different functions can be applied to the same underlying images, creating independent objects.

>>> red_images = color_channel(images, 0)
>>> green_images = color_channel(images, 1)

Pipeline functions can also be composed.

>>> @pipeline
... def rescale(image):
... return (image - image.min())/image.ptp()
...
>>> rescale(color_channel(images, 0))

The function can still be applied to ordinary images. The decorator only takes affect when a Slicerator object is passed.

>>> single_img = images[0]
>>> red_img = red_channel(single_img)  # normal behavior

Pipeline functions can take more than one slicerator.

>>> @pipeline(ancestor_count=2)
...  def sum_offset(img1, img2, offset):
...      return img1 + img2 + offset
slicerator.propagate_attr(func)

Indices and tables