core.base

pyctools.core.base.Component	Base class for all Pyctools components, ie objects to be used in processing pipelines / graph networks.
pyctools.core.base.Transformer	Base class for simple components with one input and one output.
pyctools.core.base.InputBuffer	Input object buffer.
pyctools.core.base.ObjectPool	Output object "pool".
pyctools.core.base.ThreadEventLoop	Event loop using threading.Thread.

class Component(config={}, **kwds)

Bases: ConfigMixin

Base class for all Pyctools components, ie objects to be used in processing pipelines / graph networks.

By default every component has one input and one output. To help other software introspect the component the input and output names are listed in inputs and outputs. Redefine these attributes if your component has different inputs and outputs.

The base class creates a thread-safe input buffer for each of your inputs. This allows the component to run in its own thread.

To help with load balancing, components usually have a limited size ObjectPool of output Frame objects. To disable this your class should set with_outframe_pool to False. The base class creates an output frame pool for each of your outputs.

A logging.Logger object is created for every component. Use this to report any errors or warnings from your component, rather than using print statements. The component may get used in situations where there is no console to print messages to.

Every component also has configuration methods. See ConfigMixin for more information. The configuration can be initialised by passing appropriate (key, value) pairs or a config dict to a component’s constructor. These values are applied after calling initialise.

Variables:

• ~Component.with_outframe_pool (bool) – Whether to use an outframe pool.

• ~Component.inputs (list) – The component’s inputs.

• ~Component.outputs (list) – The component’s outputs.

• ~Component.event_loop (class) – The type of event loop to use. Default is ThreadEventLoop.

• logger (logging.Logger) – logging object for the component.

Parameters:

config (dict) – Initial configuration values.

with_outframe_pool = True

inputs = ['input']

outputs = ['output']

event_loop

alias of ThreadEventLoop

logger

initialise()

Over ride this in your derived class if you need to do any initialisation, such as adding to the config object.

on_connect(output_name)

Over ride this in your derived class if you need to do anything when an output is connected.

on_set_config()

Over ride this in your derived class if you need to do anything when the configuration is updated.

The config isn’t actually changed until update_config is called, so be sure to do this in your method.

on_start()

Over ride this in your derived class if you need to do anything when the component is started.

on_stop()

Over ride this in your derived class if you need to do anything when the component is stopped.

This method is called before None is sent to all the component’s outputs, so you can use it to flush any remaining output.

send(output_name, frame)

Send an output frame.

The frame is sent to each input the output is connected to. If there are no connections this is a null operation with little overhead.

Parameters:

• output_name (str) – the output to use. Must be a member of outputs.

• frame (Frame) – the frame to send.

connect(output_name, input_method)

Connect an output to any callable object.

on_connect is called after the connection is made to allow components to do something when an output is conected.

Parameters:

• output_name (str) – the output to connect. Must be a member of outputs.

• input_method (callable) – the thread-safe callable to invoke when send is called.

bind(source, dest, destmeth)

Guild compatible version of connect.

This allows Pyctools components to be used in Guild pipelines.

start_event()

Called by the event loop when it is started.

Creates the output frame pools (if used) then calls on_start. Creating the output frame pools now allows their size to be configured before starting the component.

stop()

Thread-safe method to stop the component.

stop_event()

Called by the event loop when it is stopped.

Calls on_stop, then sends None to each output to shut down the rest of the processing pipeline.

is_pipe_end()

Is component the last one in a pipeline.

When waiting for a network of components to finish processing it’s not necessary to wait for every component to stop, and in many cases they won’t all stop anyway.

This method makes it easier to choose which components to wait for. See the Compound component for an example.

Return type:

bool

new_config()

Thread-safe method to alert the component to new config values.

new_config_event()

Called by the event loop when new config is available.

new_frame()

Thread-safe method to alert the component to a new input or output frame.

Called by the component’s input buffer(s) when an input frame arrives, and by its output frame pool(s) when a new output frame is available.

new_frame_event()

Called by the event loop when a new input or output frame is available.

Inputs are correlated by comparing their frame numbers. If there is a complete set of inputs, and all output frame pools are ready, the process_frame method is called.

If an input frame has a negative frame number it is not correlated with other inputs, it is merely required to exist. This allows frame objects to be used as control inputs when processing video sequences. The derived class should use the input buffer’s peek method to get the frame without removing it from the buffer. See the Matrix component for an example.

process_frame()

Process an input frame (or set of frames).

Derived classes must implement this method, unless they have no inputs and do not use any output frame pools.

It is called when all input buffers and all output frame pools have a frame available. The derived class should use the buffers’ and frame pools’ get methods to get the input and output frames, do its processing, and then call the output send methods to send the results to the next components in the pipeline.

See the Transformer base class for a typical implementation.

class Transformer(config={}, **kwds)

Bases: Component

Base class for simple components with one input and one output.

When an input Frame object is received, and an output Frame object is available from a pool, the transform method is called to do the component’s actual work.

process_frame()

Get the input and output frame, then call transform.

transform(in_frame, out_frame)

Process an input Frame.

You must implement this in your derived class.

Typically you will set out_frame’s data to a new image created from in_frame’s data. You must not modify the input Frame – it might be being used by another component running in parallel!

Return True if your processing was successful. Otherwise return False, after logging an appropriate error message.

Parameters:

• in_frame (Frame) – The input frame to read.

• out_frame (Frame) – The output frame to write.

Returns:

Should processing continue.

Return type:

bool

class InputBuffer(notify, **kwds)

Bases: object

Input object buffer.

Frame objects sent to the component are placed on a thread-safe queue before notifying the component that an input is available.

Parameters:

notify (callable) – a thread-safe function to be called when an input frame is available.

input(frame)

Put a frame object on the input queue (thread-safe).

Parameters:

frame (object) – the input Frame object.

available()

Get length of input queue (thread-safe).

Return type:

int

peek(idx=0)

Get a frame object from the input queue without removing it from the queue (thread-safe).

Parameters:

idx (int) – the queue position to fetch. Defaults the 0, the oldest frame in the queue.

Return type:

object

get()

Get the oldest frame object from the input queue, removing it from the queue (thread-safe).

Return type:

object

class ObjectPool(factory, notify, size, **kwds)

Bases: object

Output object “pool”.

In a pipeline of components it is useful to have some way of “load balancing”, to prevent the first component in the pipeline doing all its work before the next component starts. A simple way to do this is to use a limited size “pool” of objects. When the first component has used up all of the objects in its pool it has to wait for the next component in the pipeline to consume and release an object thus ensuring the first component doesn’t get too far ahead.

This object pool uses Python’s weakref.ref class to trigger the release of a new object when Python no longer holds a reference to an old object, i.e. when it gets deleted.

Note that the factory and notify functions must both be thread safe. They are usually called from the thread that deleted the old object, not the ObjectPool owner’s thread.

Parameters:

• factory (callable) – The function to call to create new objects.

• notify (callable) – A function to call when a new object is available, e.g. Component.new_frame.

• size (int) – The maximum number of objects allowed to exist at any time.

available()

Number of objects currently available from the pool.

Return type:

int

get()

Get an object from the pool.

Return type:

the object or None

class ThreadEventLoop(owner, **kwds)

Bases: Thread

Event loop using threading.Thread.

This is the standard Pyctools event loop. It runs a component in a Python thread, allowing Pyctools components to run “concurrently”.

The event loop provides three methods that the owning component should “adopt” as its own: start, running & join.

The owner component must provide four methods that the event loop calls in response to events: start_event, stop_event, new_frame_event & new_config_event.

Parameters:

owner (Component) – the component that is using this event loop instance.

start()

Start the thread’s activity.

It must be called at most once per thread object. It arranges for the object’s run() method to be invoked in a separate thread of control.

This method will raise a RuntimeError if called more than once on the same thread object.

join(timeout=None)

Wait until the thread terminates.

This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

running()

Return whether the thread is alive.

This method returns True just before the run() method starts until just after the run() method terminates. See also the module function enumerate().

queue_command(command)

Put a command on the queue to be called in the component’s thread.

Parameters:

command (callable) – the method to be invoked, e.g. new_frame_event.

run()

The actual event loop.

Calls the owner’s start_event method, then calls its new_frame_event and new_config_event methods as required until stop is called. Finally the owner’s stop_event method is called before the thread terminates.

---

Comments or questions? Please email jim@jim-easterbrook.me.uk.