Parallel connect jobs (pele.concurrent)

Concurrent programming concepts allow pele connect jobs to be run in parallel. We use the package Pyro4 which makes this amazingly simple.

The server manages the central database and decides which connections to try.

ConnectServer(system, database[, ...]) Server which receives requests from, and passes connect jobs to the workers

An arbitrary number of workers can connect to the server and process the connect jobs.

ConnectWorker(uri[, system, strategy]) worker class to execute connect runs.


see the example in the examples/connecting_in_parallel/ folder for more details.

start the server in one terminal:

$ python

The provides an uri (which is also stored in pyros.uri). Clients can connect to this uri and access the connect manager. To start a worker:

$ python

Run on cluster / with remote workers

Start the server on a workstation (or node) which should be the master node. To allow for incoming remote connections, a hostname must be specified. Then specify the hostname to connect to in and submit as many jobs a needed.

Table Of Contents

Previous topic


Next topic