solprop.driver.executors#

Execute independent scan points under an explicit resource policy.

Executor is the protocol: run(points, work) yields one result per point, possibly in completion order, each carrying its global index. Four policies satisfy it, one per module: Local (sequential, in the calling process), Pool (spawned workers sized by CPUs and memory), Queue (a durable file queue drained by controllers sharing a filesystem) and Slurm (an array job of queue controllers). The design is docs/dev/scans.md.

class solprop.driver.executors.Executor(*args, **kwargs)[source]#

Bases: Protocol

Run point work while preserving the identity carried by every result.

run_scan() needs only run(), and calls a validate(work) method before it writes anything where an executor has one. The rest of what it does is decided by the four executors’ classes, a closed set: an executor of any other class runs as Local and Pool do, its points’ results written by the calling process, without the backend Local keeps between points, or the stop refusal, durable preparation and collection of Queue and Slurm.

run(points, work)[source]#

Execute work for each point.

Parameters:
pointsiterable

Descriptions of independent work.

workcallable

Turn one description into a result containing its global index.

Yields:
object

Result, possibly in completion order. Storage uses its global index.

Modules

local

Execute scan points sequentially in the calling process.

pool

Run bounded spawned workers with explicit threads and memory admission control.

queue

Drain durable point work across controllers sharing a filesystem.

slurm

Emit and submit quoted Slurm array jobs with explicit controller resources.