Skip to content

Simulation workflow qualification

This record qualifies the development recording/restart workflow against native CPU, Warp CPU and RTX 5060 CUDA execution. It accompanies the usage guide and adaptive-method assessment. It establishes implementation and numerical behavior for the profiles below, not experimental validity or a GPU performance claim.

The qualification extends Stage 2A c6e8bf0967 and Stage 2B 7a53906faa with the forcing, reduced observation, GPU/split restart and writer implementation in the commit introducing this record. The selected checkout was /tmp/particula-stage-02b, with PYTHONPATH=. and its existing .venv. The backend matrix includes native NumPy CPU RK23/ROS2, Warp 1.11.1 CPU and native CUDA on RTX 5060 (sm_120; driver 13.4, toolkit 12.9).

The source package identifies itself as 0.3.1. Installed distribution metadata in the example environment still identifies 0.2.14; no dependency upgrade was used to hide that mismatch. Example provenance includes the executing source, revision and dirty state. Exact restart checks package/runtime versions and configuration, but a version string alone does not identify modified code. Retain the source revision and study inputs alongside saved research output.

Requirement Evidence
Physical time, final output and resumed joins recording_test.py: exact decay at requested times, final row, duplicate-free continuation and malformed-schedule rejection
Observation is passive for a fixed schedule Native CPU, adaptive rejected trials, complete split cycles and prepared canonical state/RNG comparisons
Different output schedules retain scientific accuracy Independent radius quadrature at rtol=1e-5, per-particle atol 1e-23 kg and gas atol 1e-13 kg/m³; both schedules meet radius relative error 2e-5
Environmental events forcing_test.py: step/linear tables, absolute stage times, right-continuous output, automatic knot landing, rejected-trial restoration and restart at a jump
Independent forcing accuracy Segmented radius/reservoir equation with Sutherland viscosity; DOP853/Radau agree within 1e-10 in normalized radius; CPU RK23/ROS2 and Warp CPU/CUDA meet test-specific radius and gas targets
Analysis provenance and bounded storage Units, dimensions, gas order, material densities, population measure, volume and caller-supplied configuration/numerics/source; detached arrays, opt-in slots and byte/row limits
Atomic writes and corruption analysis_io_test.py and checkpoint tests: failed replacement preserves prior file; pending AnalysisWriter row can be retried without advancing; hashes, schemas, NPY dimensions, duplicate keys/members and unknown types reject
Native CPU restart cpu_restart_test.py: known native processes, stochastic/condensation RNG, non-unit volumes, fields and ledgers, latent diagnostics and whole-cycle failure preservation
Adaptive restart CPU RK23/ROS2/ROS3 and GPU RK23/ROS3 retain exact scaled coordinates, controller settings/counters, accepted time and table forcing; identical future schedules give bitwise continuation
Split restart Nested sequential/Strang built-in condensation/dilution groups, shortened cycles, custom native fields and matching complete-cycle future state
Prepared restart Dilution, Brownian coagulation, wall loss, sequence, latent condensation and an integral nucleation pulse on Warp CPU and CUDA; canonical state and RNG match after disk round trip
Registered prepared extensions Custom clock/field/state and multiple managed stochastic streams survive disk restart with the caller’s registry; missing registry rejects
Mixed CPU/GPU restart Explicit CPUTransfer preserves host PCG64 continuation, custom fields and canonical resident state after disk restart
Explicit transfers and capture boundary Prepared observer performs one setup snapshot, then declared reduced/full downloads; slot requests reject reduced mode; observation rejects active capture while qualified captured replay remains usable
Save functions outside capture Existing host analysis, CPU, adaptive GPU and prepared checkpoint saves reject active native CUDA capture before replacing output; CPU archive writing does not initialize Warp

The independent forcing tests specify all equations and reference inputs in forcing_test.py; they do not derive their oracle from the production rate. Restart tests compare future state and random continuation, not merely the loaded starting arrays. CUDA cases ran rather than being replaced by Warp CPU.

Analysis uses versioned ZIP/JSON/NPY arrays. HDF5 and Zarr offer chunking but add dependencies and more complex multi-object publication; bounded single-file histories fit atomic replacement. Full trajectories are deliberately opt-in and limited. Exact restart uses a distinct schema and a fixed record/enum codec, never pickle, arbitrary imports or serialized live process callbacks.

Prepared files preserve SimulationCheckpoint v2 and ResidentCheckpoint v5 canonical bytes; lossy CPU inspection carriers are regenerated from that authority. Native CPU and split records use schema 1. Adaptive CPU records use schema 2 for explicit table forcing; adaptive GPU records use schema 3 for ROS3 and exact unchanged-coordinate unscaling. Coupled GPU records use their own schema 1 for complete event boundaries. Container schemas remain separate from record versions. Unsupported schemas are rejected without guessed migration. Prepared/custom adapters remain caller-supplied; loading is data reconstruction, and resume validates the owner-specific configuration.

Opaque adaptive forcing and arbitrary split callbacks have no exact disk identity. GPU restart requires the saved device profile, with CUDA UUID matching for adaptive owners. Existing prepared-device constraints remain authoritative. These restrictions are supported-profile boundaries, not claims that any Python callback or machine can reproduce the same future trajectory.

Commands ran from the selected worktree. Logs remain under ignored .artifacts/.

Terminal window
PYTHONPATH=. .venv/bin/python -m pytest particula -q -Werror \
-m 'not slow and not performance' \
--cov=particula --cov-report=term-missing --cov-fail-under=80
PYTHONPATH=. .venv/bin/python -m pytest scripts/tests examples_tests \
-q -Werror --no-cov
.venv/bin/ruff check particula/ examples_tests/ scripts/ --no-fix
.venv/bin/ruff format particula/ examples_tests/ scripts/ --check
.venv/bin/python -m mypy particula/ --ignore-missing-imports
PYTHONPATH=. .venv/bin/python scripts/docs_examples.py refresh first-cpu-workflow
PYTHONPATH=. .venv/bin/python scripts/docs_topics.py
npm --prefix website run build
PYTHONPATH=. .venv/bin/python scripts/check_docs_structure.py --site-dir website/dist
  • Full package: 10679 passed, 19 skipped, 57 deselected, 1 xfailed; 91.18% coverage, exceeding the normal 80% gate. Skipped/slow/performance cases are outside that execution claim. This run preceded the small archive capture guard; the affected save/restart suites then passed 57 tests.
  • Tooling and examples: 565 passed, 1 skipped. The skip remains outside the execution claim. Standard Ruff/format passed (753 files); standard Mypy passed (673 sources). Untyped function bodies are outside standard Mypy’s scope.
  • Final combined numerical/workflow suite: 158 passed with warnings as errors, including actual CUDA. All fourteen runtime modules changed by Stage 2B have 90% aggregate focused coverage, using directory-based coverage discovery and explicit report paths. This avoids importing/unloading NumPy during dotted discovery.
  • All six new workflow guide Python blocks executed, including CUDA adaptive restart and prepared observation/restart. The full first CPU example refreshed successfully; its two-case 300-DPI figure was visually inspected. The saved bundle includes compact plot summaries, not full restart files.
  • Website: 554 pages built; 684 HTML pages, 319 API modules and 130 redirects verified with zero errors. Rendered structure check also passed. The final prepared codec matrix, including mixed host RNG, passed 23 tests.

The first example’s 10 and 20 µg/m³ vapor cases reach about 136.69 and 162.13 nm from 100 nm over 600 s, with species mass residuals below 7e-16 relative. These are illustrative model predictions, not measurements. Conserved mass and exact restart do not establish physical accuracy. Local adaptive tolerances are not global error guarantees, and changing output boundaries can change a numerical or stochastic trajectory. Compression/download cost is explicit and no workload speedup is inferred from correctness checks.

Review examined state detachment, native measures and species maps, accepted versus trial publication, exact coordinate restoration, forcing one-sidedness, RNG and field continuation, atomic replacement and non-executable loading. The capture-save gap was corrected and covered by an actual CUDA regression. Publication and merge status belong to the plan delivery records; these results do not assert a released package or a deployed documentation site.

The follow-through implementation adds GPUCoupledSimulation: fixed native source/collision/removal/dilution events and a separately controlled GPU condensation flow. It preserves persistent event buffers/RNG, explicit sequential order and full host transfer boundaries. The controller resets at every event boundary. Its distinct schema-1 checkpoint contains canonical event continuation, condensation identity, scalar controls, order and work counters; the outer archive codec remains explicit data, without pickle.

The focused coupled/numerics/archive/restart selection passed 65 tests with one deliberate Warp-CPU capture skip on Warp CPU and the RTX 5060. It includes exact disk continuation with source and Brownian collisions, captured events, rejected condensation trials without repeated event draws, unchanged caller state and event-buffer identity, independent dilution/inventory checks, empty-population continuation, shorter-bound rejection and faulted-cycle observation rejection. The latter retains the last complete public clock while rejecting further snapshots and checkpoints after partial physical work. Changed condensation physics rejects before device access. Unsupported event families reject before allocation. All-package Mypy and Ruff checks passed.

These tests do not substitute for the full published workloads. The cyclic chamber still requires all 32 boxes, 8,192 slots, 600 one-second cycles, 128-event pulses and complete second-half restart. The forcing example retains 16 boxes, 4,096 slots, 6,000 one-microsecond intervals and the full doubled-step refinement with unchanged control times. Their updated sources and compact recording are implemented. Both full publications were launched in a detached checkout pinned to 480ecf2c1a, separately from subsequent edits. The forcing publication and figure inspection subsequently completed, as recorded below. The cyclic chamber still requires full-workload completion and figure review. No GPU speedup is inferred from this explicit transfer path.

The broader integration gate completed with 10,809 package tests passed, 21 skips and one expected failure in 436.94 s; unfiltered coverage was 91.33%. This includes actual CUDA. The additional intentional skip is the coupled native-capture case under the Warp-CPU fixture. Example/tooling checks passed 554 tests with one skip in 141.64 s. The 563-page website build and rendered structure check passed with zero errors. A subsequent explicit checkpoint-plan type guard passed checked-body Mypy and all 17 coupled tests, with the same single Warp-CPU capture skip. The full suite began before that small guard was added; its coverage is complemented by this focused rerun, not represented as an untouched-source final-head run. Logs are retained in .artifacts/gpu-coupled-package-coverage.log, .artifacts/gpu-coupled-tooling-examples.log, .artifacts/gpu-coupled-final-focused.log and .artifacts/gpu-coupled-website.log.

Direct checkpoint_coupled(owner) calls now hold the same reentrant cycle lock as owner.checkpoint(). Review found that the standalone helper could otherwise copy the event checkpoint and aggregate counters at different cycle boundaries. A deterministic threaded regression fails against the previous unlocked helper and passes with the lock. The full coupled test module then passed 18 tests with one intentional Warp-CPU capture skip in 62.25 s. Ruff, formatting and checked-body Mypy passed for the affected modules. The reproducer and focused logs are .artifacts/gpu-coupled-lock-regression-before.log and .artifacts/gpu-coupled-checkpoint-lock.log. The ongoing full publications use the already-locked owner method; this correction does not change their physics.

The subsequent unfiltered package gate ran against executable revision 2b4bddefd8, including the checkpoint lock and plan-type guards:

Terminal window
PYTHONPATH=. .venv/bin/python -m pytest \
--cov=particula --cov-report=term-missing --cov-fail-under=80

It passed 10,810 tests, with 21 skips and one expected failure, in 514.17 s; coverage was 91.34%. Actual CUDA cases ran. Full Ruff/format checks passed for 764 files, and ordinary Mypy passed for 681 sources. Documentation structure reported zero errors, and the topic graph validated 33 topics, 170 resources and 35 supporting pages. Logs are .artifacts/stage-02-current-package-coverage.log, .artifacts/stage-02-current-ruff.log, .artifacts/stage-02-current-mypy.log, .artifacts/stage-02-current-structure.log and .artifacts/stage-02-current-topics.log.

The separate respiratory-refinement diagnostics passed all three tests with warnings as errors. They verify complete source setup reuse without time advancement, detection of a minor-species budget defect and normalization that does not hide a small solute behind water. These are checks of the assessment, not completion of its full 20 s numerical comparison. The pinned full comparison and the outstanding full example publications remain required.

The complete gpu-prepared-forcing publication subsequently passed on the RTX 5060 from pinned revision 480ecf2c1aa95da42985098c53e4ecdc66133d7b. Its package source hash is bbe6cb05017e86b19f7dc166c18cac96c0d762b5300c03990d91cd1a4e08ce8b. The guarded run took 3,139.79 s, retaining all sixteen boxes, 4,096 slots per box, 49,152 initially active particles, four materials and five gas lanes. The coarse run used 6,000 one-microsecond cycles; the fine run used 12,000 half-microsecond cycles and rtol=1e-7 rather than 1e-5. Both retained the same physical duration and 120 forcing control times.

  • Maximum final per-particle total-mass relative difference: 2.32234e-7 against the predeclared 1% threshold.
  • Maximum final water-vapor relative difference: 8.85905e-9 against 0.1%.
  • Maximum per-box/species inventory residual: 1.15463e-14 coarse and 2.93099e-14 fine, with external vapor changes and dilution accounted for.
  • Coarse adaptive work: 9,682 accepted and 1,711 rejected trials; fine work: 17,281 accepted and 2,797 rejected trials. Both had zero domain rejections.
  • Coarse coupling used 12,000 full downloads and 12,000 full uploads; the fine run used 24,000 of each. These measured costs preclude claiming fully resident execution or inferring a GPU speedup from this correctness comparison.

All saved-file hashes and scientific inputs were verified. The retained compact report contains thirteen frames and numerical refinement/work summaries, without full particle states. Figures were inspected before and after CPU-only replot; the replot preserves the original CUDA execution revision, device and timestamp. Its invocation now explicitly shows the figure, avoiding duplicate notebook display of the helper’s returned figure. The extracted download’s source and helpers matched byte-for-byte, imported from its own directory, and rendered the saved data with GPU calls disabled and no data mutation. That download check tested import/rendering, not a second execution of the full scientific workload.

The sparse displayed history records the supplied initial state before the first control update, then complete cycles before the next control update. It does not resolve all control changes or pulse peaks. The final-state refinement test is numerical consistency evidence for the declared case, not an independent experimental reference or an error bound at every intermediate time. The full cyclic chamber and the remaining CPU scientific studies are still required. The imported bundle passed the 563-page website build and rendered verification with zero errors; see .artifacts/coupled-forcing-website.log. The final ZIP’s executable source and all helper bytes match the independently tested download.

CUDA-resident material solvers and respiratory refinement

Section titled “CUDA-resident material solvers and respiratory refinement”

Revision 9c548d9d617a93cd23a45773426f751e9e6ed9ec qualifies the expanded multi-material RK23 and ROS3 GPU paths, device-owned adaptive decisions, selective gas forcing, and deterministic parallel reservoir reductions. The full package gate passed 10,874 tests, with 21 skips and one expected failure, in 331.88 s; coverage was 91.02%. Actual CUDA ran on the RTX 5060. Full Ruff/format checks and Mypy (696 sources) passed. Focused independent reservoir and dense-solve oracles passed 38 tests, including 12-material and larger-than-one-reduction-lane-group cases.

The complete 1,000-particle, twelve-material respiratory example ran to 20 s on that revision in 286.82 s. Its separate coarse/fine assessment completed all 201 observations in 1,229.39 s, retaining the original inventory assertions. The fine run used 100-fold tighter relative and absolute tolerances, half the maximum step and half the observation interval. Both prospective 0.1% consistency targets passed:

  • Maximum per-particle relative radius difference: 3.92121347e-6.
  • Maximum per-species phase-inventory difference: 3.22563458e-8 of that species’ initial total inventory.
  • Coarse: 8,921 accepted, one rejected step; fine: 41,841 accepted, one rejected step. Neither run had domain rejections.

The complete repository report at docs/data/numerics/resident-respiratory-refinement.json retains every observation, material identity, controls, source hashes and work counts. These elapsed times include concurrent GPU work and are not isolated performance benchmarks. This is same-method numerical consistency evidence; it does not independently validate the chemistry or correct the example’s previously documented chemical-name matches. The refreshed publication records CUDA execution, fixed revision and source hashes. The cyclic chamber has completed its 600 s trajectory and exact checkpoint continuation; the first publication failed during plot-data serialization, as recorded below.

The 10,000-particle, three-material biomass workload completed on the same revision and RTX 5060 in 4,148.09 s with concurrent GPU work. It retained 0.01 s initial adjustment, 600 s preparation and all four independent 3,600 s branches, each with 10,000 outer intervals of 0.36 s. The user-authorized model change replaces AO2008/DNS with ST1956 turbulent shear, retaining epsilon = 0.1 m²/s³ and fluid density 1.225 kg/m³. This is a changed collision model, not a CUDA implementation or validation of AO2008.

All four branches passed the unchanged per-species inventory assertion (rtol=1e-12, atol=1e-30). Each coupled branch asserted zero physical-state uploads and downloads between process stages. Adaptive decisions remained in native CUDA conditional graphs. Physical-state downloads were limited to declared observations; control metadata was read at completed outer intervals.

Coupled branch Accepted condensation steps Rejected steps Domain rejections
Brownian 31,450 7,221 0
Brownian + SP2016 settling 21,902 3,612 0
ST1956 turbulent shear 27,602 6,101 0

The refreshed biomass bundle retains the executed source, package hash, revision, device identity, scientific stdout and all four figures. The final size-distribution and mixing-state figures explicitly identify turbulent shear. These are completion and conservation checks for this illustration, not a collision-model convergence study or experimental validation.

The full cyclic trajectory exceeded the registry’s nominal two-hour limit. Inspection found that the synchronous Jupyter client blocked the notebook event loop while waiting for output, preventing the timeout from firing. The publisher now uses an asynchronous kernel manager and retains explicit kernel cleanup. A real-kernel regression (a seven-second cell with a five-second deadline) failed before this fix and passes after it, including verification that the timed-out kernel is released. All 23 publisher tests pass with warnings treated as errors; website tooling passes 61 Python and 14 JavaScript tests. This runner change does not alter simulation physics. The already-running chamber retained its original loaded runner.

Full resident cyclic trajectory and output recovery

Section titled “Full resident cyclic trajectory and output recovery”

The first complete resident cyclic run at 9c548d9d61 passed all 600 one-second cycles on the RTX 5060, with 32 boxes, 8,192 slots per box, 196,608 initially active particles and twelve prescribed source pulses. It finished with 37,064 active particles. The maximum observed box/species inventory relative residual was 1.19682042e-13, below the unchanged rtol=2e-12 gate. Charge, unit counts and finite-state checks passed at all thirty observation times.

The 300-second checkpoint reproduced the complete second half with byte-exact final state and random continuation. The original trajectory used 620,558 accepted and 378 rejected condensation steps, with zero domain rejections, 1,200 control readbacks and zero physical-state coupling uploads/downloads. These results are retained in the captured completion stdout; the first attempt did not publish a result bundle. Its subsequent plot-data writer rejected NumPy arrays, and kernel cleanup released the in-memory observations. No missing trajectory data have been reconstructed or substituted.

The observation boundary now converts detached arrays to plain lists. This changes output representation only. A regression using the actual observation, JSON writer and CPU-only plotting path reproduces the failure before the fix and passes after it. Together with the publisher tests, 24 tests passed with warnings treated as errors. The timeout is now 14,400 seconds to accommodate the measured full workload with enforced deadlines. The fresh full publication from 3d3753d2b completed in 6,307.33 s, using identical library bytes and scientific settings. It reproduced all work counts and accounting results above, including exact same-device restart and zero stage transfers. The retained 659,430-byte data bundle contains all 31 frames from 0 through 600 s. The four-panel figure was inspected before and after CPU-only replot. Replot preserved the data hash and original CUDA execution provenance; no simulation was rerun for rendering. The initial displayed point precedes the first forcing update, and the 20-second observations do not resolve every jump or peak. This completion time is not an isolated performance benchmark.

The corrected runner also validated all 59 CPU-policy registry examples without replacing their saved publications. This check called the existing validate_example path for each CPU-policy entry; all-library scientific bytes remained fixed during that pass and the chamber rerun.

Key validation commands for the resident changes and output fixes were:

Terminal window
PYTHONPATH=. .venv/bin/python -m pytest particula \
--cov=particula --cov-report=term-missing --cov-fail-under=80 -Werror -q
PYTHONPATH=. MPLBACKEND=Agg .venv/bin/python -m pytest \
examples_tests/gpu_cyclic_chamber_example_test.py \
scripts/tests/docs_examples_test.py -q --no-cov -Werror
.venv/bin/ruff check particula/ examples_tests/ scripts/ --no-fix
.venv/bin/ruff format particula/ examples_tests/ scripts/ --check

Final local documentation verification found all 89 registered bundles current and zero structure errors. The website built 571 pages; rendered verification checked 701 HTML pages, 332 API modules, 1,860 legacy anchors and 130 redirects with zero errors. The local Stage 2B branch retains the qualified outputs; GitHub publication remains pending direct push authorization.

Explore this topic: Execution and release qualification