Using the Resume Feature and Iteration Loggers in QMCPy¶

This notebook is the compact, recipe-style guide. For background and a deeper discussion of the “How much accuracy do you need?” question, see the long demo.

The resume parameter lets you restart a stopped integration from where it left off.

Typical workflow:

Run solution, data = sc.integrate() with a loose tolerance for a quick estimate.
(Optional) Save data to disk with data.save(path) so the state is preserved.
(Optional) Later (or in a new session), load it: data = Data.load(path).
Reconstruct a compatible solver (same problem definition and settings), tighten tolerance, and call integrate(resume=data).

Note: Steps 2 and 3 are optional. If they are skipped, the workflow continues using the data held in memory. If they are executed, the workflow uses a persistent, disk-backed state.

This demonstrates workflow and checkpointing correctness. For small examples, wall-clock timing differences can be negligible; performance gains become clearer when the initial run already used substantial work.

In [1]:

Copied!





from pathlib import Path
from qmcpy import CubQMCLatticeG, Lattice, Genz
from qmcpy.util.data import Data
import resume_util as ru
from pathlib import Path
from qmcpy import CubQMCLatticeG, Lattice, Genz
from qmcpy.util.data import Data
import resume_util as ru

Step 1: Quick Estimate (Loose Tolerance)¶

Define a 3-D Genz integral over the unit cube and run CubQMCLatticeG with a loose tolerance to get a fast initial estimate.

We fix the seed so notebook output is reproducible.

If you want iteration-by-iteration diagnostics (sample counts, n_min, m, shape growth, and interval half-width), set TRACE_ITERATIONS = True in the helper cell above.

In [2]:

Copied!





def make_CubQMCLatticeG_solver(abs_tol=1e-4, rel_tol=0, seed=7, dimension=3):
    """Build a CubQMCLatticeG solver for the demo case."""
    integrand = Genz(Lattice(dimension=dimension, seed=seed), kind_func="oscillatory", kind_coeff=1)
    return CubQMCLatticeG(integrand, abs_tol=abs_tol, rel_tol=rel_tol)
def make_CubQMCLatticeG_solver(abs_tol=1e-4, rel_tol=0, seed=7, dimension=3):
    """Build a CubQMCLatticeG solver for the demo case."""
    integrand = Genz(Lattice(dimension=dimension, seed=seed), kind_func="oscillatory", kind_coeff=1)
    return CubQMCLatticeG(integrand, abs_tol=abs_tol, rel_tol=rel_tol)

In [3]:

Copied!





abs_tol_loose = 1e-4
rel_tol = 0
solver = make_CubQMCLatticeG_solver(abs_tol=abs_tol_loose, rel_tol=0, seed=7, dimension=3)
solver.trace_iterations = True  # True to print iteration log, one line per iteration
solver.verbose = True           # True to print every iteration log line, False to print only a subset
solution1, data1 = solver.integrate()
abs_tol_loose = 1e-4
rel_tol = 0
solver = make_CubQMCLatticeG_solver(abs_tol=abs_tol_loose, rel_tol=0, seed=7, dimension=3)
solver.trace_iterations = True  # True to print iteration log, one line per iteration
solver.verbose = True           # True to print every iteration log line, False to print only a subset
solution1, data1 = solver.integrate()

stage        iter    solution comb_bound_diff      n_min    n_total    elapsed_time      m      xfull.shape
-----------------------------------------------------------------------------------------------------------
ITER            1  -0.4289211       1.943e-03          0       1024       6.821e-04     10        (1024, 3)
ITER            2  -0.4289245       8.371e-04       1024       2048       1.244e-03     11        (2048, 3)
ITER            3  -0.4289312       1.955e-04       2048       4096       1.715e-03     12        (4096, 3)

Step 2: Save the Integration State¶

data1.save(path) pickles the Data object to disk. Pass compress=True to create a smaller gzip-compressed file (.gz suffix added automatically).

In [4]:

Copied!





output_dir = Path("output")
output_dir.mkdir(parents=True, exist_ok=True)
save_path = output_dir / "resume_example_data1.pkl"

# By default, save() skips writing if the file already exists.
# Pass overwrite=True to replace an existing checkpoint.
data1.save(save_path, overwrite=True)
print(f"Uncompressed file: ({save_path.stat().st_size:,} bytes)")

# Compressed variant — the .gz suffix is added automatically
data1.save(output_dir / "resume_example_data1_compressed.pkl", compress=True, overwrite=True)
save_path_compressed = output_dir / "resume_example_data1_compressed.pkl.gz"
gz_size = save_path_compressed.stat().st_size
print(f"Compressed file:  {gz_size:,} bytes  ({100*(1 - gz_size/save_path.stat().st_size):.0f}% smaller)")
output_dir = Path("output")
output_dir.mkdir(parents=True, exist_ok=True)
save_path = output_dir / "resume_example_data1.pkl"

# By default, save() skips writing if the file already exists.
# Pass overwrite=True to replace an existing checkpoint.
data1.save(save_path, overwrite=True)
print(f"Uncompressed file: ({save_path.stat().st_size:,} bytes)")

# Compressed variant — the .gz suffix is added automatically
data1.save(output_dir / "resume_example_data1_compressed.pkl", compress=True, overwrite=True)
save_path_compressed = output_dir / "resume_example_data1_compressed.pkl.gz"
gz_size = save_path_compressed.stat().st_size
print(f"Compressed file:  {gz_size:,} bytes  ({100*(1 - gz_size/save_path.stat().st_size):.0f}% smaller)")

Out[4]:

'output/resume_example_data1.pkl'

Uncompressed file: (234,546 bytes)

Out[4]:

'output/resume_example_data1_compressed.pkl.gz'

Compressed file:  136,474 bytes  (42% smaller)

Step 3: Resume with a Tighter Tolerance¶

Load the saved state and resume with a compatible solver (same integrand family, dimension, and randomization settings). Only the incremental samples needed to meet the tighter tolerance are generated.

In [5]:

Copied!





loaded_data = Data.load(save_path)  # optional
old_n_total = int(loaded_data.n_total)

abs_tol_tight = 1e-7
solver.set_tolerance(abs_tol=abs_tol_tight)
solution2, data2 = solver.integrate(resume=loaded_data)
loaded_data = Data.load(save_path)  # optional
old_n_total = int(loaded_data.n_total)

abs_tol_tight = 1e-7
solver.set_tolerance(abs_tol=abs_tol_tight)
solution2, data2 = solver.integrate(resume=loaded_data)

stage        iter    solution comb_bound_diff      n_min    n_total    elapsed_time      m      xfull.shape
-----------------------------------------------------------------------------------------------------------
RESUME          3  -0.4289312       1.955e-04       4096       4096       1.752e-03     12        (4096, 3)
ITER            4  -0.4289320       1.101e-04       4096       8192       3.140e-03     13        (8192, 3)
ITER            5  -0.4289320       1.444e-04       8192      16384       4.226e-03     14       (16384, 3)
ITER            6  -0.4289321       1.865e-05      16384      32768       6.040e-03     15       (32768, 3)
ITER            7  -0.4289321       9.302e-06      32768      65536       9.214e-03     16       (65536, 3)
ITER            8  -0.4289321       1.870e-06      65536     131072       1.634e-02     17      (131072, 3)
ITER            9  -0.4289321       7.475e-07     131072     262144       3.089e-02     18      (262144, 3)
ITER           10  -0.4289321       2.536e-07     262144     524288       6.287e-02     19      (524288, 3)
ITER           11  -0.4289321       9.139e-08     524288    1048576       1.269e-01     20     (1048576, 3)

Step 4: Compare with a Fresh Run¶

For reference, solve from scratch with the same tight tolerance to see how much work was saved.

In [6]:

Copied!





solver_fresh = make_CubQMCLatticeG_solver(abs_tol_tight, rel_tol=rel_tol, seed=7, dimension=3)
solver_fresh.trace_iterations = True
solution3, data3 = solver_fresh.integrate()

ru.print_stage_summary(
    resume_solver=solver,
    loose_data=data1,
    resume_data=data2,
    fresh_solver=solver_fresh,
    fresh_data=data3,
)
print(f"Solutions match: {abs(solution2.item() - solution3.item()) < 2 * abs_tol_tight}")
solver_fresh = make_CubQMCLatticeG_solver(abs_tol_tight, rel_tol=rel_tol, seed=7, dimension=3)
solver_fresh.trace_iterations = True
solution3, data3 = solver_fresh.integrate()

ru.print_stage_summary(
    resume_solver=solver,
    loose_data=data1,
    resume_data=data2,
    fresh_solver=solver_fresh,
    fresh_data=data3,
)
print(f"Solutions match: {abs(solution2.item() - solution3.item()) < 2 * abs_tol_tight}")

stage        iter    solution comb_bound_diff      n_min    n_total    elapsed_time      m      xfull.shape
-----------------------------------------------------------------------------------------------------------
ITER            1  -0.4289211       1.943e-03          0       1024       5.181e-04     10        (1024, 3)
ITER            2  -0.4289245       8.371e-04       1024       2048       1.242e-03     11        (2048, 3)
ITER            3  -0.4289312       1.955e-04       2048       4096       1.893e-03     12        (4096, 3)
ITER            4  -0.4289320       1.101e-04       4096       8192       2.536e-03     13        (8192, 3)
ITER            5  -0.4289320       1.444e-04       8192      16384       3.660e-03     14       (16384, 3)
ITER            6  -0.4289321       1.865e-05      16384      32768       5.331e-03     15       (32768, 3)
ITER            7  -0.4289321       9.302e-06      32768      65536       8.488e-03     16       (65536, 3)
ITER            8  -0.4289321       1.870e-06      65536     131072       1.523e-02     17      (131072, 3)
ITER            9  -0.4289321       7.475e-07     131072     262144       2.977e-02     18      (262144, 3)
ITER           10  -0.4289321       2.536e-07     262144     524288       6.166e-02     19      (524288, 3)
ITER           11  -0.4289321       9.139e-08     524288    1048576       1.232e-01     20     (1048576, 3)

Stage summary
---------------------------------------------------------------------------
Stage   abs_tol   total n     new n  iters     solution half-width time (s)
---------------------------------------------------------------------------
Loose     1e-04     4,096     4,096      3  -0.42893121   9.77e-05   0.0018
Resumed   1e-07 1,048,576 1,044,480     11  -0.42893206   4.57e-08   0.1256
Fresh     1e-07 1,048,576 1,048,576     11  -0.42893206   4.57e-08   0.1236
---------------------------------------------------------------------------
Solutions match: True

The above log prints one row per iteration as the solver runs. If the user sets trace_iterations to False, the log is not printed live, but it is still captured and stored in memory. The user can then retrieve the full, reviewable log later using get_iteration_log().

In [7]:

Copied!

solver.get_iteration_log()
solver_fresh.get_iteration_log()
solver.get_iteration_log()
solver_fresh.get_iteration_log()

Out[7]:

	stage	iter	solution	comb_bound_diff	n_min	n_total	elapsed_time	m	xfull.shape
0	ITER	1	-0.42892111	1.943e-03	0	1024	6.821e-04	10	(1024, 3)
1	ITER	2	-0.42892451	8.371e-04	1024	2048	1.244e-03	11	(2048, 3)
2	ITER	3	-0.42893121	1.955e-04	2048	4096	1.715e-03	12	(4096, 3)
3	RESUME	3	-0.42893121	1.955e-04	4096	4096	1.752e-03	12	(4096, 3)
4	ITER	4	-0.42893197	1.101e-04	4096	8192	3.140e-03	13	(8192, 3)
5	ITER	5	-0.42893200	1.444e-04	8192	16384	4.226e-03	14	(16384, 3)
6	ITER	6	-0.42893212	1.865e-05	16384	32768	6.040e-03	15	(32768, 3)
7	ITER	7	-0.42893205	9.302e-06	32768	65536	9.214e-03	16	(65536, 3)
8	ITER	8	-0.42893206	1.870e-06	65536	131072	1.634e-02	17	(131072, 3)
9	ITER	9	-0.42893206	7.475e-07	131072	262144	3.089e-02	18	(262144, 3)
10	ITER	10	-0.42893206	2.536e-07	262144	524288	6.287e-02	19	(524288, 3)
11	ITER	11	-0.42893206	9.139e-08	524288	1048576	1.269e-01	20	(1048576, 3)

Out[7]:

	stage	iter	solution	comb_bound_diff	n_min	n_total	elapsed_time	m	xfull.shape
0	ITER	1	-0.42892111	1.943e-03	0	1024	5.181e-04	10	(1024, 3)
1	ITER	2	-0.42892451	8.371e-04	1024	2048	1.242e-03	11	(2048, 3)
2	ITER	3	-0.42893121	1.955e-04	2048	4096	1.893e-03	12	(4096, 3)
3	ITER	4	-0.42893197	1.101e-04	4096	8192	2.536e-03	13	(8192, 3)
4	ITER	5	-0.42893200	1.444e-04	8192	16384	3.660e-03	14	(16384, 3)
5	ITER	6	-0.42893212	1.865e-05	16384	32768	5.331e-03	15	(32768, 3)
6	ITER	7	-0.42893205	9.302e-06	32768	65536	8.488e-03	16	(65536, 3)
7	ITER	8	-0.42893206	1.870e-06	65536	131072	1.523e-02	17	(131072, 3)
8	ITER	9	-0.42893206	7.475e-07	131072	262144	2.977e-02	18	(262144, 3)
9	ITER	10	-0.42893206	2.536e-07	262144	524288	6.166e-02	19	(524288, 3)
10	ITER	11	-0.42893206	9.139e-08	524288	1048576	1.232e-01	20	(1048576, 3)

Key Takeaways¶

The resume is most useful when the initial run already performed meaningful work or when checkpointing across long-running sessions is needed. For small examples, wall-clock timing differences may be negligible; the main demonstrated benefit here is correctness of state reuse.