Discovering a force law with controlled probes

A language model probes a black box that pulls a test body toward a centre, one control at a time, and submits a force law; the build scores the claim on distances it could not probe and sets it beside three baselines that use no model.

Course

Classical mechanics, lesson 18 of 28

Slot

D1, number 1 of the 13 slots of the path Scientific law discovery

Methods

Simulation, LLM agents

Difficulty

Intermediate, level 2 of 3

What you’ll learn

Design controlled probes that separate the effect of each control on a force
Read an exponent from the ratio of two forces at two distances, and say how the noise limits it
Score a claimed law on settings outside the probed range, and compare it with baselines that use no model
Say what a good score of a recorded run does and does not show about the agent

Prerequisites

An agent runs an experiment, Inverse physics: parameters from observations.

Before you begin

Logarithms: the logarithm of a product and of a power
Large language models: a program that continues a text

Related

none

Previous

An agent that designs the measurements

Next

Full, partial, and no prior knowledge under one protocol

With scripts, this page keeps a record in your browser of whether you completed the lesson. Nothing is stored without them.

With scripts, this place suggests what to do next, with the reason, from the progress saved in your browser. Without them, follow the learning path.

How did anyone find out that gravity falls with the square of the distance? Not by guessing a formula and admiring it. A law of force is found by changing one thing at a time: keep the body, move it away, and see how much weaker the pull becomes; keep the distance, double the body, and see whether the pull doubles. Each such controlled probe isolates the effect of one control. A few of them, chosen well, say more than many chosen at random.

This lesson gives that job to a language model. Behind the tools is a simulated black box: a fixed centre that pulls a test body with a force that depends on the distance and on the mass of the body. The law, its coefficients, and the settings on which the claim will be judged are hidden from the model, and from this page. The model may set the two controls, read a noisy force, and finally submit two numbers, a strength and an exponent.

The build then does what the model cannot influence. It replays the recorded run and recomputes every reading. It scores the submitted law on distances beyond any the model could reach, where a wrong exponent shows. And it runs three methods without a model on the same box with the same budget, so that the agent’s result has something to be compared with.

Assumptions

The black box is a simulation with a known family of laws. The force is $F = k\,m\,r^{-p}$ , strength $k$ times mass $m$ times distance $r$ to the power $-p$ , for some hidden strength and exponent: the central force of the Discovery evaluation protocol (CentralForceEnvironment of pbc.discovery). A claim is made in that family. If the true law were outside it, no claim could be right, and the least-squares baseline would say so through its chi-square.
Each reading has Gaussian noise of known size. The noise is a function of the seed and the controls, so the same probe gives the same reading and the replay is exact. A new probe gives an independent reading only when a control differs.
The rules come first. The budget, the step limit, the tolerance on the held-out error that counts as recovering the law, and the hidden world were fixed in pbc.discovery.force_lab before the run was recorded. Nothing is widened afterwards.
The agent is untrusted. It reaches the box only through an allowlist of two tools with bounded arguments, and its words are data. The score is a function of its tool calls alone.
The prior knowledge is “partial”. The model is told the controls, the observation, and the names of the two parameters to report, not the form of the law. The tool that takes the claim describes its parameters, so the names hint at a power of the distance; that is part of the condition.

Controlled probes and a held-out test

Write the force as

$F(r, m) = k\, m\, r^{-p},$

with $r$ the distance, $m$ the mass, $k$ the strength, and $p$ the exponent. Two kinds of probe separate the controls.

Hold the distance, change the mass. If $F$ doubles when $m$ doubles, the force is proportional to the mass, and $F/m$ depends on the distance alone.
Hold the mass, change the distance. Take the logarithm: $\ln F = \ln(k m) - p \ln r$ . Between two distances $r_1 < r_2$ the exponent is the slope $p = \frac{\ln F(r_1) - \ln F(r_2)}{\ln r_2 - \ln r_1}.$

Without noise two readings would fix $p$ exactly. With noise of standard deviation $\sigma$ on each force, the error of each logarithm is about $\sigma / F$ , so the error of the slope is about

$\delta p \approx \frac{\sigma\sqrt{1/F_1^{2} + 1/F_2^{2}}}{\ln(r_2/r_1)}.$

That formula is the design, and it holds a trade-off. A wide ratio of distances makes the denominator large. But the force falls as $r^{-p}$ , so the far reading is the weak one, its relative noise $\sigma/F_2$ grows as $r_2^{\,p}$ , and it dominates the error. Moving the far probe out helps only while the logarithm in the denominator grows faster than the noise in the numerator. A heavy body makes every force larger and every relative noise smaller, at no cost. A probe at a middle distance also tests whether the logarithm is a straight line at all, which is a check of the family, not only a sharper estimate.

Why the claim is scored where nobody probed. The controls are bounded. A claim that fits the readings inside the bounds can still be wrong outside them, and the further outside, the more a small error in $p$ grows: the relative error of a prediction at distance $r$ is about $\delta p \ln(r/r_0)$ for a law matched near $r_0$ . The protocol therefore scores the claim on held-out settings beyond the largest distance the tools accept. The held-out error is the root-mean-square difference between the claim’s prediction and the noise-free law there, relative to the size of the force. The law counts as recovered when that error is at most the tolerance of the protocol, 0.05, and the run ended on its own.

What the agent may and may not do

measure: Set the controls and read the force (N), with noise. Each call uses one of 20 probes.
- distance: distance from the centre (m); allowed values 1 to 4
- mass: mass of the test body (kg); allowed values 0.5 to 5
submit: State your claim as the parameters of the law and end the run. It uses no probe.
- strength: coefficient of the law (N m^p/kg); allowed values 0.1 to 100
- exponent: power p of the distance (1); allowed values 0 to 6

The agent may call measure at most 20 times, call submit to state its claim, send at most 12 messages with at most 4 tool calls each, and write text. It may not run code, read a file, reach the network, or call the judge. The law, its parameters, and the held-out settings are in no tool, no prompt, and no tool description; a test of pbc.discovery checks that for every environment.

Code

The score is recomputed from the calls of the transcript alone. A fresh box of the same seed receives the same calls, the last accepted claim is scored, and nothing the agent wrote is read.

def score_transcript(protocol: Protocol, transcript: Transcript, seed: int) -> Score:
    """Score ``transcript`` by running its tool calls again on a fresh
    environment of ``seed``. The same calls give the same score."""
    environment = protocol.make(seed)
    allowlist = environment_allowlist(environment)
    probes = refused = 0
    claim: dict[str, float] | None = None
    for step in transcript.steps:
        for number, call in enumerate(step.message.tool_calls):
            # The loop refuses the calls of a message beyond its limit unrun.
            if number >= MAX_CALLS_PER_MESSAGE:
                refused += 1
                continue
            result = allowlist.dispatch(call)
            if not result.ok:
                refused += 1
            elif call.name == MEASURE:
                probes += 1
            elif call.name == SUBMIT:
                claim = {k: float(v) for k, v in call.arguments.items()}
    error = math.inf if claim is None else environment.held_out_error(claim)
    recovered = transcript.finished and error <= protocol.tolerance
    return Score(seed, probes, refused, claim, error, recovered)

src/pbc/discovery/protocol.py, lines 161 to 183, read from the repository when this page was built.

The baselines spend the same budget without a model: on a regular grid, at random, and on the grid again with sparse identification, which looks for the force as a short sum of the terms $m\,r^{-n}$ for whole numbers n. Each reports success only when its fit is consistent with the noise.

def run_baselines(
    environment: Environment, seed: int = 0
) -> tuple[BaselineResult, ...]:
    """Every baseline on ``environment`` with its budget: least squares on a
    grid and on random probes, and sparse identification on the grid."""
    grid = grid_probe(environment)
    scattered = random_probe(environment, seed)
    terms = library(environment)
    results = (
        least_squares(environment, grid, "grid + least squares"),
        least_squares(environment, scattered, "random + least squares"),
        sparse_identification(
            environment, grid, terms, name="grid + sparse identification"
        ),
    )
    environment.reset()
    return results

src/pbc/discovery/baselines.py, lines 213 to 229, read from the repository when this page was built.

def least_squares(
    environment: Environment, data: Dataset, name: str = "least-squares"
) -> BaselineResult:
    """Fit the environment's public family to ``data`` with
    ``pbc.data.fit.fit_curve``, from each of its starts, and keep the best."""
    names = tuple(c.name for c in environment.claim)
    bounds = (
        [c.low for c in environment.claim],
        [c.high for c in environment.claim],
    )
    if len(data.settings) <= len(names):
        return _failed(name, data, "fewer probes than parameters")

    def curve(index: Array, *params: float) -> Array:
        claim = dict(zip(names, params, strict=True))
        return np.array(
            [environment.predict(claim, data.settings[int(i)]) for i in index]
        )

    best = None
    for start in environment.starts:
        try:
            fit = fit_curve(
                curve,
                np.arange(len(data.values), dtype=float),
                data.values,
                start,
                sigma=data.sigma,
                names=names,
                bounds=bounds,
            )
        except ValueError:
            continue
        if best is None or fit.chi2 < best.chi2:
            best = fit
    if best is None:
        return _failed(name, data, "no fit could be made")
    claim = dict(zip(names, map(float, best.params), strict=True))
    error = environment.held_out_error(claim)
    if not consistent(best.chi2, best.dof):
        reason = (
            f"chi-square {best.chi2:.3g} for {best.dof} degrees of freedom: the"
            " family does not describe the measurements"
        )
    elif not all(best.identifiable):
        reason = "the data do not determine the parameters separately"
    else:
        reason = ""
    return BaselineResult(name, len(data.settings), not reason, claim, error, reason)

src/pbc/discovery/baselines.py, lines 107 to 155, read from the repository when this page was built.

Worked examples

The recorded run

    recording, transcript, score = replay_run(RUN, LESSON)

src/pbc/mechanics/experiments/black_box_force_law.py, line 239, read from the repository when this page was built.

Recorded run. The model messages below were recorded from claude-sonnet-5-5 in a Claude Code session on 2026-10-16. This build did not call a model. It ran every tool call again; the results shown are the ones it computed.

Step 1. Model message (recorded):

Taking measurements to find the law. Four probes: distance 1, 2 and 4 m at mass 1 kg, and distance 2 m at mass 4 kg.

measure(distance=1, mass=1): recomputed by this build: force = 20.0041
measure(distance=2, mass=1): recomputed by this build: force = 5.00673
measure(distance=4, mass=1): recomputed by this build: force = 1.2573
measure(distance=2, mass=4): recomputed by this build: force = 19.9597

Step 2. Model message (recorded):

Force scales linearly with mass and falls as 1/d^2: 20.0 N at 1 m, 5.0 N at 2 m and 1.26 N at 4 m for 1 kg, and 4 times larger at 4 kg. Submitting strength 20 and exponent 2.

submit(strength=20, exponent=2): recomputed by this build: submitted = 1

Step 3. Model message (recorded):

The submission was accepted (submitted: 1.0). The claim is force = 20 * mass / distance^2, so strength 20 N m^2/kg and exponent 2. It rests on four probes: 20.0 N, 5.0 N and 1.26 N at 1, 2 and 4 m with 1 kg, and 19.96 N at 2 m with 4 kg. The run is finished.

The model ended the run with a message that requests no tool.

The model was a Claude model working in a Claude Code session, recorded on 2026-10-16: a fresh sub-agent that was given the system prompt, the task, and the tool descriptions above, and nothing else. It chose every call after reading the readings the harness computed; the session relayed them. It was not called through the provider API, and the session reported no token count or price.

The score, recomputed

def score_table(experiment: Experiment, score) -> None:
    """The agent's run scored again from its tool calls; record the table."""
    claim = score.claim
    experiment.table(
        "d1-score",
        [
            ("probes used", score.probes, ""),
            ("calls refused", score.refused, ""),
            ("claimed strength", claim["strength"], "N m^p/kg"),
            ("claimed exponent", claim["exponent"], ""),
            ("held-out error", shown(score.held_out_error), "relative"),
            ("tolerance", TOLERANCE, "relative"),
            ("outcome", score.outcome, ""),
        ],
        [("quantity", "", ""), ("value", "", ""), ("unit", "", "")],
        "The agent's run scored again by the build from its tool calls alone.",
    )

src/pbc/mechanics/experiments/black_box_force_law.py, lines 61 to 77, read from the repository when this page was built.

The agent's run scored again by the build from its tool calls alone.
quantity	value	unit
probes used	4
calls refused	0
claimed strength	20.0	N m^p/kg
claimed exponent	2.0
held-out error	below 1e-12	relative
tolerance	0.05	relative
outcome	recovered

Numerically verified (the simulated central force of this protocol, one recorded run, one seed): The law the agent submitted predicts the force at the held-out distances within the tolerance of the protocol: the outcome is recovered. The model used 4 of its 20 probes: three distances at one mass and one mass at a second distance, the two kinds of controlled probe above, spanning the whole range of distances at a light body.

What the probes themselves support

The agent submitted round numbers. Were they what its four readings show, or what it expected? Fit the family to exactly the readings it saw, with the standard errors that the noise gives:

def probe_fit(experiment: Experiment, transcript):
    """The family fitted to exactly the readings the agent saw; record the
    table and return the readings and the fitted law."""
    data = probes(RUN, transcript)
    fit = fit_probes(RUN, data)
    own = dict(zip(fit.names, map(float, fit.params), strict=True))
    experiment.table(
        "d1-probe-fit",
        [
            (name, value, error)
            for name, value, error in zip(
                fit.names, fit.params, fit.stderr, strict=True
            )
        ]
        + [("held-out error of this fit", environment(RUN).held_out_error(own), None)],
        [
            ("parameter", "", ""),
            ("fitted value", "", ".4g"),
            ("standard error", "", ".2g"),
        ],
        "The family fitted to the agent's four readings, as recomputed by the replay.",
    )
    return data, own

src/pbc/mechanics/experiments/black_box_force_law.py, lines 80 to 102, read from the repository when this page was built.

The family fitted to the agent's four readings, as recomputed by the replay.
parameter	fitted value	standard error
strength	20	0.02
exponent	2.003	0.002
held-out error of this fit	0.005837

Numerically verified (the four readings of the recorded run): A least-squares fit to the agent’s own readings also recovers the law, with standard errors of the size the noise allows. The fit does not land exactly on the round numbers. The agent’s claim scored better than its data alone could guarantee, because its rounding happened to agree with the world. A claim that matches by rounding and a claim that matches by measurement look the same in one score; the lesson on a modified law asks what happens when the familiar round number is wrong.

Beside the baselines

def beside_the_baselines(experiment: Experiment, score) -> None:
    """The agent's run beside three baselines without a model; record the
    table."""
    rows = comparison((score,), baselines(RUN), TOLERANCE)
    experiment.table(
        "d1-baselines",
        [
            (
                r["who"],
                r["probes"],
                shown(r["held_out_error"]),
                "yes" if r["recovered"] else "no",
                r["note"],
            )
            for r in rows
        ],
        [
            ("method", "", ""),
            ("probes", "", "d"),
            ("held-out error", "", ""),
            ("recovered", "", ""),
            ("note", "", ""),
        ],
        "The agent's run and three baselines without a model, on the same box"
        " with the same budget.",
    )

src/pbc/mechanics/experiments/black_box_force_law.py, lines 105 to 130, read from the repository when this page was built.

The agent's run and three baselines without a model, on the same box with the same budget.
method	probes	held-out error	recovered	note
agent, seed 0	4	below 1e-12	yes	recovered
grid + least squares	20	0.0013	yes	fit consistent with the noise
random + least squares	20	9.46e-05	yes	fit consistent with the noise
grid + sparse identification	20	1.21e-05	yes	fit consistent with the noise

The baselines spent the whole budget and the agent a fraction of it. Least squares on the grid and on random settings both recover the law; sparse identification does as well here, since the law is one of its library’s terms. The comparison is not a ranking of methods: it is one box, one seed, and one recorded run.

The probes on a logarithmic scale

def draw_force_probes(data, own) -> Figure:
    """Force per unit mass against distance on logarithmic axes: the grid's
    readings, the agent's, and the law fitted to the agent's, extrapolated
    beyond the bounds."""
    grid = grid_probe(environment(RUN))
    distances = np.geomspace(1.0, 12.0, 100)
    fig, ax = plt.subplots(figsize=(6.4, 3.6))
    ax.loglog(
        [s["distance"] for s in grid.settings],
        grid.values / np.array([s["mass"] for s in grid.settings]),
        "o",
        color="#9aa4ae",
        markersize=4,
        label="grid baseline readings",
    )
    ax.loglog(
        [s["distance"] for s in data.settings],
        data.values / np.array([s["mass"] for s in data.settings]),
        "o",
        color="#93003a",
        markersize=7,
        label="agent's readings",
    )
    inside = distances <= 4.0
    line = own["strength"] * distances ** -own["exponent"]
    ax.loglog(
        distances[inside],
        line[inside],
        color="#00429d",
        label="fit to the agent's readings",
    )
    ax.loglog(
        distances[~inside],
        line[~inside],
        "--",
        color="#00429d",
        label="extrapolated beyond the bounds",
    )
    ax.axvline(4.0, color="#57606a", linestyle=":")
    ax.set_xlabel("distance (m)")
    ax.set_ylabel("force per unit mass (N/kg)")
    ax.legend(fontsize=8)
    return fig

src/pbc/mechanics/experiments/black_box_force_law.py, lines 133 to 175, read from the repository when this page was built.

A log-log chart of force per unit mass against distance from one to four metres. Grey dots of the grid baseline and four larger dots of the agent fall on one straight descending line. The line is drawn from the fit to the agent's readings and extended past four metres as a dashed line toward the region that is scored. — Figure 1: simulated Force per unit mass against distance on logarithmic axes: the agent’s four readings lie on the straight line of the law fitted to them, and the grid baseline’s readings lie on the same line; its slope is the exponent.

The figure shows simulated readings, recomputed by this build. On logarithmic axes the law is a straight line, and the controlled probes of the agent cover its whole width. The dotted line is the largest distance the tools accept. Everything to its right is the extrapolation the held-out score tests, and no reading exists there.

Run the agent yourself

This needs your own key and costs a few cents. The build does not run it, so the commands are marked.

Not verified. The build of this site did not run or check the material in this box.

uv sync --locked --no-dev --extra openai
export OPENAI_API_KEY="..."
cd site/lessons/agents-llm/05-discovering-a-force-law
uv run --locked --no-dev --extra openai python -m pbc.discovery.force_lab black-box

The program prints each message and reading and the held-out error it computed. With --record it writes replay.json in the current directory; the record of the run in pbc.discovery.force_lab then needs the new model, date, and cost, which the program prints.

Limits

One run of one model. The page shows what one model once did with this box. It does not say how often a model finds the law, and a live run will probe elsewhere.
A score is about the simulated box. It says how well a claim predicts this simulation beyond the probed range. Nothing here is a measurement of a real force, and nothing says the law is true of nature.
The family is given by the claim’s parameters. A claim can only be a strength and an exponent. A law of another form, such as a force that changes sign, cannot be stated, and the agent is not tested on finding the form.
A perfect score can come from rounding. The agent’s round numbers matched the world exactly, which its four readings alone could not guarantee. The score cannot tell good judgement from a lucky prior.
The recording is not an API run. The messages are those of a Claude model in a Claude Code session, which was told by the session to answer in the harness’s format and to use no tool of its own; it was asked once, before its first message, to send its reply again in that format. A run through the provider API may differ.
Sparse identification is fragile on a collinear library. Its threshold is relative to the largest coefficient. When the terms are nearly proportional over the probed range it can drop the right term; the first exercise shows it.

Self-check. The agent’s claim scored a held-out error that rounds to zero. Does that show that its four readings determined the strength and the exponent to that precision?

Yes: the held-out error measures how precisely the readings determine the parameters.

The held-out error compares the claim with the law; it knows nothing of how the claim was reached. A guess that happens to be right scores as well as a measurement.
No: the fit to the same readings has standard errors well above zero, so the exact match came from the agent’s rounding, not from the precision of its data.

Right. The readings support the law within their standard errors, and the claim fell inside them on a round number. The score is the same either way.

Solution to the self-check

No. The score compares the claim with the hidden law on the held-out settings and says nothing about how the claim was reached. The least-squares fit to the agent’s four readings gives values close to the claim but not equal to it, with standard errors set by the noise. The claim’s exact match is the agreement of a round number with the world. That is why a protocol reports the score together with the evidence the claim rests on, and why the modified-law lesson puts a world where the round number is wrong.

Exercises

Exercise 1. Make a box of your own with strength 9 and exponent 2, spend a budget of 20 on the grid, and run sparse identification and least squares on it. One succeeds and one fails. Which, and why?

Solution to exercise 1

def sparse_on_a_collinear_library(experiment: Experiment) -> None:
    """Exercise 1: least squares and sparse identification on a box of
    strength 9 and exponent 2; record the table."""
    own_box = CentralForceEnvironment(strength=9.0, exponent=2.0, budget=20)
    own_grid = grid_probe(own_box)
    results = [
        least_squares(own_box, own_grid, "grid + least squares"),
        sparse_identification(
            own_box, own_grid, library(own_box), name="grid + sparse identification"
        ),
    ]
    experiment.table(
        "own-box",
        [
            (
                r.name,
                "yes" if r.success else "no",
                r.reason or "fit consistent with the noise",
            )
            for r in results
        ],
        [("method", "", ""), ("success", "", ""), ("reason", "", "")],
        "Two baselines on a box with an inverse-square law of strength 9.",
    )

src/pbc/mechanics/experiments/black_box_force_law.py, lines 178 to 201, read from the repository when this page was built.

Two baselines on a box with an inverse-square law of strength 9.
method	success	reason
grid + least squares	yes	fit consistent with the noise
grid + sparse identification	no	chi-square 1.37e+04 for 18 degrees of freedom: no sparse sum of the library's terms describes the measurements

Least squares fits the two parameters of the family and succeeds. Sparse identification fails, although the law is exactly one of its terms. Over distances from one to four metres the terms of the library are nearly proportional to each other, so the first unconstrained fit spreads the force over several of them with large coefficients of opposite sign. The threshold is relative to the largest coefficient, and the right term falls under it and is removed. The baseline then reports that no sparse sum fits, which is a statement about its procedure, not about the law. A baseline is a method with failure modes of its own; that is why its failures are shown.

Exercise 2. Use the error formula of the explanation. The noise is 0.02 N on each reading. For a body of 5 kg, estimate the error of the exponent from two readings at 1 m and 4 m, given a force of about 100 N at 1 m and 6 N at 4 m. Then do the same at 1 m and 2 m, given 100 N and 25 N. Which pair is better, and by how much?

Solution to exercise 2

def slope_errors(experiment: Experiment) -> None:
    """Exercise 2: the error of the exponent from two readings, for a wide
    and a narrow pair of distances; record the table."""
    sigma = environment(RUN).observation.sigma

    def slope_error(f1, f2, r1, r2):
        return sigma * math.sqrt(1 / f1**2 + 1 / f2**2) / math.log(r2 / r1)

    wide, narrow = slope_error(100, 6, 1, 4), slope_error(100, 25, 1, 2)
    experiment.table(
        "slope-errors",
        [
            ("1 m and 4 m", wide),
            ("1 m and 2 m", narrow),
            ("ratio, narrow over wide", narrow / wide),
        ],
        [("pair of distances", "", ""), ("error of the exponent", "", ".2g")],
        "The error of the exponent from two readings, by the formula of the"
        " explanation.",
    )

src/pbc/mechanics/experiments/black_box_force_law.py, lines 204 to 223, read from the repository when this page was built.

The error of the exponent from two readings, by the formula of the explanation.
pair of distances	error of the exponent
1 m and 4 m	0.0024
1 m and 2 m	0.0012
ratio, narrow over wide	0.49

The narrower pair is better, by the ratio in the table. Going from two metres to four doubles the logarithm in the denominator, but it makes the far force four times weaker and its relative noise four times larger, and the far reading dominates. When it does, the error behaves as $r_2^{\,p}/\ln(r_2/r_1)$ , which is smallest at $r_2/r_1 = e^{1/p}$ , well inside the bounds here. The widest span is still useful for another reason: it tests the straight line over the whole range, and the held-out settings lie beyond it.

Replay this experiment

This page was built from commit 6417e2ff90258bd2d2ba21d217eb7666fd24f680 of the repository.

In a clone at that commit, with the project’s Python packages installed, this command runs the experiment of this page:

uv run --locked --no-dev python -m pbc.mechanics.experiments.black_box_force_law results

Files, tests, and every command

The files behind it:

site/lessons/agents-llm/05-discovering-a-force-law/index.qmd: this page, with every code cell.
src/pbc/discovery/force_lab.py: code the experiment runs.
src/pbc/discovery/environment.py: code the experiment runs.
src/pbc/discovery/protocol.py: code the experiment runs.
src/pbc/discovery/baselines.py: code the experiment runs.
src/pbc/mechanics/experiments/black_box_force_law.py: code the experiment runs.
src/pbc/experiment.py: code the experiment runs.
tests/unit/test_force_lab.py: tests of that code.
tests/unit/test_discovery.py: tests of that code.
tests/unit/test_black_box_force_law_experiment.py: tests of that code.

With Git and uv installed (see the setup page), run:

git clone https://github.com/taspinar/physics-by-construction.git
cd physics-by-construction
git checkout 6417e2ff90258bd2d2ba21d217eb7666fd24f680
uv sync --locked --no-dev
uv run --locked --no-dev python -m pbc.mechanics.experiments.black_box_force_law results

The experiment writes every figure of this page into results/, as <label>.png, and every number and table of the page, as the page shows it, into results/results.json. Its numbers match this page to the digits shown, and its figure files are the ones the page shows; digits beyond those can differ between machines, because floating-point libraries differ. Change the code of the experiment and run it again to see what changes.

The experiment replays the recorded run of the agent: it runs every tool call of the recording again and needs no model and no key.

Not verified. The build of this site did not run or check the material in this box.

Optional: the live run. It asks a language model to run the experiment again, with your own account. It needs an OpenAI account and its API key in the variable OPENAI_API_KEY, and the optional dependency openai; a run costs a few cents. The key stays in your shell, never in a file.

# needs: an OpenAI account, its API key, and the extra openai
uv sync --locked --no-dev --extra openai
export OPENAI_API_KEY="..."   # this shell only
uv run --locked --no-dev --extra openai python -m pbc.discovery.force_lab black-box