Compiling a smart contract does not prove behavioral equivalence

A contract can compile, expose the expected functions, and still execute rules that differ from the original. In programmable finance, that difference is not an implementation detail: it can alter balances, permissions, supply, events, reverts, or valid transaction sequences.

Status: Evidence Brief approved for publication. Editorial series: Papers we read for you. This text analyzes the preprint *SolBehave: A Behavioral Benchmark for LLM-Generated Smart Contracts*. It does not claim that Trustyu Forge reproduced the experiments, that equivalence under the evaluated oracles proves universal correctness, or that the percentages transfer to another model, prompt, or domain.

What SolBehave measures

The authors started from 8,360 verified Ethereum contracts. They reproduced 2,098 and selected 250 non-proxy contracts with one to twenty nontrivial functions; the reported median was 16 functions per contract. The sample favors token-like contracts and excludes larger systems, proxies, and a relevant share of production complexity.

Seven open-weight models were tasked with regenerating the contract. Each combination had ten generations and one self-repair round, with 4-bit quantization and a uniform configuration. The comparison separated compilation success from observable equivalence.

Compilation ranged from 26.4% to 92.7%

Compilation rates ranged from 26.4% to 92.7%, depending on the model. This gap already shows that producing a syntactically accepted artifact remains sensitive to capability and configuration. The most important result, however, appears after the build: in aggregate, only 31.4% of compiled function bodies were classified as equivalent under differential fuzzing.

Compilation answers whether the artifact satisfies the toolchain's grammar, types, and static rules. It does not answer whether state transitions, authorization, or observable effects remain the same.

Three oracles looked for divergence

The benchmark combined three forms of observation:

1. Differential fuzzing: executes inputs on the original and regenerated contracts and compares observable outputs, state, events, and failures.
2. Historical replay: replays known real transactions, with 1,000 transactions per contract in the described experiment.
3. Dependency-guided synthetic replay: creates sequences from read-write relations between functions, seeking states that isolated calls do not reach.

For six of the seven models, 13% to 31% of contracts diverged under historical replay. Under dependency-guided synthetic replay, the range was 13% to 44%. Gemma-4-31B stayed below 3% in both reported replays. This result is specific to the configuration and does not prove general superiority, absence of defects, or safety of generated code.

Observed equivalence is not proven security

A contract can agree on every executed input and still diverge on an ungenerated sequence. It can also faithfully reproduce unsafe behavior from the original. SolBehave's oracles therefore add evidence, but they do not replace specification, access analysis, invariants, security audit, and review of economic changes.

Dependency-guided replay helps reach compound states, but it may miss paths that accept invalid transactions or depend on uncaptured relations. Comparing status, return values, and state is an observational approximation—not a formal proof of equivalence.

A minimum contract for generation and acceptance

1. Source and rights: record the observed contract, version, license or authorization, and purpose of the reimplementation.
2. Specification: declare balance, supply, access, event, failure, and transition invariants before accepting the code.
3. Reproducible build: pin the compiler, dependencies, optimizations, network, and parameters.
4. Independent oracles: combine property tests, differential fuzzing, historical replay, and adversarial sequences.
5. Separation: do not let the same trajectory generate the implementation and decide by itself whether it is equivalent.
6. Receipts: preserve the model, prompt, artifact, bytecode, seeds, transactions, divergences, and human decision.
7. Gradual promotion: keep value limits, least privilege, pause, and rollback controls proportional to the impact.

Architecture rule

The “compiled” receipt proves only one stage. Promotion requires a chain connecting intent, invariants, reproducible build, differential behavior, security, and human acceptance. When software moves assets or executes irreversible rules, the absence of that chain is evidence debt, not merely a testing gap. [R2-C3]

In Forge, the harness must prevent a single signal—compilation, a green test, or textual similarity—from being promoted to complete proof. Complementary evidence must remain identifiable, with its own boundaries and owners.

Limits and conflicts of interest

The paper is a preprint submitted to IEEE. The final sample contains 250 contracts, derived from a funnel of 8,360 candidates, and favors token-like contracts with up to 20 functions. Executions begin from a fresh state, which does not represent every migration or production history. Seven open-weight models, a uniform configuration, and one self-repair round do not cover every possible agent, toolchain, or strategy.

Dependency validation was performed manually by one expert across 30 contracts and 200 relations, which limits the error estimate for this component. By the editorial cutoff, we found no complete independent reproduction or specific public response from vendors. The authors report affiliations with the University of Bologna and the University of Copenhagen; we found no model-vendor commercial conflict. Absence of an identified conflict is not proof of absolute independence.

Direct sources

1. SolBehave: A Behavioral Benchmark for LLM-Generated Smart Contracts — arXiv. Preprint, authors, abstract, version, and date.
2. SolBehave — full text. Method, sample funnel, configurations, oracles, results, and limitations.

Editorial and responsibility note

This text combines facts attributed to sources with the author's analysis and technical proposal. Personal and professional views are not proven facts; data, denominators, limitations, affiliations, and the absence of independent reproduction are stated when available. The content is informational and does not replace technical, legal, financial, or security assessment. Ethereum, contracts, products, and models mentioned belong to their respective owners; there is no sponsorship or endorsement. Tech Human and Trustyu operate commercially in digital-product development and governance. Research, structure, and writing were AI-assisted; factual review, authorial approval, and publication were authorized by Fernando Parreiras on October 14, 2026, without additional independent human review.

Research cutoff: October 14, 2026. Editorial status: special Evidence Brief authorized for October 27, 2026 at 09:00 BRT.