Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Changelog

## Unreleased

### New Features

- Allow `simple_stub_any_instance_of` and `simple_stub` inside `around_suite`
and `around_test`, with automatic hook-scoped reset and nested stub
restoration. If the wrapped operation and stub teardown both fail, report
both errors without replacing the primary failure.

## 0.7.0

### Breaking Changes
Expand Down
42 changes: 38 additions & 4 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,15 @@ smartest/
parameter_extractor.rb

execution_context.rb
hook_contexts.rb
hook_chains.rb

expectations.rb
expectation_target.rb
matchers.rb
simple_stub.rb
simple_stub_dsl.rb
teardown_error_aggregator.rb
constant_stub_helpers.rb

runner.rb
Expand Down Expand Up @@ -132,9 +136,13 @@ around_suite(&block)
around_test(&block)
```

`use_fixture(klass)` and `use_matcher(matcher_module)` are not top-level DSL
methods. They are available only from hook execution contexts: `around_suite`
and `around_test`.
`use_fixture(klass)`, `use_matcher(matcher_module)`,
`simple_stub_any_instance_of`, and `simple_stub` are not top-level DSL methods.
They are available only from the relevant hook execution contexts:
`around_suite` and `around_test`. Method stub helpers are also available from
fixture blocks, where fixture teardown owns their reset.
If a lifecycle owner cannot register an applied stub for teardown, the helper
immediately resets it before propagating the registration error.

Possible later methods:

Expand Down Expand Up @@ -295,12 +303,36 @@ Responsibilities:
- cache fixture values for its scope
- collect teardown blocks
- run teardown blocks in reverse order
- drain the teardown queue before callbacks and run it at most once
- expose teardown failures through `teardown_errors`
- detect duplicate fixture names
- detect circular dependencies

Important: regular fixture values must not leak across tests. Suite fixture
values are intentionally shared across the run.

### Hook contexts and chains

`AroundSuiteContext` and `AroundTestContext` own the state for one hook
invocation. Their common `HookContext` resets method stubs from
`run_teardowns` at most once and exposes any cleanup failures through
`teardown_errors`. Repeated or reentrant teardown calls preserve the errors
recorded by the first call.

`AroundSuiteHookChain` and `AroundTestHookChain` own hook composition and the
required `suite.run` / `test.run` protocol. Each chain runs its contexts'
teardowns in `ensure` and uses `TeardownErrorAggregator` to collect their
cleanup failures. The chain then acts as one teardown error source for the
runner.

### `Smartest::TeardownErrorAggregator`

Collects teardown failures from completed lifecycle owners. Sources expose
`teardown_errors`; the aggregator reads that state after cleanup instead of
passing a shared mutable error array into the operation that performs cleanup.
These lifecycle implementation classes are intentionally omitted from the
public DSL RBS.

### `Smartest::ExecutionContext`

Object used as `self` when running a test body.
Expand All @@ -326,7 +358,7 @@ Runs tests.
Responsibilities:

- iterate over registered test cases
- run registered `around_suite` hooks around the suite body
- run registered hooks through their hook chain
- create a lazy suite-scoped `FixtureSet`
- create a fresh `ExecutionContext` per test
- create a fresh `FixtureSet` per test
Expand All @@ -335,6 +367,7 @@ Responsibilities:
- run teardown in `ensure`
- track skipped and pending test state
- run suite fixture teardown after all tests
- aggregate teardown errors from fixture sets and hook chains
- produce `TestResult`
- notify reporter

Expand Down Expand Up @@ -671,6 +704,7 @@ A practical approach:
- snapshot file-local hooks when each test is registered
- run hooks around fixture setup, test body, and fixture teardown
- expose `use_fixture` and `use_matcher` only inside hook contexts
- expose method stub helpers inside hook contexts with automatic hook-scoped reset
- make `around_test` registered from `around_suite` suite-wide

### Phase 9: Skipped and pending tests
Expand Down
54 changes: 42 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -722,18 +722,23 @@ Register teardown immediately after acquiring the resource, before later setup s

## Stubs

Use simple stub helpers when a fixture needs to temporarily replace a Ruby
method and reset it during teardown:
Put method stubs in fixtures by default so the stubbed state is visible in the
test signature:

```ruby
class ApplicationTestFixture < Smartest::Fixture
fixture :payment_gateway_stub do
simple_stub_any_instance_of(PaymentGateway, :charge) { :approved }
fixture :user do
User.create!(name: "Alice")
end

fixture :logged_in_user do |user:|
simple_stub_any_instance_of(ApplicationController, :current_user) { user }
user
end
end
```

Register the fixture class from `around_suite` before tests request the fixture:
Register the fixture class with `use_fixture` before tests request it:

```ruby
around_suite do |suite|
Expand All @@ -743,16 +748,36 @@ end
```

`use_fixture` is available inside `around_suite` or `around_test` blocks, not as
a top-level method in a test file.
a top-level method.

The stub affects existing instances and new instances of the target class until
it is reset. Method stubs are shared across Fibers and Threads, including a
Rails test server running in another thread. Tests can request the fixture to
make the side effect explicit:
Rails test server running in another thread. Tests can request a data-dependent
fixture to make the side effect explicit:

```ruby
test("checkout succeeds") do |payment_gateway_stub:|
expect(Checkout.call).to eq(:paid)
test("shows the dashboard") do |logged_in_user:|
expect(Dashboard.call.user).to eq(logged_in_user)
end
```

For intentionally broad setup that does not depend on fixture data, hooks offer
an autouse-fixture-style alternative. This keeps an environment-wide rule from
adding a stub-only keyword to every test:

```ruby
around_test do |test|
simple_stub(PushNotifier, :deliver_later) { :stubbed }
test.run
end
```

Use `around_suite` instead when one replacement should cover the whole suite:

```ruby
around_suite do |suite|
simple_stub_any_instance_of(PaymentGateway, :charge) { :approved }
suite.run
end
```

Expand All @@ -768,7 +793,12 @@ test. They do not provide isolation for multi-threaded parallel test execution:
one test can observe or reset another test's method or constant stub.

The method stub helpers call `Smartest::SimpleStub` internally, apply the stub,
register `on_teardown { stub.reset }`, and return the stub object.
register `stub.reset` with the current fixture or hook teardown scope, and
return the stub object. Hook cleanup still runs when the suite, test, or hook
fails. Nested hook, fixture, and suite fixture stubs restore the previous active
stub when the inner scope exits.
If the test or hook and its stub cleanup both fail, Smartest reports the primary
failure and the teardown failure separately.
`with_stub_const` records the previous constant value, replaces it, yields to
the block, and restores or removes the constant with `ensure`.

Expand Down Expand Up @@ -1002,7 +1032,7 @@ Smartest currently focuses on a small runner API:
- fixture dependencies through keyword arguments
- fixture teardown
- suite-scoped fixtures through `suite_fixture`
- fixture-scoped method stubs and block-scoped constant stubs
- fixture-scoped method stubs, optional hook-scoped broad setup, and block-scoped constant stubs
- suite hooks with `around_suite`
- test hooks with `around_test`
- skipped and pending tests through `skip` and `pending`
Expand Down
42 changes: 42 additions & 0 deletions SMARTEST_DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,48 @@ end
For `around_test`, those registrations are test-run local and must happen before
`test.run`.

`simple_stub_any_instance_of` and `simple_stub` are also available inside both
hook contexts:

```ruby
around_test do |test|
simple_stub(NotificationClient, :push) { :ok }
test.run
end
```

Each hook invocation owns the stubs it applies and resets them in reverse order
when the hook exits. Cleanup runs when the wrapped suite or test fails, when the
hook itself raises, and when the hook returns without calling its required run
target. Nested hook, test fixture, and suite fixture stubs use the
`Smartest::SimpleStub` stack, so removing an inner stub restores the previous
active implementation.

`Smartest::SimpleStubDSL` owns only the user-facing operations that create and
apply method stubs. Fixture instances register those stubs directly with
fixture teardown. The common `Smartest::HookContext` owns a
`Smartest::SimpleStubTeardownScope` for one hook invocation. Around-suite and
around-test contexts inherit that lifecycle and expose cleanup failures through
`teardown_errors`. If registering an already-applied stub with its lifecycle
owner fails, the DSL immediately resets that stub before propagating the
registration error. Hook and fixture lifecycle owners drain their teardown
queues before invoking callbacks, run teardown at most once, and preserve the
first call's `teardown_errors` on repeated or reentrant calls.

`Smartest::AroundSuiteHookChain` and `Smartest::AroundTestHookChain` own hook
composition, run every context's teardowns in `ensure`, and aggregate the
errors exposed by those contexts. `FixtureSet` likewise exposes errors after
running its teardowns. The runner uses `Smartest::TeardownErrorAggregator` to
pull errors from the completed fixture set and hook chain into the appropriate
test or suite result. No lifecycle operation receives a shared mutable reporter
error array. This keeps lifecycle state out of the DSL module and reporting
concerns out of hook contexts. The lifecycle owners, hook chains, and teardown
error aggregator remain internal implementation types; public RBS lists the
stub helper methods directly on fixtures and hook contexts.

Method stub helpers are intentionally not top-level DSL methods. Without a hook
or fixture owner, Smartest would have no safe lifecycle in which to reset them.

Fixture classes registered from `around_test` must not define `suite_fixture`.
Suite-scoped fixtures need suite-level cache and teardown ownership, so classes
with suite-scoped fixtures must be registered from `around_suite`.
Expand Down
2 changes: 1 addition & 1 deletion documentation/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Top-level categories should answer "what reader goal is this for?", not
| `running-test-suites.md` (Running Tests) | `autorun` and the CLI. |
| `skipping-tests.md` | `skip` / `pending`. |
| `fixtures.md` | The class-based fixture model in depth — test-scoped, suite-scoped, dependencies, teardown. |
| `stubs.md` | Method stubs that reset from fixture teardown. |
| `stubs.md` | Fixture-scoped method stubs, plus optional autouse-style hook setup, with automatic teardown. |
| `matchers.md` | Built-in and custom matchers. |
| `helpers.md` | Registering helper modules via `around_test`. |
| `playwright-browser-tests.md` (Why Smartest browser tests) | Browser Tests entry. Why-style positioning followed by Quick Start and "How it works" for the generic Playwright scaffold. |
Expand Down
4 changes: 2 additions & 2 deletions documentation/docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ end
- [Skipping Tests](./skipping-tests.md) covers skipped tests and expected failures.
- [Running Tests](./running-test-suites.md) covers autorun and the CLI.
- [Fixtures](./fixtures.md) explains class-based fixtures, dependencies, and teardown.
- [Stubs](./stubs.md) shows method stubs that reset from fixture teardown.
- [Stubs](./stubs.md) shows fixture-scoped method stubs, plus optional autouse-style hook setup.
- [Helpers](./helpers.md) explains registering helper modules from `around_test`.
- [Why Smartest browser tests](./playwright-browser-tests.md) explains why pytest-style fixtures change Ruby browser testing, and walks through the Playwright-powered scaffold.
- [Test a Rails app locally](./rails-browser-tests.md) shows the generated same-process Rails server fixture.
Expand All @@ -66,7 +66,7 @@ Smartest currently focuses on a small runner API:
- fixture dependencies through keyword arguments
- fixture teardown
- suite-scoped fixtures through `suite_fixture`
- fixture-scoped method stubs and block-scoped constant stubs
- fixture-scoped method stubs, optional hook-scoped broad setup, and block-scoped constant stubs
- suite hooks through `around_suite`
- test hooks through `around_test`
- skipped and pending tests through `skip` and `pending`
Expand Down
34 changes: 34 additions & 0 deletions documentation/docs/pytest-style-fixtures-for-ruby.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,39 @@ end
Teardown runs even if a later setup step or the test body fails. Regular fixture
teardown runs after the test. `suite_fixture` teardown runs after the suite.

## Optional Autouse-Style Setup

Smartest normally keeps method stubs in explicitly requested fixtures, just like
the fixture examples above. When Pytest would use an autouse fixture because
broad setup should apply without appearing in every test signature, Smartest
can use a hook-scoped method stub:

```python title="pytest"
@pytest.fixture(autouse=True)
def stub_payment_gateway(monkeypatch):
monkeypatch.setattr(
PaymentGateway,
"charge",
lambda self, *args, **kwargs: "approved",
)
```

```ruby title="Smartest"
around_test do |test|
simple_stub_any_instance_of(PaymentGateway, :charge) { :approved }
test.run
end
```

Neither test signature needs a stub-only dependency. Pytest undoes the
monkeypatch when the autouse fixture finishes; Smartest resets the method stub
when the hook exits. Use `around_suite` instead when one stub should cover the
whole Smartest suite.

Keep scenario-specific stubs and all stubs that need fixture data in regular
Smartest fixtures. Use this hook form only when intentionally implicit,
autouse-style setup is clearer than a fixture keyword.

## Concept Map

| Pytest concept | Smartest equivalent | Notes |
Expand All @@ -107,6 +140,7 @@ teardown runs after the test. `suite_fixture` teardown runs after the suite.
| Test function parameter | Required keyword argument | `test("name") do |user:|` requests `fixture :user`. |
| Fixture parameter | Fixture block keyword argument | `fixture :client do |server:|` depends on `server`. |
| `scope="session"` | `suite_fixture` | Created lazily and reused for the suite. |
| `autouse=True` setup with `monkeypatch` | Optional `around_test` or `around_suite` method stub | Intentionally broad setup applies without a stub-only test keyword. |
| `yield` teardown or finalizer | `on_teardown` | Register teardown immediately after acquiring the resource. |
| `conftest.py` discovery | `require` fixture files and `use_fixture` | Smartest registers fixture classes explicitly. |

Expand Down
Loading
Loading