diff --git a/CHANGELOG.md b/CHANGELOG.md index e709c03..a3ba449 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## Unreleased + +### New Features + +- Allow `simple_stub_any_instance_of` and `simple_stub` inside `around_suite` + and `around_test`, with automatic hook-scoped reset and nested stub + restoration. If the wrapped operation and stub teardown both fail, report + both errors without replacing the primary failure. + ## 0.7.0 ### Breaking Changes diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index c5080e7..01b6fa5 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -41,11 +41,15 @@ smartest/ parameter_extractor.rb execution_context.rb + hook_contexts.rb + hook_chains.rb expectations.rb expectation_target.rb matchers.rb simple_stub.rb + simple_stub_dsl.rb + teardown_error_aggregator.rb constant_stub_helpers.rb runner.rb @@ -132,9 +136,13 @@ around_suite(&block) around_test(&block) ``` -`use_fixture(klass)` and `use_matcher(matcher_module)` are not top-level DSL -methods. They are available only from hook execution contexts: `around_suite` -and `around_test`. +`use_fixture(klass)`, `use_matcher(matcher_module)`, +`simple_stub_any_instance_of`, and `simple_stub` are not top-level DSL methods. +They are available only from the relevant hook execution contexts: +`around_suite` and `around_test`. Method stub helpers are also available from +fixture blocks, where fixture teardown owns their reset. +If a lifecycle owner cannot register an applied stub for teardown, the helper +immediately resets it before propagating the registration error. Possible later methods: @@ -295,12 +303,36 @@ Responsibilities: - cache fixture values for its scope - collect teardown blocks - run teardown blocks in reverse order +- drain the teardown queue before callbacks and run it at most once +- expose teardown failures through `teardown_errors` - detect duplicate fixture names - detect circular dependencies Important: regular fixture values must not leak across tests. Suite fixture values are intentionally shared across the run. +### Hook contexts and chains + +`AroundSuiteContext` and `AroundTestContext` own the state for one hook +invocation. Their common `HookContext` resets method stubs from +`run_teardowns` at most once and exposes any cleanup failures through +`teardown_errors`. Repeated or reentrant teardown calls preserve the errors +recorded by the first call. + +`AroundSuiteHookChain` and `AroundTestHookChain` own hook composition and the +required `suite.run` / `test.run` protocol. Each chain runs its contexts' +teardowns in `ensure` and uses `TeardownErrorAggregator` to collect their +cleanup failures. The chain then acts as one teardown error source for the +runner. + +### `Smartest::TeardownErrorAggregator` + +Collects teardown failures from completed lifecycle owners. Sources expose +`teardown_errors`; the aggregator reads that state after cleanup instead of +passing a shared mutable error array into the operation that performs cleanup. +These lifecycle implementation classes are intentionally omitted from the +public DSL RBS. + ### `Smartest::ExecutionContext` Object used as `self` when running a test body. @@ -326,7 +358,7 @@ Runs tests. Responsibilities: - iterate over registered test cases -- run registered `around_suite` hooks around the suite body +- run registered hooks through their hook chain - create a lazy suite-scoped `FixtureSet` - create a fresh `ExecutionContext` per test - create a fresh `FixtureSet` per test @@ -335,6 +367,7 @@ Responsibilities: - run teardown in `ensure` - track skipped and pending test state - run suite fixture teardown after all tests +- aggregate teardown errors from fixture sets and hook chains - produce `TestResult` - notify reporter @@ -671,6 +704,7 @@ A practical approach: - snapshot file-local hooks when each test is registered - run hooks around fixture setup, test body, and fixture teardown - expose `use_fixture` and `use_matcher` only inside hook contexts +- expose method stub helpers inside hook contexts with automatic hook-scoped reset - make `around_test` registered from `around_suite` suite-wide ### Phase 9: Skipped and pending tests diff --git a/README.md b/README.md index e26e81a..2f9b2e1 100644 --- a/README.md +++ b/README.md @@ -722,18 +722,23 @@ Register teardown immediately after acquiring the resource, before later setup s ## Stubs -Use simple stub helpers when a fixture needs to temporarily replace a Ruby -method and reset it during teardown: +Put method stubs in fixtures by default so the stubbed state is visible in the +test signature: ```ruby class ApplicationTestFixture < Smartest::Fixture - fixture :payment_gateway_stub do - simple_stub_any_instance_of(PaymentGateway, :charge) { :approved } + fixture :user do + User.create!(name: "Alice") + end + + fixture :logged_in_user do |user:| + simple_stub_any_instance_of(ApplicationController, :current_user) { user } + user end end ``` -Register the fixture class from `around_suite` before tests request the fixture: +Register the fixture class with `use_fixture` before tests request it: ```ruby around_suite do |suite| @@ -743,16 +748,36 @@ end ``` `use_fixture` is available inside `around_suite` or `around_test` blocks, not as -a top-level method in a test file. +a top-level method. The stub affects existing instances and new instances of the target class until it is reset. Method stubs are shared across Fibers and Threads, including a -Rails test server running in another thread. Tests can request the fixture to -make the side effect explicit: +Rails test server running in another thread. Tests can request a data-dependent +fixture to make the side effect explicit: ```ruby -test("checkout succeeds") do |payment_gateway_stub:| - expect(Checkout.call).to eq(:paid) +test("shows the dashboard") do |logged_in_user:| + expect(Dashboard.call.user).to eq(logged_in_user) +end +``` + +For intentionally broad setup that does not depend on fixture data, hooks offer +an autouse-fixture-style alternative. This keeps an environment-wide rule from +adding a stub-only keyword to every test: + +```ruby +around_test do |test| + simple_stub(PushNotifier, :deliver_later) { :stubbed } + test.run +end +``` + +Use `around_suite` instead when one replacement should cover the whole suite: + +```ruby +around_suite do |suite| + simple_stub_any_instance_of(PaymentGateway, :charge) { :approved } + suite.run end ``` @@ -768,7 +793,12 @@ test. They do not provide isolation for multi-threaded parallel test execution: one test can observe or reset another test's method or constant stub. The method stub helpers call `Smartest::SimpleStub` internally, apply the stub, -register `on_teardown { stub.reset }`, and return the stub object. +register `stub.reset` with the current fixture or hook teardown scope, and +return the stub object. Hook cleanup still runs when the suite, test, or hook +fails. Nested hook, fixture, and suite fixture stubs restore the previous active +stub when the inner scope exits. +If the test or hook and its stub cleanup both fail, Smartest reports the primary +failure and the teardown failure separately. `with_stub_const` records the previous constant value, replaces it, yields to the block, and restores or removes the constant with `ensure`. @@ -1002,7 +1032,7 @@ Smartest currently focuses on a small runner API: - fixture dependencies through keyword arguments - fixture teardown - suite-scoped fixtures through `suite_fixture` -- fixture-scoped method stubs and block-scoped constant stubs +- fixture-scoped method stubs, optional hook-scoped broad setup, and block-scoped constant stubs - suite hooks with `around_suite` - test hooks with `around_test` - skipped and pending tests through `skip` and `pending` diff --git a/SMARTEST_DESIGN.md b/SMARTEST_DESIGN.md index ad9280a..5b4bac6 100644 --- a/SMARTEST_DESIGN.md +++ b/SMARTEST_DESIGN.md @@ -1015,6 +1015,48 @@ end For `around_test`, those registrations are test-run local and must happen before `test.run`. +`simple_stub_any_instance_of` and `simple_stub` are also available inside both +hook contexts: + +```ruby +around_test do |test| + simple_stub(NotificationClient, :push) { :ok } + test.run +end +``` + +Each hook invocation owns the stubs it applies and resets them in reverse order +when the hook exits. Cleanup runs when the wrapped suite or test fails, when the +hook itself raises, and when the hook returns without calling its required run +target. Nested hook, test fixture, and suite fixture stubs use the +`Smartest::SimpleStub` stack, so removing an inner stub restores the previous +active implementation. + +`Smartest::SimpleStubDSL` owns only the user-facing operations that create and +apply method stubs. Fixture instances register those stubs directly with +fixture teardown. The common `Smartest::HookContext` owns a +`Smartest::SimpleStubTeardownScope` for one hook invocation. Around-suite and +around-test contexts inherit that lifecycle and expose cleanup failures through +`teardown_errors`. If registering an already-applied stub with its lifecycle +owner fails, the DSL immediately resets that stub before propagating the +registration error. Hook and fixture lifecycle owners drain their teardown +queues before invoking callbacks, run teardown at most once, and preserve the +first call's `teardown_errors` on repeated or reentrant calls. + +`Smartest::AroundSuiteHookChain` and `Smartest::AroundTestHookChain` own hook +composition, run every context's teardowns in `ensure`, and aggregate the +errors exposed by those contexts. `FixtureSet` likewise exposes errors after +running its teardowns. The runner uses `Smartest::TeardownErrorAggregator` to +pull errors from the completed fixture set and hook chain into the appropriate +test or suite result. No lifecycle operation receives a shared mutable reporter +error array. This keeps lifecycle state out of the DSL module and reporting +concerns out of hook contexts. The lifecycle owners, hook chains, and teardown +error aggregator remain internal implementation types; public RBS lists the +stub helper methods directly on fixtures and hook contexts. + +Method stub helpers are intentionally not top-level DSL methods. Without a hook +or fixture owner, Smartest would have no safe lifecycle in which to reset them. + Fixture classes registered from `around_test` must not define `suite_fixture`. Suite-scoped fixtures need suite-level cache and teardown ownership, so classes with suite-scoped fixtures must be registered from `around_suite`. diff --git a/documentation/AGENTS.md b/documentation/AGENTS.md index 90bd3ea..2a21792 100644 --- a/documentation/AGENTS.md +++ b/documentation/AGENTS.md @@ -73,7 +73,7 @@ Top-level categories should answer "what reader goal is this for?", not | `running-test-suites.md` (Running Tests) | `autorun` and the CLI. | | `skipping-tests.md` | `skip` / `pending`. | | `fixtures.md` | The class-based fixture model in depth — test-scoped, suite-scoped, dependencies, teardown. | -| `stubs.md` | Method stubs that reset from fixture teardown. | +| `stubs.md` | Fixture-scoped method stubs, plus optional autouse-style hook setup, with automatic teardown. | | `matchers.md` | Built-in and custom matchers. | | `helpers.md` | Registering helper modules via `around_test`. | | `playwright-browser-tests.md` (Why Smartest browser tests) | Browser Tests entry. Why-style positioning followed by Quick Start and "How it works" for the generic Playwright scaffold. | diff --git a/documentation/docs/intro.md b/documentation/docs/intro.md index 2c9ffb9..fc175f0 100644 --- a/documentation/docs/intro.md +++ b/documentation/docs/intro.md @@ -51,7 +51,7 @@ end - [Skipping Tests](./skipping-tests.md) covers skipped tests and expected failures. - [Running Tests](./running-test-suites.md) covers autorun and the CLI. - [Fixtures](./fixtures.md) explains class-based fixtures, dependencies, and teardown. -- [Stubs](./stubs.md) shows method stubs that reset from fixture teardown. +- [Stubs](./stubs.md) shows fixture-scoped method stubs, plus optional autouse-style hook setup. - [Helpers](./helpers.md) explains registering helper modules from `around_test`. - [Why Smartest browser tests](./playwright-browser-tests.md) explains why pytest-style fixtures change Ruby browser testing, and walks through the Playwright-powered scaffold. - [Test a Rails app locally](./rails-browser-tests.md) shows the generated same-process Rails server fixture. @@ -66,7 +66,7 @@ Smartest currently focuses on a small runner API: - fixture dependencies through keyword arguments - fixture teardown - suite-scoped fixtures through `suite_fixture` -- fixture-scoped method stubs and block-scoped constant stubs +- fixture-scoped method stubs, optional hook-scoped broad setup, and block-scoped constant stubs - suite hooks through `around_suite` - test hooks through `around_test` - skipped and pending tests through `skip` and `pending` diff --git a/documentation/docs/pytest-style-fixtures-for-ruby.md b/documentation/docs/pytest-style-fixtures-for-ruby.md index c96e23d..e9be214 100644 --- a/documentation/docs/pytest-style-fixtures-for-ruby.md +++ b/documentation/docs/pytest-style-fixtures-for-ruby.md @@ -99,6 +99,39 @@ end Teardown runs even if a later setup step or the test body fails. Regular fixture teardown runs after the test. `suite_fixture` teardown runs after the suite. +## Optional Autouse-Style Setup + +Smartest normally keeps method stubs in explicitly requested fixtures, just like +the fixture examples above. When Pytest would use an autouse fixture because +broad setup should apply without appearing in every test signature, Smartest +can use a hook-scoped method stub: + +```python title="pytest" +@pytest.fixture(autouse=True) +def stub_payment_gateway(monkeypatch): + monkeypatch.setattr( + PaymentGateway, + "charge", + lambda self, *args, **kwargs: "approved", + ) +``` + +```ruby title="Smartest" +around_test do |test| + simple_stub_any_instance_of(PaymentGateway, :charge) { :approved } + test.run +end +``` + +Neither test signature needs a stub-only dependency. Pytest undoes the +monkeypatch when the autouse fixture finishes; Smartest resets the method stub +when the hook exits. Use `around_suite` instead when one stub should cover the +whole Smartest suite. + +Keep scenario-specific stubs and all stubs that need fixture data in regular +Smartest fixtures. Use this hook form only when intentionally implicit, +autouse-style setup is clearer than a fixture keyword. + ## Concept Map | Pytest concept | Smartest equivalent | Notes | @@ -107,6 +140,7 @@ teardown runs after the test. `suite_fixture` teardown runs after the suite. | Test function parameter | Required keyword argument | `test("name") do |user:|` requests `fixture :user`. | | Fixture parameter | Fixture block keyword argument | `fixture :client do |server:|` depends on `server`. | | `scope="session"` | `suite_fixture` | Created lazily and reused for the suite. | +| `autouse=True` setup with `monkeypatch` | Optional `around_test` or `around_suite` method stub | Intentionally broad setup applies without a stub-only test keyword. | | `yield` teardown or finalizer | `on_teardown` | Register teardown immediately after acquiring the resource. | | `conftest.py` discovery | `require` fixture files and `use_fixture` | Smartest registers fixture classes explicitly. | diff --git a/documentation/docs/rails-browser-tests.md b/documentation/docs/rails-browser-tests.md index 86cc261..f5dfaa0 100644 --- a/documentation/docs/rails-browser-tests.md +++ b/documentation/docs/rails-browser-tests.md @@ -226,7 +226,8 @@ several of those into a named test state. | --- | --- | | `let(:user) { create(:user) }` | `fixture :user do create(:user) end` | | `before { login_as(user) }` | `fixture :signed_in_page do |page:, user:| ...; page end` | -| `before { allow(...).to receive(...) }` | `fixture :stubbed_page do |page:| simple_stub(...); page end` | +| stub depending on `let` data | `fixture :stubbed_page do |page:, user:| simple_stub(...); page end` | +| autouse-style broad `before { allow(...).to receive(...) }` | `around_test do |test| simple_stub(...); test.run end` | | `scenario "..." do ... end` | `test("...") do |page:| ... end` | | hidden setup | explicit keyword fixture dependencies | @@ -252,7 +253,10 @@ Rails browser tests are often easier to read when user state, stubs, and the browser page are combined into a named page fixture. `simple_stub` and `simple_stub_any_instance_of` are Smartest stub helpers. -Stubs installed inside fixtures are automatically reset during fixture teardown. +Put them in fixtures by default so the stubbed state stays visible in the test +signature. For intentionally broad setup that should not add a stub-only +dependency to every test, they can also be used from `around_test` or +`around_suite`, like an autouse fixture. Both forms reset automatically. ```ruby class ApplicationFixture < Smartest::Fixture @@ -279,12 +283,16 @@ class ApplicationFixture < Smartest::Fixture page end +end +``` - fixture :page_with_push_stubbed do |page:| - simple_stub(PushNotifier, :deliver_later) { :stubbed } +If push delivery must be disabled for every applicable test, it is an example +of intentionally broad setup that can instead live in an `around_test` hook: - page - end +```ruby +around_test do |test| + simple_stub(PushNotifier, :deliver_later) { :stubbed } + test.run end ``` @@ -310,10 +318,10 @@ test("suspended user sees account restriction page") do |suspended_user_page:| ).to be_visible end -test("push failure banner is not shown when push is stubbed") do |page_with_push_stubbed:| - page_with_push_stubbed.goto("/settings/notifications") +test("push failure banner is not shown when push is stubbed") do |page:| + page.goto("/settings/notifications") - expect(page_with_push_stubbed.get_by_text("Push failed")).not_to be_visible + expect(page.get_by_text("Push failed")).not_to be_visible end ``` @@ -508,8 +516,8 @@ Rails browser tests often need to stub methods that are called inside the Rails server thread. Because the generated Rails server runs in the same Ruby process as the test -runner, method stubs installed by Smartest fixtures are visible to the Rails -server thread. +runner, method stubs installed by Smartest fixtures or hooks are visible to the +Rails server thread. ```ruby fixture :suspended_user_page do |page:, suspended_user:| @@ -521,7 +529,12 @@ fixture :suspended_user_page do |page:, suspended_user:| end ``` -The stub is applied during fixture setup and reset from fixture teardown. +This stub depends on `suspended_user`, so fixture scope keeps the relationship +explicit. Keep fixtures as the default even when a stub has no fixture +dependencies. For an intentionally broad external-service rule that should be +implicit for every applicable test, use `simple_stub` or +`simple_stub_any_instance_of` before `test.run` in `around_test`; Smartest +resets it when the hook exits. :::warning diff --git a/documentation/docs/smartest-vs-minitest.md b/documentation/docs/smartest-vs-minitest.md index 89e8e28..67513fc 100644 --- a/documentation/docs/smartest-vs-minitest.md +++ b/documentation/docs/smartest-vs-minitest.md @@ -111,8 +111,8 @@ class CheckoutTest < Minitest::Test end ``` -In Smartest, method stubs usually belong in fixtures so tests can request the -stubbed dependency by keyword: +Smartest keeps the stub in a fixture by default, so the test signature shows +that fixed time is part of its setup: ```ruby title="Smartest" class TimeFixture < Smartest::Fixture @@ -133,6 +133,9 @@ test("uses fixed time") do |fixed_now:| end ``` +Smartest resets the method stub when the fixture is torn down, including when +the test fails. + If a Minitest suite uses RSpec mocks or Mocha-style any-instance stubs, move that setup into a fixture with `simple_stub_any_instance_of`: @@ -153,6 +156,21 @@ test("uses the current user") do |logged_in_as_user1:| end ``` +If a replacement is an intentional rule for every applicable test and does not +depend on fixture data, `around_test` is available as an autouse-fixture-style +alternative: + +```ruby title="Smartest" +around_test do |test| + simple_stub(PushNotifier, :deliver_later) { :stubbed } + test.run +end +``` + +The tests do not need a stub-only keyword. Smartest resets the method stub when +the hook exits; use `around_suite` when one replacement should cover the whole +suite. + For constants, use `with_stub_const` as a block-scoped helper in the test body, `around_test`, or `around_suite`. Smartest stubs are process-wide state, so they do not provide isolation for multi-threaded parallel test execution: diff --git a/documentation/docs/smartest-vs-rspec.md b/documentation/docs/smartest-vs-rspec.md index 29c3745..332c4e3 100644 --- a/documentation/docs/smartest-vs-rspec.md +++ b/documentation/docs/smartest-vs-rspec.md @@ -106,8 +106,8 @@ RSpec.describe Checkout do end ``` -In Smartest, put the stub in a fixture and request that fixture from tests that -depend on it: +Smartest keeps the stub in a fixture by default, so the test signature shows +that fixed time is part of its setup: ```ruby title="Smartest" class TimeFixture < Smartest::Fixture @@ -128,10 +128,9 @@ test("uses the fixed time") do |fixed_now:| end ``` -The fixture signature makes the stubbed dependency explicit, and Smartest resets -the method stub from fixture teardown. - -For `allow_any_instance_of`, use `simple_stub_any_instance_of`: +Smartest resets the stub when the fixture is torn down, including when the test +fails. For `allow_any_instance_of`, use `simple_stub_any_instance_of` in the +same fixture-first style: ```ruby title="RSpec" before do @@ -158,6 +157,21 @@ test("uses the current user") do |logged_in_as_user1:| end ``` +If a replacement is an intentional rule for every applicable test and does not +depend on fixture data, `around_test` is available as an autouse-fixture-style +alternative: + +```ruby title="Smartest" +around_test do |test| + simple_stub(PushNotifier, :deliver_later) { :stubbed } + test.run +end +``` + +The tests do not need a stub-only keyword. Smartest resets the method stub when +the hook exits; use `around_suite` when one replacement should cover the whole +suite. + Constant stubs are different from method stubs. Use `with_stub_const` with a block in a test body, `around_test`, or `around_suite`; it is intentionally not a fixture helper because Ruby constants are process-global: diff --git a/documentation/docs/stubs.md b/documentation/docs/stubs.md index 76415bc..654ac23 100644 --- a/documentation/docs/stubs.md +++ b/documentation/docs/stubs.md @@ -1,36 +1,19 @@ --- title: Stubs -description: Stub Ruby methods from Smartest fixtures with automatic teardown. +description: Stub Ruby methods from Smartest fixtures, with optional autouse-style hook setup and automatic teardown. --- # Stubs Smartest provides small stub helpers for replacing Ruby methods during a test. -They are useful when a test depends on stubbed behavior and you want that -dependency to be visible in the test signature: +Put method stubs in fixtures by default. The fixture keyword makes the test +state visible in the test signature, and Smartest resets the stub +automatically when the fixture is torn down. -```ruby -test("shows the suspended account state") do |suspended_user_logged_in:| - expect(AccountStatus.call).to eq(:suspended) -end -``` +## Fixture-Scoped Method Stubs -The fixture name makes it clear that this test depends on an authenticated -suspended user. - -If you are familiar with RSpec, this is similar to putting a stub in `before`: - -```ruby -let(:suspended_user) { create(:user, :suspended) } - -before do - allow_any_instance_of(ApplicationController) - .to receive(:current_user) - .and_return(suspended_user) -end -``` - -In Smartest, the same idea is expressed as a fixture: +Define the stub in a fixture and request that fixture from each test that needs +the replacement: ```ruby class EdgeCaseFixture < Smartest::Fixture @@ -54,6 +37,17 @@ around_suite do |suite| end ``` +Then request the fixture from the test: + +```ruby +test("shows the suspended account state") do |suspended_user_logged_in:| + expect(AccountStatus.call).to eq(:suspended) +end +``` + +The fixture name makes it clear that this test depends on an authenticated +suspended user. + `use_fixture` is available inside `around_suite` or `around_test` blocks, not as a top-level method in a test file. @@ -61,6 +55,19 @@ The stub is automatically reset when the fixture is torn down. Because the stub fixture depends on `suspended_user:`, the user record and authenticated state stay tied together in one fixture dependency graph. +If you are familiar with RSpec, the fixture replaces setup that might otherwise +live in a `before` hook: + +```ruby +let(:suspended_user) { create(:user, :suspended) } + +before do + allow_any_instance_of(ApplicationController) + .to receive(:current_user) + .and_return(suspended_user) +end +``` + ## Instance Method Stubs Use `simple_stub_any_instance_of` for instance methods: @@ -74,17 +81,6 @@ teardown resets it. Method stubs are shared across Fibers and Threads, so a stub applied by test setup is also visible to a Rails test server running in another thread. -You can also use method stubs for external services that should not run during -tests, such as push notifications or payment processing: - -```ruby -class ApplicationTestFixture < Smartest::Fixture - fixture :payment_gateway_stub do - simple_stub_any_instance_of(PaymentGateway, :charge) { :approved } - end -end -``` - ## Class Method Stubs Use `simple_stub` for singleton methods, including class methods: @@ -97,6 +93,11 @@ class TimeFixture < Smartest::Fixture frozen_time end end + +around_suite do |suite| + use_fixture TimeFixture + suite.run +end ``` ```ruby @@ -105,6 +106,42 @@ test("uses fixed time") do |fixed_time:| end ``` +## Optional Autouse-Style Setup From Hooks + +Most method stubs should remain in fixtures. When a replacement is an +intentional rule for every applicable test and does not depend on fixture data, +you can put it in a hook instead. This is similar to a Pytest autouse fixture: +the setup applies without adding a fixture keyword to every test signature. + +Use `around_test` when the replacement should be applied and reset for each +applicable test: + +```ruby +around_test do |test| + simple_stub(NotificationClient, :push) { :ok } + test.run +end +``` + +Use `around_suite` when an external service should stay stubbed for the whole +suite: + +```ruby +around_suite do |suite| + simple_stub_any_instance_of(PaymentGateway, :charge) { :approved } + suite.run +end +``` + +Call the stub helper before `suite.run` or `test.run`. Smartest resets the stub +when that hook invocation exits, including when the suite, test, or hook fails. +An `around_test` written directly in a test file applies to later tests in that +file. Register it inside `around_suite` when it should apply suite-wide. + +Use this style for broad test-environment rules such as "never contact the +payment gateway." Keep scenario-specific or data-dependent stubs in fixtures so +their side effects remain visible at the test boundary. + ## Constant Stubs Use `with_stub_const` with a block for constants. It is available in test @@ -145,7 +182,7 @@ Constant stubs are process-global until the block exits. ## How Method Stub Teardown Works You do not need to call `reset` manually when using method stub helpers inside -fixtures. The helper internally: +fixtures or hooks. The helper internally: 1. creates the stub state 2. applies the replacement @@ -157,26 +194,35 @@ Conceptually, this: simple_stub_any_instance_of(ApplicationController, :current_user) { user } ``` -behaves like: +behaves like applying a `Smartest::SimpleStub` and registering its `reset` with +the current fixture or hook teardown owner: ```ruby stub = Smartest::SimpleStub.new(ApplicationController, :current_user) { user } stub.apply -on_teardown { stub.reset } +# The current fixture or hook scope registers: stub.reset ``` -Teardown is tied to the fixture lifecycle: +Teardown is tied to the scope that applies the stub: - `fixture` method stubs reset after each test. - `suite_fixture` method stubs reset after the suite fixture scope ends. +- `around_test` method stubs reset when that hook invocation exits. +- `around_suite` method stubs reset when that hook invocation exits. + +Hook cleanup runs even when a test fails, is skipped from the hook, or a hook +fails to call its required run target. +If a test or hook and its method-stub cleanup both fail, Smartest preserves the +primary failure and reports the reset error separately as a teardown failure. When method stubs for the same class and method overlap, the most recently applied stub wins. Resetting that stub restores the previous stub instead of -resetting the whole stack. This lets a test-scoped stub temporarily override a -suite-scoped stub. +resetting the whole stack. Hook, fixture, and suite fixture stubs can therefore +nest without discarding an outer replacement. -In most cases, prefer the fixture helpers so stub lifetime is automatically tied -to the fixture lifecycle. +Prefer fixture-scoped stubs so the stubbed state stays explicit in a test's +keyword dependencies. Use hook-scoped stubs only for intentionally broad, +autouse-style setup that does not need fixture data. ## How Constant Stub Blocks Work @@ -238,8 +284,9 @@ end `Smartest::SimpleStub` object. `with_stub_const` returns the block result. `simple_stub_any_instance_of` and `simple_stub` are available inside -`Smartest::Fixture` fixture blocks, including `fixture` and `suite_fixture`, -because they need `on_teardown` to keep the stub lifetime tied to the fixture scope. +`Smartest::Fixture` fixture blocks, including `fixture` and `suite_fixture`, as +well as `around_test` and `around_suite`. They are not top-level DSL methods and +are not available in test bodies because method stubs require a teardown owner. `with_stub_const` is available in test bodies, `around_test`, and `around_suite`. diff --git a/documentation/static/llms.txt b/documentation/static/llms.txt index 77d20c6..94b6e0e 100644 --- a/documentation/static/llms.txt +++ b/documentation/static/llms.txt @@ -35,8 +35,11 @@ documentation, prefer the project README and the public docs linked below. suite. - `simple_stub_any_instance_of(Klass, :method) { ... }` and `simple_stub(object, :method) { ... }` are fixture helpers that apply a - process-wide method stub and register teardown automatically. Method stubs - are shared across Fibers and Threads until reset. + process-wide method stub and register scope-aware teardown automatically. + Prefer fixture scope so the stub is visible in the test signature. Hook + contexts also provide the helpers for optional autouse-style broad setup that + does not depend on fixture data. Method stubs are shared across Fibers and + Threads until reset. - `with_stub_const("Namespace::CONST", value) { ... }` is a block-scoped helper available in test bodies, `around_test`, and `around_suite`. It replaces a constant and restores or removes it with ensure. Constant stubs are diff --git a/lib/smartest.rb b/lib/smartest.rb index 87bdb2f..c4510a9 100644 --- a/lib/smartest.rb +++ b/lib/smartest.rb @@ -3,6 +3,8 @@ require_relative "smartest/version" require_relative "smartest/errors" require_relative "smartest/simple_stub" +require_relative "smartest/simple_stub_dsl" +require_relative "smartest/teardown_error_aggregator" require_relative "smartest/constant_stub_helpers" require_relative "smartest/parameter_extractor" require_relative "smartest/test_case" @@ -18,6 +20,7 @@ require_relative "smartest/test_run_state" require_relative "smartest/test_run" require_relative "smartest/hook_contexts" +require_relative "smartest/hook_chains" require_relative "smartest/expectations" require_relative "smartest/expectation_target" require_relative "smartest/matchers" diff --git a/lib/smartest/fixture.rb b/lib/smartest/fixture.rb index d2af4bb..067d22b 100644 --- a/lib/smartest/fixture.rb +++ b/lib/smartest/fixture.rb @@ -2,6 +2,8 @@ module Smartest class Fixture + include SimpleStubDSL + RESERVED_CONTEXT_METHODS = %i[skip pending with_stub_const].freeze class << self @@ -65,18 +67,8 @@ def on_teardown(&block) @fixture_set.add_teardown(&block) end - def simple_stub_any_instance_of(klass, method_name, &block) - apply_simple_stub(SimpleStub.new(klass, method_name, &block)) - end - - def simple_stub(object, method_name, &block) - apply_simple_stub(SimpleStub.new(object.singleton_class, method_name, &block)) - end - - def apply_simple_stub(stub) - stub.apply + def register_simple_stub(stub) on_teardown { stub.reset } - stub end def method_missing(method_name, *args, &block) diff --git a/lib/smartest/fixture_set.rb b/lib/smartest/fixture_set.rb index 64d4253..80a4eb4 100644 --- a/lib/smartest/fixture_set.rb +++ b/lib/smartest/fixture_set.rb @@ -2,6 +2,8 @@ module Smartest class FixtureSet + attr_reader :teardown_errors + def initialize(fixture_classes, context:, scope: :test, parent: nil) @fixture_classes = fixture_classes.to_a @context = context @@ -10,6 +12,8 @@ def initialize(fixture_classes, context:, scope: :test, parent: nil) @cache = {} @setup_errors = {} @teardowns = [] + @teardown_errors = [] + @teardowns_started = false @resolving = [] build_fixture_index @@ -56,17 +60,21 @@ def add_teardown(&block) end def run_teardowns - errors = [] + return self if @teardowns_started + + @teardowns_started = true + teardowns = @teardowns + @teardowns = [] - @teardowns.reverse_each do |teardown| + teardowns.reverse_each do |teardown| teardown.call rescue Exception => error raise if Smartest.fatal_exception?(error) - errors << error + @teardown_errors << error end - errors + self end private diff --git a/lib/smartest/hook_chains.rb b/lib/smartest/hook_chains.rb new file mode 100644 index 0000000..0b7fd64 --- /dev/null +++ b/lib/smartest/hook_chains.rb @@ -0,0 +1,81 @@ +# frozen_string_literal: true + +module Smartest + class AroundSuiteHookChain + attr_reader :teardown_errors + + def initialize(suite:, hooks:) + @suite = suite + @hooks = hooks + @teardown_error_aggregator = TeardownErrorAggregator.new + @teardown_errors = @teardown_error_aggregator.teardown_errors + end + + def run(&suite_body) + run_hook(0, &suite_body) + end + + private + + def run_hook(index, &suite_body) + return yield if index >= @hooks.length + + hook = @hooks[index] + suite_run = SuiteRun.new { run_hook(index + 1, &suite_body) } + context = AroundSuiteContext.new(@suite) + + begin + context.call(hook, suite_run) + ensure + context.run_teardowns + @teardown_error_aggregator.collect(context) + end + + raise AroundSuiteRunError, "around_suite hook did not call suite.run" unless suite_run.ran? + + suite_run.result + end + end + + class AroundTestHookChain + attr_reader :teardown_errors + + def initialize(hooks:, test_run:, run_state:) + @hooks = hooks + @test_run = test_run + @run_state = run_state + @teardown_error_aggregator = TeardownErrorAggregator.new + @teardown_errors = @teardown_error_aggregator.teardown_errors + end + + def run + run_hook(0) + end + + private + + def run_hook(index) + return @test_run.run if index >= @hooks.length + + hook = @hooks[index] + next_run = TestRun.new( + fixture_classes: [], + matcher_modules: [] + ) do |**_keywords| + run_hook(index + 1) + end + context = AroundTestContext.new(@test_run, run_state: @run_state) + + begin + context.call(hook, next_run) + ensure + context.run_teardowns + @teardown_error_aggregator.collect(context) + end + + raise AroundTestRunError, "around_test hook did not call test.run" unless next_run.ran? + + next_run.result + end + end +end diff --git a/lib/smartest/hook_contexts.rb b/lib/smartest/hook_contexts.rb index 8d22b9a..d4e342b 100644 --- a/lib/smartest/hook_contexts.rb +++ b/lib/smartest/hook_contexts.rb @@ -1,10 +1,34 @@ # frozen_string_literal: true module Smartest - class AroundSuiteContext + class HookContext + include SimpleStubDSL + + attr_reader :teardown_errors + + def initialize + @simple_stub_teardown_scope = SimpleStubTeardownScope.new + @teardown_errors = [] + end + + def run_teardowns + @simple_stub_teardown_scope.reset_registered_stubs + @teardown_errors = @simple_stub_teardown_scope.teardown_errors.dup + self + end + + private + + def register_simple_stub(stub) + @simple_stub_teardown_scope.register(stub) + end + end + + class AroundSuiteContext < HookContext include ConstantStubHelpers def initialize(suite) + super() @suite = suite end @@ -31,10 +55,11 @@ def around_test(&block) end end - class AroundTestContext + class AroundTestContext < HookContext include ConstantStubHelpers def initialize(test_run, run_state:) + super() @test_run = test_run @run_state = run_state end diff --git a/lib/smartest/runner.rb b/lib/smartest/runner.rb index d15911c..e6a578d 100644 --- a/lib/smartest/runner.rb +++ b/lib/smartest/runner.rb @@ -10,22 +10,29 @@ def initialize(suite: Smartest.suite, reporter: Reporter.new, tests: nil) def run results = [] - suite_teardown_errors = [] suite_errors = [] @suite_fixture_set = nil + hook_chain = AroundSuiteHookChain.new( + suite: @suite, + hooks: @suite.around_suite_hooks.dup + ) @reporter.start(@tests.count) begin - run_around_suite_hooks(@suite.around_suite_hooks.dup) do - run_tests(results, suite_teardown_errors) - end + hook_chain.run { run_tests(results) } rescue Exception => error raise if Smartest.fatal_exception?(error) suite_errors << error end + teardown_error_aggregator = TeardownErrorAggregator.new + teardown_error_aggregator.collect(@suite_fixture_set) if @suite_fixture_set + teardown_error_aggregator.collect(hook_chain) + suite_teardown_errors = teardown_error_aggregator.teardown_errors + @suite_fixture_set = nil + @reporter.finish( results, suite_teardown_errors: suite_teardown_errors, @@ -37,7 +44,7 @@ def run private - def run_tests(results, suite_teardown_errors) + def run_tests(results) begin @tests.each do |test_case| result = run_one(test_case) @@ -45,40 +52,32 @@ def run_tests(results, suite_teardown_errors) @reporter.record(result) end ensure - suite_teardown_errors.concat(@suite_fixture_set.run_teardowns) if @suite_fixture_set - @suite_fixture_set = nil + @suite_fixture_set.run_teardowns if @suite_fixture_set end end - def run_around_suite_hooks(hooks, index = 0, &block) - return yield if index >= hooks.length - - hook = hooks[index] - suite_run = SuiteRun.new do - run_around_suite_hooks(hooks, index + 1, &block) - end - - AroundSuiteContext.new(@suite).call(hook, suite_run) - raise AroundSuiteRunError, "around_suite hook did not call suite.run" unless suite_run.ran? - - suite_run.result - end - def run_one(test_case) started_at = now error = nil skipped = nil - teardown_errors = [] run_state = TestRunState.new + fixture_set = nil test_run = TestRun.new( fixture_classes: @suite.fixture_classes, matcher_modules: @suite.matcher_modules ) do |fixture_classes:, matcher_modules:, helper_modules:| - run_test_body(test_case, fixture_classes, matcher_modules, helper_modules, run_state, teardown_errors) + context = build_context(matcher_modules, run_state, helper_modules) + fixture_set = FixtureSet.new(fixture_classes, context: context, parent: suite_fixture_set) + run_test_body(test_case, fixture_set, context) end + hook_chain = AroundTestHookChain.new( + hooks: @suite.around_test_hooks + test_case.around_test_hooks, + test_run: test_run, + run_state: run_state + ) begin - run_around_test_hooks(@suite.around_test_hooks + test_case.around_test_hooks, test_run, run_state) + hook_chain.run rescue Skipped => skipped_error skipped = skipped_error rescue Exception => rescued_error @@ -87,6 +86,10 @@ def run_one(test_case) error = rescued_error end + teardown_error_aggregator = TeardownErrorAggregator.new + teardown_error_aggregator.collect(fixture_set) if fixture_set + teardown_error_aggregator.collect(hook_chain) + teardown_errors = teardown_error_aggregator.teardown_errors duration = now - started_at return TestResult.failed(test_case: test_case, error: nil, duration: duration, teardown_errors: teardown_errors) if skipped && teardown_errors.any? @@ -114,36 +117,15 @@ def run_one(test_case) end end - def run_test_body(test_case, fixture_classes, matcher_modules, helper_modules, run_state, teardown_errors) - context = build_context(matcher_modules, run_state, helper_modules) - fixture_set = nil - + def run_test_body(test_case, fixture_set, context) begin - fixture_set = FixtureSet.new(fixture_classes, context: context, parent: suite_fixture_set) fixtures = fixture_set.resolve_keywords(test_case.fixture_names) context.instance_exec(**fixtures, &test_case.block) ensure - teardown_errors.concat(fixture_set.run_teardowns) if fixture_set + fixture_set.run_teardowns end end - def run_around_test_hooks(hooks, test_run, run_state, index = 0) - return test_run.run if index >= hooks.length - - hook = hooks[index] - next_run = TestRun.new( - fixture_classes: [], - matcher_modules: [] - ) do |**_keywords| - run_around_test_hooks(hooks, test_run, run_state, index + 1) - end - - AroundTestContext.new(test_run, run_state: run_state).call(hook, next_run) - raise AroundTestRunError, "around_test hook did not call test.run" unless next_run.ran? - - next_run.result - end - def suite_fixture_set @suite_fixture_set ||= FixtureSet.new( @suite.fixture_classes, diff --git a/lib/smartest/simple_stub_dsl.rb b/lib/smartest/simple_stub_dsl.rb new file mode 100644 index 0000000..67f3f5f --- /dev/null +++ b/lib/smartest/simple_stub_dsl.rb @@ -0,0 +1,71 @@ +# frozen_string_literal: true + +module Smartest + module SimpleStubDSL + private + + def simple_stub_any_instance_of(klass, method_name, &block) + apply_simple_stub(SimpleStub.new(klass, method_name, &block)) + end + + def simple_stub(object, method_name, &block) + apply_simple_stub(SimpleStub.new(object.singleton_class, method_name, &block)) + end + + def apply_simple_stub(stub) + stub.apply + + begin + register_simple_stub(stub) + rescue Exception + rollback_simple_stub(stub) + raise + end + + stub + end + + def rollback_simple_stub(stub) + stub.reset + rescue Exception => error + raise if Smartest.fatal_exception?(error) + end + + def register_simple_stub(_stub) + raise NotImplementedError, "#{self.class} must implement #register_simple_stub" + end + end + + class SimpleStubTeardownScope + attr_reader :teardown_errors + + def initialize + @registered_stubs = [] + @teardown_errors = [] + @teardowns_started = false + end + + def register(stub) + @registered_stubs << stub + stub + end + + def reset_registered_stubs + return self if @teardowns_started + + @teardowns_started = true + stubs = @registered_stubs + @registered_stubs = [] + + stubs.reverse_each do |stub| + stub.reset + rescue Exception => error + raise if Smartest.fatal_exception?(error) + + @teardown_errors << error + end + + self + end + end +end diff --git a/lib/smartest/teardown_error_aggregator.rb b/lib/smartest/teardown_error_aggregator.rb new file mode 100644 index 0000000..88c1ae2 --- /dev/null +++ b/lib/smartest/teardown_error_aggregator.rb @@ -0,0 +1,16 @@ +# frozen_string_literal: true + +module Smartest + class TeardownErrorAggregator + attr_reader :teardown_errors + + def initialize + @teardown_errors = [] + end + + def collect(source) + @teardown_errors.concat(source.teardown_errors) + self + end + end +end diff --git a/sig/smartest.rbs b/sig/smartest.rbs index 4c8a14e..3704a1f 100644 --- a/sig/smartest.rbs +++ b/sig/smartest.rbs @@ -65,8 +65,8 @@ module Smartest private def on_teardown: () { () -> untyped } -> untyped - def simple_stub_any_instance_of: (Class | Module, String | Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub - def simple_stub: (untyped, String | Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub + def simple_stub_any_instance_of: (Class, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub + def simple_stub: (untyped, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub end class AroundSuiteContext @@ -75,6 +75,8 @@ module Smartest def use_fixture: (Class | Module) -> untyped def use_matcher: (Module) -> untyped def around_test: () { (TestRun) -> untyped } -> void + def simple_stub_any_instance_of: (Class, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub + def simple_stub: (untyped, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub def with_stub_const: (String | Symbol, untyped) { () -> untyped } -> untyped end @@ -86,6 +88,8 @@ module Smartest def use_helper: (Module) -> untyped def skip: (?String) -> void def pending: (?String) -> void + def simple_stub_any_instance_of: (Class, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub + def simple_stub: (untyped, Symbol) { (*untyped, **untyped) -> untyped } -> SimpleStub def with_stub_const: (String | Symbol, untyped) { () -> untyped } -> untyped end diff --git a/smartest/simple_stub_test.rb b/smartest/simple_stub_test.rb index 2355552..f62ed71 100644 --- a/smartest/simple_stub_test.rb +++ b/smartest/simple_stub_test.rb @@ -106,6 +106,130 @@ module SimpleStubSelfTestConfig expect(SimpleStubSelfTestSubject.new("Alice").greeting("Hi")).to eq("Hi, Alice") end +test("simple stub teardown scope resets registered stubs in reverse order and collects errors") do + resettable_stub_class = Class.new do + def initialize(name, events, error: nil) + @name = name + @events = events + @error = error + end + + def reset + @events << @name + raise @error if @error + end + end + + events = [] + reset_error = RuntimeError.new("inner reset failed") + outer_stub = resettable_stub_class.new(:outer, events) + inner_stub = resettable_stub_class.new(:inner, events, error: reset_error) + scope = Smartest::SimpleStubTeardownScope.new + + expect(scope.register(outer_stub)).to eq(outer_stub) + expect(scope.register(inner_stub)).to eq(inner_stub) + + expect(scope.reset_registered_stubs).to eq(scope) + expect(scope.teardown_errors).to eq([reset_error]) + expect(events).to eq(%i[inner outer]) + expect(scope.reset_registered_stubs).to eq(scope) + expect(scope.teardown_errors).to eq([reset_error]) + expect(events).to eq(%i[inner outer]) +end + +test("hook context teardown is idempotent during reentry") do + events = [] + reset_error = RuntimeError.new("reset failed") + context = Smartest::HookContext.new + reentrant_stub = Object.new + reentrant_stub.define_singleton_method(:reset) do + events << :reset_started + context.run_teardowns + events << :reset_finished + raise reset_error + end + + context.__send__(:register_simple_stub, reentrant_stub) + + expect(context.run_teardowns).to eq(context) + expect(context.teardown_errors).to eq([reset_error]) + expect(events).to eq(%i[reset_started reset_finished]) + expect(context.run_teardowns).to eq(context) + expect(context.teardown_errors).to eq([reset_error]) + expect(events).to eq(%i[reset_started reset_finished]) +end + +test("teardown error aggregator reads and combines errors from teardown sources") do + teardown_source_class = Class.new do + attr_reader :teardown_errors + + def initialize(teardown_errors) + @teardown_errors = teardown_errors + end + end + + fixture_error = RuntimeError.new("fixture teardown failed") + hook_error = RuntimeError.new("hook teardown failed") + fixture_source = teardown_source_class.new([fixture_error]) + hook_source = teardown_source_class.new([hook_error]) + aggregator = Smartest::TeardownErrorAggregator.new + + expect(aggregator.collect(fixture_source)).to eq(aggregator) + expect(aggregator.collect(hook_source)).to eq(aggregator) + expect(aggregator.teardown_errors).to eq([fixture_error, hook_error]) + expect(fixture_source.teardown_errors).to eq([fixture_error]) + expect(hook_source.teardown_errors).to eq([hook_error]) +end + +test("simple stub rolls back when lifecycle registration fails") do + observed_values = [] + registration_error = RuntimeError.new("registration failed") + dsl_host_class = Class.new do + include Smartest::SimpleStubDSL + + define_method(:initialize) do |values, error| + @values = values + @error = error + end + + def apply_clock_stub + simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + end + + private + + def register_simple_stub(_stub) + @values << SimpleStubSelfTestClock.now + raise @error + end + end + + error = SimpleStubSelfTest.capture_error(RuntimeError) do + dsl_host_class.new(observed_values, registration_error).apply_clock_stub + end + + expect(error).to eq(registration_error) + expect(observed_values).to eq([:stubbed_now]) + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("simple stub rolls back when SimpleStubDSL has no lifecycle owner") do + dsl_host_class = Class.new do + include Smartest::SimpleStubDSL + + def apply_clock_stub + simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + end + end + + error = SimpleStubSelfTest.capture_error(NotImplementedError) do + dsl_host_class.new.apply_clock_stub + end + + expect(error.message).to include("must implement #register_simple_stub") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + test("simple_stub_any_instance_of applies and resets from fixture teardown") do fixture_class = Class.new(Smartest::Fixture) do fixture :stubbed_name do @@ -201,6 +325,435 @@ module SimpleStubSelfTestConfig expect(status).to eq(0) end +test("around_test method stub helpers apply only for each hook scope") do + events = [] + suite = Smartest::Suite.new + + suite.around_test_hooks << proc do |test_run| + events << [:before_hook, SimpleStubSelfTestSubject.new("Alice").name, SimpleStubSelfTestClock.now] + + instance_stub = simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "hook #{@name}" } + singleton_stub = simple_stub(SimpleStubSelfTestClock, :now) { :hook_now } + + events << [ + :before_test, + SimpleStubSelfTestSubject.new("Alice").name, + SimpleStubSelfTestClock.now, + instance_stub.class, + singleton_stub.class + ] + test_run.run + events << [:after_test, SimpleStubSelfTestSubject.new("Alice").name, SimpleStubSelfTestClock.now] + end + 2.times do |index| + suite.tests.add( + SimpleStubSelfTest.test_case( + "uses around_test method stubs #{index}", + proc do + events << [:test, SimpleStubSelfTestSubject.new("Alice").name, SimpleStubSelfTestClock.now] + end + ) + ) + end + + status, = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(0) + expect(events).to eq( + [ + [:before_hook, "original Alice", :original_now], + [:before_test, "hook Alice", :hook_now, Smartest::SimpleStub, Smartest::SimpleStub], + [:test, "hook Alice", :hook_now], + [:after_test, "hook Alice", :hook_now], + [:before_hook, "original Alice", :original_now], + [:before_test, "hook Alice", :hook_now, Smartest::SimpleStub, Smartest::SimpleStub], + [:test, "hook Alice", :hook_now], + [:after_test, "hook Alice", :hook_now] + ] + ) + expect(SimpleStubSelfTestSubject.new("Alice").name).to eq("original Alice") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("around_suite method stub helpers apply for the whole hook scope") do + events = [] + suite = Smartest::Suite.new + + suite.around_suite_hooks << proc do |suite_run| + instance_stub = simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "suite #{@name}" } + singleton_stub = simple_stub(SimpleStubSelfTestClock, :now) { :suite_now } + + events << [ + :before_suite, + SimpleStubSelfTestSubject.new("Alice").name, + SimpleStubSelfTestClock.now, + instance_stub.class, + singleton_stub.class + ] + suite_run.run + events << [:after_suite, SimpleStubSelfTestSubject.new("Alice").name, SimpleStubSelfTestClock.now] + end + 2.times do |index| + suite.tests.add( + SimpleStubSelfTest.test_case( + "uses around_suite method stubs #{index}", + proc do + events << [:test, SimpleStubSelfTestSubject.new("Alice").name, SimpleStubSelfTestClock.now] + end + ) + ) + end + + status, = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(0) + expect(events).to eq( + [ + [:before_suite, "suite Alice", :suite_now, Smartest::SimpleStub, Smartest::SimpleStub], + [:test, "suite Alice", :suite_now], + [:test, "suite Alice", :suite_now], + [:after_suite, "suite Alice", :suite_now] + ] + ) + expect(SimpleStubSelfTestSubject.new("Alice").name).to eq("original Alice") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("nested around hooks restore the previous hook stub as each hook exits") do + events = [] + subject = SimpleStubSelfTestSubject.new("Alice") + suite = Smartest::Suite.new + + suite.around_suite_hooks << proc do |suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "outer suite #{@name}" } + events << [:outer_suite_before, subject.name] + suite_run.run + events << [:outer_suite_after, subject.name] + end + suite.around_suite_hooks << proc do |suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "inner suite #{@name}" } + events << [:inner_suite_before, subject.name] + suite_run.run + events << [:inner_suite_after, subject.name] + end + suite.tests.add( + Smartest::TestCase.new( + name: "uses nested hook stubs", + metadata: {}, + location: caller_locations(1, 1).first, + around_test_hooks: [ + proc do |test_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "outer test #{@name}" } + events << [:outer_test_before, subject.name] + test_run.run + events << [:outer_test_after, subject.name] + end, + proc do |test_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "inner test #{@name}" } + events << [:inner_test_before, subject.name] + test_run.run + events << [:inner_test_after, subject.name] + end + ], + block: proc { events << [:test, subject.name] } + ) + ) + + status, = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(0) + expect(events).to eq( + [ + [:outer_suite_before, "outer suite Alice"], + [:inner_suite_before, "inner suite Alice"], + [:outer_test_before, "outer test Alice"], + [:inner_test_before, "inner test Alice"], + [:test, "inner test Alice"], + [:inner_test_after, "inner test Alice"], + [:outer_test_after, "outer test Alice"], + [:inner_suite_after, "inner suite Alice"], + [:outer_suite_after, "outer suite Alice"] + ] + ) + expect(subject.name).to eq("original Alice") +end + +test("hook, test fixture, and suite fixture stubs restore the previous active stub") do + events = [] + subject = SimpleStubSelfTestSubject.new("Alice") + + fixture_class = Class.new(Smartest::Fixture) do + suite_fixture :suite_name_stub do + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "suite fixture #{@name}" } + end + + fixture :test_name_stub do |suite_name_stub:| + events << [:test_fixture_setup, suite_name_stub.class] + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "test fixture #{@name}" } + end + end + + suite = Smartest::Suite.new + suite.fixture_classes.add(fixture_class) + suite.around_suite_hooks << proc do |suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "suite hook #{@name}" } + events << [:suite_hook_before, subject.name] + suite_run.run + events << [:suite_hook_after, subject.name] + end + suite.around_test_hooks << proc do |test_run| + events << [:test_hook_before_apply, subject.name] + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "test hook #{@name}" } + events << [:test_hook_before_run, subject.name] + test_run.run + events << [:test_hook_after_run, subject.name] + end + suite.tests.add( + SimpleStubSelfTest.test_case( + "activates suite fixture stub", + proc do |suite_name_stub:| + events << [:suite_fixture_test, suite_name_stub.class, subject.name] + end + ) + ) + suite.tests.add( + SimpleStubSelfTest.test_case( + "temporarily overrides every outer stub", + proc do |test_name_stub:| + events << [:test_fixture_test, test_name_stub.class, subject.name] + end + ) + ) + + status, = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(0) + expect(events).to eq( + [ + [:suite_hook_before, "suite hook Alice"], + [:test_hook_before_apply, "suite hook Alice"], + [:test_hook_before_run, "test hook Alice"], + [:suite_fixture_test, Smartest::SimpleStub, "suite fixture Alice"], + [:test_hook_after_run, "suite fixture Alice"], + [:test_hook_before_apply, "suite fixture Alice"], + [:test_hook_before_run, "test hook Alice"], + [:test_fixture_setup, Smartest::SimpleStub], + [:test_fixture_test, Smartest::SimpleStub, "test fixture Alice"], + [:test_hook_after_run, "test hook Alice"], + [:suite_hook_after, "suite hook Alice"] + ] + ) + expect(subject.name).to eq("original Alice") +end + +test("hook method stubs reset when tests or suite hooks fail") do + subject = SimpleStubSelfTestSubject.new("Alice") + + test_failure_suite = Smartest::Suite.new + test_failure_suite.around_test_hooks << proc do |test_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "failing test #{@name}" } + test_run.run + end + test_failure_suite.tests.add( + SimpleStubSelfTest.test_case( + "fails while stubbed", + proc do + expect(subject.name).to eq("failing test Alice") + raise "test failed" + end + ) + ) + + test_status, test_output = SimpleStubSelfTest.run_suite(test_failure_suite) + + expect(test_status).to eq(1) + expect(test_output).to include("RuntimeError: test failed") + expect(subject.name).to eq("original Alice") + + failing_test_suite = Smartest::Suite.new + failing_test_suite.around_suite_hooks << proc do |suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "failing suite test #{@name}" } + suite_run.run + end + failing_test_suite.tests.add( + SimpleStubSelfTest.test_case( + "fails inside suite stub", + proc do + expect(subject.name).to eq("failing suite test Alice") + raise "suite test failed" + end + ) + ) + + failing_test_status, failing_test_output = SimpleStubSelfTest.run_suite(failing_test_suite) + + expect(failing_test_status).to eq(1) + expect(failing_test_output).to include("RuntimeError: suite test failed") + expect(subject.name).to eq("original Alice") + + suite_failure_suite = Smartest::Suite.new + suite_failure_suite.around_suite_hooks << proc do |suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "failing suite #{@name}" } + suite_run.run + raise "suite hook failed" + end + suite_failure_suite.tests.add( + SimpleStubSelfTest.test_case( + "passes before suite hook failure", + proc { expect(subject.name).to eq("failing suite Alice") } + ) + ) + + suite_status, suite_output = SimpleStubSelfTest.run_suite(suite_failure_suite) + + expect(suite_status).to eq(1) + expect(suite_output).to include("RuntimeError: suite hook failed") + expect(subject.name).to eq("original Alice") +end + +test("hook method stubs reset when hooks do not call their run target") do + subject = SimpleStubSelfTestSubject.new("Alice") + + missing_test_run_suite = Smartest::Suite.new + missing_test_run_suite.around_test_hooks << proc do |_test_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "missing test run #{@name}" } + end + missing_test_run_suite.tests.add( + SimpleStubSelfTest.test_case("not reached", proc { raise "test body should not run" }) + ) + + test_status, test_output = SimpleStubSelfTest.run_suite(missing_test_run_suite) + + expect(test_status).to eq(1) + expect(test_output).to include("around_test hook did not call test.run") + expect(subject.name).to eq("original Alice") + + missing_suite_run_suite = Smartest::Suite.new + missing_suite_run_suite.around_suite_hooks << proc do |_suite_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "missing suite run #{@name}" } + end + missing_suite_run_suite.tests.add( + SimpleStubSelfTest.test_case("not reached", proc { raise "test body should not run" }) + ) + + suite_status, suite_output = SimpleStubSelfTest.run_suite(missing_suite_run_suite) + + expect(suite_status).to eq(1) + expect(suite_output).to include("around_suite hook did not call suite.run") + expect(subject.name).to eq("original Alice") +end + +test("around_test method stubs reset when the hook skips before test.run") do + subject = SimpleStubSelfTestSubject.new("Alice") + suite = Smartest::Suite.new + + suite.around_test_hooks << proc do |_test_run| + simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "skipped #{@name}" } + skip "unsupported" + end + suite.tests.add( + SimpleStubSelfTest.test_case("skipped while stubbed", proc { raise "test body should not run" }) + ) + + status, output = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(0) + expect(output).to include("skipped: unsupported") + expect(subject.name).to eq("original Alice") +end + +test("hook method stub teardown resets remaining stubs when one reset fails") do + subject = SimpleStubSelfTestSubject.new("Alice") + suite = Smartest::Suite.new + + suite.around_suite_hooks << proc do |suite_run| + manually_reset_stub = simple_stub_any_instance_of(SimpleStubSelfTestSubject, :name) { "stubbed #{@name}" } + simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + manually_reset_stub.reset + suite_run.run + end + suite.tests.add( + SimpleStubSelfTest.test_case("passes before teardown failure", proc { expect(true).to eq(true) }) + ) + + status, output = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(1) + expect(output).to include("Smartest::SimpleStub::NotAppliedError") + expect(subject.name).to eq("original Alice") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("around_test stub teardown failures do not replace primary test failures") do + suite = Smartest::Suite.new + + suite.around_test_hooks << proc do |test_run| + manually_reset_stub = simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + manually_reset_stub.reset + test_run.run + end + suite.tests.add( + SimpleStubSelfTest.test_case("fails before hook teardown", proc { raise "primary test failure" }) + ) + + status, output = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(1) + expect(output).to include("RuntimeError: primary test failure") + expect(output).to include("teardown failed: Smartest::SimpleStub::NotAppliedError") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("aggregates fixture and hook teardown failures in cleanup order") do + fixture_class = Class.new(Smartest::Fixture) do + fixture :value do + on_teardown { raise "fixture teardown failure" } + :value + end + end + suite = Smartest::Suite.new + suite.fixture_classes.add(fixture_class) + suite.around_test_hooks << proc do |test_run| + manually_reset_stub = simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + manually_reset_stub.reset + test_run.run + end + suite.tests.add( + SimpleStubSelfTest.test_case("has two teardown failures", proc { |value:| expect(value).to eq(:value) }) + ) + + status, output = SimpleStubSelfTest.run_suite(suite) + fixture_error_index = output.index("teardown failed: RuntimeError: fixture teardown failure") + hook_error_index = output.index("teardown failed: Smartest::SimpleStub::NotAppliedError") + + expect(status).to eq(1) + expect(fixture_error_index.nil?).to eq(false) + expect(hook_error_index.nil?).to eq(false) + expect(fixture_error_index < hook_error_index).to eq(true) + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + +test("around_suite stub teardown failures do not replace primary suite failures") do + suite = Smartest::Suite.new + + suite.around_suite_hooks << proc do |suite_run| + manually_reset_stub = simple_stub(SimpleStubSelfTestClock, :now) { :stubbed_now } + manually_reset_stub.reset + suite_run.run + raise "primary suite failure" + end + suite.tests.add( + SimpleStubSelfTest.test_case("passes before suite failure", proc { expect(true).to eq(true) }) + ) + + status, output = SimpleStubSelfTest.run_suite(suite) + + expect(status).to eq(1) + expect(output).to include("RuntimeError: primary suite failure") + expect(output).to include("teardown failed: Smartest::SimpleStub::NotAppliedError") + expect(SimpleStubSelfTestClock.now).to eq(:original_now) +end + test("with_stub_const applies and resets existing constants in test body blocks") do result = with_stub_const("SimpleStubSelfTestConfig::PROVIDER", :stubbed_provider) do expect(SimpleStubSelfTestConfig::PROVIDER).to eq(:stubbed_provider) diff --git a/smartest/smartest_test.rb b/smartest/smartest_test.rb index 5ead59f..6c88835 100644 --- a/smartest/smartest_test.rb +++ b/smartest/smartest_test.rb @@ -1196,10 +1196,12 @@ def late_helper_value expect(output).to include("1 test, 0 passed, 1 failed") end -test("use_fixture and use_matcher are only available inside hooks") do +test("hook registration and method stub helpers are not top-level DSL methods") do { "use_fixture Object" => "use_fixture", - "use_matcher Module.new" => "use_matcher" + "use_matcher Module.new" => "use_matcher", + "simple_stub(Object, :name) { :stubbed }" => "simple_stub", + "simple_stub_any_instance_of(Object, :to_s) { :stubbed }" => "simple_stub_any_instance_of" }.each do |registration, method_name| Dir.mktmpdir do |dir| smartest_dir = File.join(dir, "smartest") @@ -1274,6 +1276,26 @@ def late_helper_value expect(output).to include("1 test, 1 passed, 0 failed, 1 suite teardown failed") end +test("fixture set teardown is idempotent during reentry") do + events = [] + teardown_error = RuntimeError.new("inner teardown failed") + fixture_set = Smartest::FixtureSet.new([], context: Object.new) + fixture_set.add_teardown { events << :outer_teardown } + fixture_set.add_teardown do + events << :inner_teardown_started + fixture_set.run_teardowns + events << :inner_teardown_finished + raise teardown_error + end + + expect(fixture_set.run_teardowns).to eq(fixture_set) + expect(fixture_set.teardown_errors).to eq([teardown_error]) + expect(events).to eq(%i[inner_teardown_started inner_teardown_finished outer_teardown]) + expect(fixture_set.run_teardowns).to eq(fixture_set) + expect(fixture_set.teardown_errors).to eq([teardown_error]) + expect(events).to eq(%i[inner_teardown_started inner_teardown_finished outer_teardown]) +end + test("runs teardown in reverse order after failures") do events = []