Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 44 additions & 5 deletions docs/dbtool.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,11 @@ Multi-profile shape:
defaultProfile: prod
defaultPluginsDir: ./plugins # store-wide fallback for any profile
# that omits its own `pluginsDir`
defaultBackupDir: /var/backups/lightweight # store-wide fallback backup folder
profiles:
prod:
schema: dbo
backupDir: /var/backups/lightweight/prod # per-profile override of defaultBackupDir
connectionString: "DRIVER={ODBC Driver 18 for SQL Server};Server=...;Database=prod"
dev:
pluginsDir: ./dev-plugins # per-profile override
Expand Down Expand Up @@ -141,6 +143,24 @@ The effective plugin directory for a given run resolves as: `--plugins-dir`
CLI option → profile's own `pluginsDir` → top-level `defaultPluginsDir`
(possibly a list) → current working directory.

#### Backup folder (`defaultBackupDir` / `backupDir`)

`defaultBackupDir` names a folder backups are written to, and a profile can override it with its
own `backupDir`. This lets a deployment (an installer, say) pre-configure where backups go, so
`dbtool` and `dbtool-gui` use the same folder without every user setting it up. The effective
folder resolves like `pluginsDir`: the profile's own `backupDir` → top-level `defaultBackupDir` →
none (the behaviour described below for `--output`).

- [`dbtool backup`](#backup) uses it for a bare file name in `--output`, or for a generated name
when `--output` is omitted. A path with a directory part is used exactly as given. The folder is
created when it is missing.
- `dbtool-gui` uses `defaultBackupDir` as its default backup folder, unless a folder was chosen in
its Settings. It keeps every profile's archive in one folder, so a per-profile `backupDir` is
not used there.
- As with the other profile defaults, a run that gives `--connection-string` without `--profile`
applies no profile and therefore no backup folder.
- Relative paths are used as written, i.e. relative to the working directory.

### Profile passwords

A profile can carry its password in `dbtool.yml`. dbtool stores it **encrypted**:
Expand Down Expand Up @@ -221,11 +241,13 @@ any platform. Values of `PWD=` / `Password=` inside a profile's raw
$ dbtool list-profiles
Profiles (from /home/me/project/dbtool.yml, found via current directory):

NAME DEFAULT CONNECTION AUTH SCHEMA PLUGINSDIR
prod * DRIVER={ODBC Driver 18 for SQL Se... encrypted dbo ./migrations
dev DRIVER=SQLite3;Database=dev.db - ./dev-plugins
NAME DEFAULT CONNECTION AUTH SCHEMA PLUGINSDIR BACKUPDIR
prod * DRIVER={ODBC Driver 18 for SQL Se... encrypted dbo ./migrations /var/backups/lightweight/prod
dev DRIVER=SQLite3;Database=dev.db - ./dev-plugins /var/backups/lightweight
```

`BACKUPDIR` is the profile's effective backup folder: its own `backupDir`, else `defaultBackupDir`.

Use `--config <FILE>` to inspect a non-default configuration file. When no
config file is found, `list-profiles` prints where it looked and exits successfully.

Expand Down Expand Up @@ -497,7 +519,7 @@ printf '%s\n' "$DB_PASSWORD" | dbtool add-profile --name ci \
| `--name <NAME>` | Profile name (required) |
| `--connection-string <STR>` / `--dsn <DSN>` | Exactly one is required. A `PWD=` inside the connection string is moved into the encrypted `password` field |
| `--uid <UID>` | User name for a DSN profile |
| `--schema <S>`, `--plugins-dir <DIR>` | Stored on the profile |
| `--schema <S>`, `--plugins-dir <DIR>`, `--backup-dir <DIR>` | Stored on the profile (`--backup-dir` becomes its `backupDir`) |
| `--no-password` | Store the profile without a password |
| `--set-default` | Make it the `defaultProfile` |
| `--force` | Replace an existing profile of the same name |
Expand Down Expand Up @@ -528,6 +550,23 @@ Create a compressed backup of the database:
dbtool backup --output backup.zip --compression zstd --jobs 4
```

**Where the file goes.** When `dbtool.yml` names a backup folder for the selected profile
([`backupDir` / `defaultBackupDir`](#backup-folder-defaultbackupdir--backupdir)):

```bash
# a bare file name is written into that folder
dbtool backup --profile prod --output nightly.zip

# no --output: writes <profile>-<YYYYMMDD-HHMMSS>.zip (local time) into that folder
dbtool backup --profile prod

# a path with a directory part is used exactly as given
dbtool backup --profile prod --output /srv/exports/once.zip
```

The folder is created if it is missing, and the file that was written is printed. Without a
configured folder `--output` is required and is used as given.

**Compression methods:** `none`, `deflate`, `bzip2`, `lzma`, `zstd`, `xz`

**Filter tables with wildcards:**
Expand Down Expand Up @@ -602,7 +641,7 @@ common table is identical and both archives hold the same set of tables, or `1`
| `--schema <NAME>` | Database schema to use | |
| `--config <FILE>` | Path to configuration file | nearest `dbtool.yml` upward, then the per-user file (see [Configuration](#configuration)) |
| `--plugins-dir <DIR>` | Directory to scan for migration plugins | `.` (current directory) |
| `--output <FILE>` | Output file for backup | |
| `--output <FILE>` | Output file for backup. A bare file name, or no `--output`, goes into the profile's `backupDir` / `defaultBackupDir` when one is configured | |
| `--input <FILE>` | Input file for restore | |
| `--left <FILE>` | First (baseline) backup archive for `backup-diff` | |
| `--right <FILE>` | Second (candidate) backup archive for `backup-diff` | |
Expand Down
1 change: 1 addition & 0 deletions src/tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ endif()

set(SOURCE_FILES
ConfigDiscoveryTests.cpp
ConfigBackupOutputTests.cpp
ConfigProfileStoreTests.cpp
ColumnTypeRoundTripTests.cpp
CoreTests.cpp
Expand Down
103 changes: 103 additions & 0 deletions src/tests/ConfigBackupOutputTests.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// SPDX-License-Identifier: Apache-2.0
//
// Unit tests for where `dbtool backup` writes its archive when dbtool.yml names a backup folder.

#include <catch2/catch_test_macros.hpp>

#include <chrono>
#include <filesystem>
#include <regex>
#include <string>

#include <Config/BackupOutput.hpp>

namespace fs = std::filesystem;
namespace Cfg = Lightweight::Config;

namespace
{

Cfg::BackupOutputRequest Request(std::string requested, std::string folder, std::string_view profile = "prod")
{
return Cfg::BackupOutputRequest { .requested = std::move(requested),
.folder = std::move(folder),
.profileName = profile,
.now = std::chrono::system_clock::now() };
}

} // namespace

TEST_CASE("ResolveBackupOutput puts a bare file name into the configured folder", "[backup-dir]")
{
auto const output = Cfg::ResolveBackupOutput(Request("nightly.zip", "/srv/backups"));

if (!output)
{
FAIL("no backup output");
return;
}
CHECK(output->path == fs::path { "/srv/backups" } / "nightly.zip");
CHECK(output->inConfiguredFolder);
}

TEST_CASE("ResolveBackupOutput keeps a path that has a directory part", "[backup-dir]")
{
for (auto const* given:
{ "sub/nightly.zip", "./nightly.zip", "../nightly.zip", "/abs/nightly.zip", "D:/abs/nightly.zip" })
{
INFO(given);
auto const output = Cfg::ResolveBackupOutput(Request(given, "/srv/backups"));

if (!output)
{
FAIL_CHECK("no backup output");
continue;
}
CHECK(output->path == fs::path { given });
CHECK_FALSE(output->inConfiguredFolder);
}
}

TEST_CASE("ResolveBackupOutput generates a name when --output is omitted", "[backup-dir]")
{
auto const output = Cfg::ResolveBackupOutput(Request("", "/srv/backups", "prod"));

if (!output)
{
FAIL("no backup output");
return;
}
CHECK(output->inConfiguredFolder);
CHECK(output->path.parent_path() == fs::path { "/srv/backups" });
auto const name = output->path.filename().string();
CHECK(std::regex_match(name, std::regex { R"(prod-\d{8}-\d{6}\.zip)" }));
}

TEST_CASE("ResolveBackupOutput without a configured folder changes nothing", "[backup-dir]")
{
auto const named = Cfg::ResolveBackupOutput(Request("nightly.zip", ""));
if (!named)
{
FAIL("no backup output");
return;
}
CHECK(named->path == fs::path { "nightly.zip" });
CHECK_FALSE(named->inConfiguredFolder);

// Nothing requested and nowhere configured: there is no file to write.
CHECK_FALSE(Cfg::ResolveBackupOutput(Request("", "")).has_value());
}

TEST_CASE("GeneratedBackupFileName is safe in a file name", "[backup-dir]")
{
auto const now = std::chrono::system_clock::now();

auto const unsafe = Cfg::GeneratedBackupFileName("acme/prod: eu*", now);
CHECK(std::regex_match(unsafe, std::regex { R"(acme_prod__eu_-\d{8}-\d{6}\.zip)" }));

auto const anonymous = Cfg::GeneratedBackupFileName("", now);
CHECK(std::regex_match(anonymous, std::regex { R"(backup-\d{8}-\d{6}\.zip)" }));

// Non-ASCII letters are kept; they are fine in a file name.
CHECK(Cfg::GeneratedBackupFileName("M\xC3\xBCller", now).starts_with("M\xC3\xBCller-"));
}
91 changes: 91 additions & 0 deletions src/tests/ConfigProfileStoreTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -518,3 +518,94 @@ TEST_CASE("ProfileStore — a password field next to an inline PWD is ambiguous"
REQUIRE_FALSE(store.has_value());
CHECK(store.error().contains("PWD"));
}

TEST_CASE("ProfileStore — a profile's backupDir wins over defaultBackupDir", "[ProfileStore][backup-dir]")
{
ScopedTempYaml const yaml(R"(
defaultBackupDir: /srv/backups
profiles:
prod:
dsn: PROD
backupDir: /srv/backups/prod
dev:
dsn: DEV
)");

auto const result = Lightweight::Config::ProfileStore::LoadOrDefault(yaml.Path());

REQUIRE(result.has_value());
auto const& store = *result;
CHECK(store.DefaultBackupDir().generic_string() == "/srv/backups");
REQUIRE(store.Find("prod") != nullptr);
CHECK(store.Find("prod")->backupDir.generic_string() == "/srv/backups/prod");
CHECK(store.EffectiveBackupDir(*store.Find("prod")).generic_string() == "/srv/backups/prod");
// A profile without its own backupDir falls back to the store-wide default.
CHECK(store.Find("dev")->backupDir.empty());
CHECK(store.EffectiveBackupDir(*store.Find("dev")).generic_string() == "/srv/backups");
}

TEST_CASE("ProfileStore — no backup keys means no backup folder", "[ProfileStore][backup-dir]")
{
ScopedTempYaml const yaml(R"(
profiles:
prod:
dsn: PROD
)");

auto const result = Lightweight::Config::ProfileStore::LoadOrDefault(yaml.Path());

REQUIRE(result.has_value());
CHECK(result->DefaultBackupDir().empty());
CHECK(result->EffectiveBackupDir(*result->Find("prod")).empty());
}

TEST_CASE("ProfileStore — a blank backup key means none, not the text \"null\"", "[ProfileStore][backup-dir]")
{
ScopedTempYaml const yaml(R"(
defaultBackupDir:
profiles:
prod:
dsn: PROD
backupDir:
)");

auto const result = Lightweight::Config::ProfileStore::LoadOrDefault(yaml.Path());

REQUIRE(result.has_value());
CHECK(result->DefaultBackupDir().empty());
CHECK(result->Find("prod")->backupDir.empty());
}

TEST_CASE("ProfileStore — a file with only defaultBackupDir still carries it", "[ProfileStore][backup-dir]")
{
ScopedTempYaml const yaml("defaultBackupDir: /srv/backups\n");

auto const result = Lightweight::Config::ProfileStore::LoadOrDefault(yaml.Path());

REQUIRE(result.has_value());
CHECK(result->Empty());
CHECK(result->DefaultBackupDir().generic_string() == "/srv/backups");
}

TEST_CASE("ProfileStore — backup folders round-trip through Save", "[ProfileStore][backup-dir]")
{
Lightweight::Config::ProfileStore store;
store.SetDefaultBackupDir(std::filesystem::path { "/srv/backups" });
store.Upsert(Lightweight::Config::Profile { .name = "prod",
.pluginsDir = {},
.schema = {},
.dsn = "PROD",
.connectionString = {},
.uid = {},
.secretRef = {},
.password = {},
.backupDir = std::filesystem::path { "/srv/backups/prod" } });
ScopedTempYaml const yaml("");

REQUIRE(store.Save(yaml.Path()).has_value());
auto const reloaded = Lightweight::Config::ProfileStore::LoadOrDefault(yaml.Path());

REQUIRE(reloaded.has_value());
CHECK(reloaded->DefaultBackupDir().generic_string() == "/srv/backups");
CHECK(reloaded->Find("prod")->backupDir.generic_string() == "/srv/backups/prod");
}
9 changes: 6 additions & 3 deletions src/tests/ProfileAddTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ Cfg::ProfileRequest DsnRequest(std::string name = "warehouse")
.uid = "deploy",
.schema = "dbo",
.pluginsDir = {},
.password = "s3cret!" };
.password = "s3cret!",
.backupDir = {} };
}

} // namespace
Expand Down Expand Up @@ -133,7 +134,8 @@ TEST_CASE("AddProfileToFile moves a password out of the connection string and st
.uid = {},
.schema = {},
.pluginsDir = {},
.password = {} };
.password = {},
.backupDir = {} };

REQUIRE(
Cfg::AddProfileToFile(dir.File(), request, Cfg::ReplaceExisting::No, Cfg::MakeDefault::No, DevCipher()).has_value());
Expand All @@ -158,7 +160,8 @@ TEST_CASE("AddProfileToFile prefers an explicit password over one inside the con
.uid = {},
.schema = {},
.pluginsDir = {},
.password = "new" };
.password = "new",
.backupDir = {} };

REQUIRE(
Cfg::AddProfileToFile(dir.File(), request, Cfg::ReplaceExisting::No, Cfg::MakeDefault::No, DevCipher()).has_value());
Expand Down
22 changes: 22 additions & 0 deletions src/tests/ProfileFileEditorTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -387,3 +387,25 @@ TEST_CASE("ProfileFileEditor — edits that would not round-trip exactly are ref
INFO(forced.error());
CHECK(forced.error().contains("left unchanged"));
}

TEST_CASE("AddProfileText writes backupDir and leaves the rest of the file alone", "[ProfileFileEditor][backup-dir]")
{
auto const profile = Cfg::NewProfile { .name = "prod",
.connectionString = {},
.dsn = "PROD",
.uid = {},
.schema = {},
.pluginsDir = {},
.password = {},
.backupDir = "D:\\backups\\prod" };
auto const existing =
std::string { "# team file\ndefaultBackupDir: \"D:/backups\"\nprofiles:\n dev:\n dsn: \"DEV\"\n" };

auto const edited = Cfg::AddProfileText(existing, profile, Cfg::ReplaceExisting::No);

REQUIRE(edited.has_value());
CHECK(edited->contains("# team file"));
CHECK(edited->contains("defaultBackupDir: \"D:/backups\""));
CHECK(edited->contains("backupDir: \"D:\\\\backups\\\\prod\""));
CHECK(edited->contains("dev:"));
}
Loading
Loading