Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 76 additions & 1 deletion src/reference/specs/type-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -475,7 +475,82 @@ The `json` database type:
- Used as dtype by built-in codecs (`<object@>`, `<hash@>`, `<filepath@store>`)
- Stores arbitrary JSON-serializable data
- Automatically uses appropriate type for database backend
- Supports JSON path queries where available
- Supports JSON path access in restriction and projection — see below

### JSON path access

A field inside a `json` attribute is addressed as `attribute.path`, in both
restriction and projection. DataJoint translates the path per backend, so the
same expression runs on MySQL and PostgreSQL:

```python
@schema
class Equipment(dj.Manual):
definition = """
name : varchar(32)
---
specs : json
"""
```

**Restriction** — a mapping whose key carries the path:

```python
Equipment & {"specs.vendor": "Acme"}
```

**Projection** — a named attribute whose value is the path:

```python
Equipment.proj(vendor="specs.vendor")
```

Both emit `json_value()` on MySQL and `jsonb_extract_path_text()` on
PostgreSQL. Nested fields and array elements use dots and brackets:
`specs.probe.serial`, `specs.channels[0]`.

#### Extracted values are text

Both backend functions return text, so a numeric or boolean field comes back as
a string unless a type is requested:

```python
Equipment.proj(ch="specs.channels") # '64' — str
Equipment.proj(ch="specs.channels:unsigned") # 64 — int
```

The annotation is `path:type`. Portable spellings are `unsigned`, `signed` and
`decimal(M,D)`.

!!! warning "Two defects to know about when filtering"

**Compare against a string.** A restriction value that is a Python `int` or
`bool` is not coerced to match the extracted text. `& {"specs.calibrated":
True}` returns **no rows** on MySQL and raises on PostgreSQL; write
`& {"specs.calibrated": "true"}` and `& {"specs.channels": "64"}` until
[#1564](https://github.com/datajoint/datajoint-python/issues/1564) is fixed.

**`:int` is not portable.** It works on PostgreSQL and raises on MySQL, whose
`RETURNING` clause does not accept it. Use `unsigned` or `signed`, which work
on both — [#1563](https://github.com/datajoint/datajoint-python/issues/1563).

#### Ordering comparisons

A path yields text, so `>` and `<` need a typed projection first:

```python
big = Equipment.proj(ch="specs.channels:unsigned") & "ch > 32"
```

#### What is not supported

Comparing a whole JSON object — `& {"specs": {"vendor": "Acme", "channels": 64}}` —
raises a syntax error. Restrict on a path, or normalize the field into a column.

Hidden attributes — platform-managed columns whose names begin with `_`, such as
`_job_version` — cannot be reached by the mapping form at all: the predicate is
dropped and every row is returned
([#1561](https://github.com/datajoint/datajoint-python/issues/1561)).

## Built-in Codecs

Expand Down
Loading