Skip to content

Commit c738c0e

Browse files
authored
Merge pull request #22810 from github/jacknojo-llm-output-threat-model
Add opt-in AI output threat model
2 parents 6bcc93a + 9ce464a commit c738c0e

6 files changed

Lines changed: 20 additions & 0 deletions

File tree

‎docs/codeql/reusables/threat-model-description.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ Note that subcategories can be turned included or excluded separately, so you ca
1111

1212
The less commonly used categories are:
1313

14+
- ``ai`` which represents output generated by large language models (LLMs). This threat model is opt-in.
1415
- ``android`` which represents reads from external files in Android (``android-external-storage-dir``) and parameter of an entry-point method declared in a ``ContentProvider`` class (``contentprovider``). Currently only used by Java/Kotlin.
1516
- ``database-access-result`` which represents a database access. Currently only used by JavaScript.
1617
- ``file-write`` which represents opening a file in write mode. Currently only used in C#.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
| ai |
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
extensions:
2+
- addsTo:
3+
pack: codeql/threat-models
4+
extensible: threatModelConfiguration
5+
data:
6+
- ["ai", true, 0]
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
private import codeql.threatmodels.ThreatModels
2+
3+
where
4+
knownThreatModel("ai") and
5+
currentThreatModel("ai")
6+
select "ai"
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
category: feature
3+
---
4+
* Added the opt-in `ai` threat model for LLM-generated data. No built-in source models are included.

‎shared/threat-models/ext/threat-model-grouping.model.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,5 @@ extensions:
3030
- ["file-write", "all"]
3131
- ["reverse-dns", "all"]
3232
- ["view-component-input", "all"]
33+
# AI threat models
34+
- ["ai", "all"]

0 commit comments

Comments
 (0)