Skip to content

fix(runtime): resolve authenticated Sol 401 and missing Azure models after gateway recovery #262

Closed as not planned

Description

@ChefGroep

Observed on 2026-09-19 22:04 UTC against existing OCX v1.4.2 at bc-scan-2 (100.65.83.86:10100):

  • Service-token-authorized GET /v1/models returns 200 but only seven IDs: gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex-spark, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna. No Azure deployment entry appears. Do not equate catalog presence with available upstream.
  • Minimal POST /v1/responses using the same valid service token, model=gpt-5.6-sol, input='Reply with OK only.', max_output_tokens=16 returns HTTP 401 in 0.19 s. This is a routed-inference failure, not a gateway reachability failure; precise internal/upstream origin of the 401 is NOT established. Do not print credential values or unredacted HTTP bodies while diagnosing.
  • Relay recovery has been applied with backups: AWS nginx points to existing OCX on bc-scan-2 with Host ocx.chefgroep.online; /healthz 200; unauthenticated /api/usage and /v1/models 401. See issue fix(deploy): use verified Cloudflare Access session for remote OCX dashboard, never prompt for admin token #258 execution comment.

Tasks: (1) Inspect safe, redacted runtime provider discovery and credential validity; distinguish admission 401 from upstream 401. (2) Reconcile OpenAI Pool/Direct routes and stale bearer keys without replacing working login or data-plane token. (3) Verify actual Azure Foundry account/deployments, correct azure-openai endpoint and api-key header, and only add real available deployments. (4) Repeat minimal authenticated inference smoke, streaming/tool-call test and CLI client smoke. (5) Record per-provider request ID, HTTP status, model and runtime source identity in a secret-free diagnostic receipt. Stop on quota/spend and never invent deployments.

Completion requires Sol inference 2xx with correct provider route and Azure visibility supported by actual deployed endpoint evidence, not just a 200 healthcheck. No DNS cutover or release deploy is authorized by this issue.

Activity

  1. github-actions commented on Sep 19, 2026

    @github-actions
    Contributor

    Automated translation bookkeeping — detected language: unknown.

  2. github-actions commented on Sep 19, 2026

    @github-actions
    Contributor

    Issue closed: use an issue template

    Thanks for taking the time to submit this issue. It was closed automatically because it was not opened with a recognized issue template (Bug report, Feature request, Documentation, or Provider compatibility). Freeform and API-opened issues cannot skip this check.

    • This issue does not use a recognized issue template.

    Please edit the issue to add:

    • Open a new issue with the Bug report, Feature request, Documentation, or Provider compatibility template.
    • Or edit this issue so the body uses the template section headings for the kind of report you are filing.

    See the README for details.

    Once the report passes the automated checks, it will be reopened automatically unless a maintainer has changed its state.

  3. ChefGroep commented on Sep 19, 2026

    @ChefGroep
    ContributorAuthor

    Execution evidence, 2026-09-20 around 00:10 Europe/Amsterdam (read-only except bounded Azure inference):

    1. Existing bc-scan-2 OCX v1.4.2 remains healthy. AWS relay /healthz 200, dedicated service-token /v1/models 200, public/unauthenticated data and management routes 401. AWS OCX runner active. No production cutover performed.
    2. Authenticated management /api/providers shows exactly one configured provider, openai (openai-responses, authMode=forward, host chatgpt.com, hasApiKey=false); /api/config lists only openai. No Azure provider is wired. The observed Sol 401 must not be called an Azure failure. Forward mode expects caller upstream credentials; the bare gateway service-token smoke did not establish such a credential. See src/adapters/openai-responses.ts and its FORWARD_HEADERS allowlist.
    3. The logged-in Azure subscription on joep has three AI resources. openaichef-se (Sweden): grok-4-6, DeepSeek-V4-1-Flash; openaichef (East US): grok-4-6, DeepSeek-V4-1-Flash, DeepSeek-V4-Pro, DeepSeek-V4-Pro-dz; azure-foundry-us has no deployments. Azure ARM lists all six with provisioning Succeeded; no Astra/Sol Azure deployment on this subscription.
    4. Actual data-plane verification using az rest --resource https://ai.azure.com with cached Entra auth, no keys printed: Sweden grok-4-6 Responses completed; East US grok-4-6 Responses completed; East US DeepSeek-V4-Pro and DeepSeek-V4-Pro-dz chat completions each returned one choice. Sweden and East US DeepSeek-V4-1-Flash both returned unknown_model (backend reports DeepSeek-V4.1-Flash), despite ARM Succeeded. Mark Flash unavailable until its backend discrepancy is resolved.
    5. Existing local 0600 env files on joep have key material but contain stale deployment names (gpt-4.1-mini, not deployed on these resources) and an older jort-7512-resource account not shown in the active subscription. Do not copy them blindly or push values to Git. The Azure adapter currently requires a nonempty provider apiKey, uses api-key header and disallows forward mode. Attempts to execute token-bearing diagnostic requests through remote tooling were blocked by its safety controls; no secret was exported, rotated, or inserted into the live OCX config.

    Next executable gate: use approved file-backed ChefVault delivery or another authorized secret injection for an actual resource key, create Azure provider(s) at real https://<resource>.services.ai.azure.com/openai/v1 endpoints with tested deployment aliases only, then verify /v1/responses through OCX. Native openai forward must be tested with a legitimate Codex caller's upstream Authorization separately, not with an OCX service token alone. Hold Flash, 1.5.0 release, DNS and OIDC cutover until exact-head tests and runtime inference are proven.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions