Conversation
…ntation add setting to allow default AWS profile selection for LLMs
make AWS guardrails settings configurable in admin
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds AWS Bedrock as a new provider option in the GenAI Orchestrator stack (server + core/client + Studio UI), including optional inline Bedrock Guardrails support, and introduces an AWS Bedrock-based reranking (document compressor) implementation plus new AWS RAG setup guides.
Changes:
- Add AWS Bedrock LLM + embedding model settings/providers end-to-end (Python orchestrator-server, Kotlin core/client models/mappers, and Bot Admin UI).
- Add AWS Bedrock rerank-based document compressor (implementation + factory + tests), wired into the provider selection factory and provider routers.
- Add documentation updates, including step-by-step RAG-on-AWS guides (EN/FR) and API doc schema refresh.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| gen-ai/orchestrator-server/src/main/python/server/tests/services/test_langchain_factory.py | Adds tests for Bedrock LLM/EM/compressor factories and guardrail intervention check. |
| gen-ai/orchestrator-server/src/main/python/server/tests/services/langchain/impls/document_compressor/test_aws_bedrock_rerank.py | Adds unit tests for Bedrock rerank compressor behavior and fault tolerance. |
| gen-ai/orchestrator-server/src/main/python/server/tests/routers/test_llm_providers_router.py | Adds router test for Bedrock LLM setting example endpoint. |
| gen-ai/orchestrator-server/src/main/python/server/tests/routers/test_em_providers_router.py | Adds router test for Bedrock embedding setting example endpoint. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/impls/document_compressor/aws_bedrock_rerank.py | Implements Bedrock Rerank-based document compressor. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/factories/llm/aws_bedrock_llm_factory.py | Implements Bedrock LLM factory (incl. optional inline guardrails config). |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/factories/langchain_factory.py | Wires Bedrock LLM/EM/compressor factories into the main factory selection. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/factories/em/aws_bedrock_em_factory.py | Implements Bedrock embeddings factory. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/factories/document_compressor/aws_bedrock_compressor_factory.py | Adds factory for Bedrock rerank compressor. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/routers/llm_providers_router.py | Adds Bedrock LLM provider “setting example” endpoint support. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/routers/em_providers_router.py | Adds Bedrock embedding provider “setting example” endpoint support. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/routers/document_compressor_providers_router.py | Adds Bedrock compressor provider “setting example” endpoint support. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/llm/llm_types.py | Extends LLMSetting union to include AwsBedrockLLMSetting. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/llm/llm_provider.py | Adds AWS_BEDROCK to LLMProvider enum. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/llm/awsbedrock/aws_bedrock_llm_setting.py | Adds Pydantic setting model for Bedrock LLM (incl. guardrail fields). |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/llm/awsbedrock/init.py | Adds Bedrock LLM model package init. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/em/em_types.py | Extends EMSetting union to include AwsBedrockEMSetting. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/em/em_provider.py | Adds AWS_BEDROCK to EMProvider enum. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/em/awsbedrock/aws_bedrock_em_setting.py | Adds Pydantic setting model for Bedrock embeddings. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/em/awsbedrock/init.py | Adds Bedrock EM model package init. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/document_compressor/document_compressor_types.py | Extends compressor setting union to include AwsBedrockCompressorSetting. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/document_compressor/document_compressor_provider.py | Adds AWS_BEDROCK to DocumentCompressorProvider enum. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/models/document_compressor/awsbedrock/aws_bedrock_compressor_setting.py | Adds Pydantic setting model for Bedrock rerank compressor. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/errors/handlers/aws_bedrock/aws_bedrock_exception_handler.py | Adds Bedrock-specific exception mapping for botocore errors. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/errors/handlers/aws_bedrock/init.py | Adds Bedrock handler package init. |
| gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/configurations/environment/settings.py | Adds Bedrock credential profile configuration settings. |
| gen-ai/orchestrator-server/src/main/python/server/pyproject.toml | Adds langchain-aws dependency. |
| gen-ai/orchestrator-core/src/test/kotlin/ai/tock/genai/orchestratorcore/models/llm/LLMSettingReasoningEffortTest.kt | Adds Bedrock LLM test asserting no reasoningEffort. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/llm/LLMSettingBase.kt | Registers Bedrock LLM subtype in Jackson polymorphism. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/llm/LLMProvider.kt | Adds AwsBedrock to Kotlin LLMProvider enum. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/llm/AwsBedrockLLMSetting.kt | Adds Kotlin Bedrock LLM setting (incl. guardrail fields). |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/em/EMSettingBase.kt | Registers Bedrock EM subtype in Jackson polymorphism. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/em/EMProvider.kt | Adds AwsBedrock to Kotlin EMProvider enum. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/em/AwsBedrockEMSetting.kt | Adds Kotlin Bedrock embedding setting model. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/models/Constants.kt | Adds AWS_BEDROCK discriminator constant. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/mappers/LLMSettingMapper.kt | Adds mapping support for Bedrock LLM settings. |
| gen-ai/orchestrator-core/src/main/kotlin/ai/tock/genai/orchestratorcore/mappers/EMSettingMapper.kt | Adds mapping support for Bedrock embedding settings. |
| docs/docs/fr/user/studio/gen-ai/features/gen-ai-feature-rag.md | Fixes wording for “Retrieval augmented generation”. |
| docs/docs/fr/guides/rag-aws.md | Adds French “RAG on AWS” setup guide. |
| docs/docs/fr/dev/gen_ai_orchestrator/api.md | Updates schema documentation (Mermaid diagram incl. Bedrock). |
| docs/docs/en/guides/rag-aws.md | Adds English “RAG on AWS” setup guide. |
| docs/docs/en/dev/gen_ai_orchestrator/api.md | Updates schema documentation (Mermaid diagram incl. Bedrock). |
| bot/admin/web/src/app/shared/model/ai-settings.ts | Adds Bedrock provider + model lists for UI selections. |
| bot/admin/web/src/app/rag/rag-settings/models/engines-configurations.ts | Adds Bedrock LLM/EM configuration options in RAG settings UI. |
| bot/admin/web/src/app/configuration/sentence-generation-settings/models/engines-configuration.ts | Adds Bedrock as sentence-generation engine option. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+129
to
+133
| @aws_bedrock_exception_handler(provider='AwsBedrock') | ||
| async def invoke( | ||
| self, _input: Input, config: Optional[RunnableConfig] = None | ||
| ) -> Output: | ||
| return await super().invoke(_input, config) |
Comment on lines
+58
to
+72
| if message.response_metadata.get('stopReason') == 'guardrail_intervened': | ||
| guardrail_trace = ( | ||
| message.response_metadata.get('trace', {}) | ||
| .get('guardrail', {}) | ||
| .get('actionReason') | ||
| ) | ||
| cause = guardrail_trace or 'The AWS Bedrock guardrail intervened on the model output.' | ||
| raise GenAIGuardCheckException( | ||
| ErrorInfo( | ||
| provider='AwsBedrock', | ||
| error='GuardrailIntervened', | ||
| cause=cause, | ||
| ) | ||
| ) | ||
| return message |
Comment on lines
+35
to
+43
| def get_compressor(self) -> AwsBedrockRerank: | ||
| return AwsBedrockRerank( | ||
| model_arn=self.setting.model_arn, | ||
| credentials_profile_name=application_settings.aws_bedrock_credentials_profile_name, | ||
| min_score=self.setting.min_score, | ||
| max_documents=self.setting.max_documents, | ||
| fill_to_max_documents=self.setting.fill_to_max_documents, | ||
| is_fault_tolerant=self.is_fault_tolerant, | ||
| ) |
Collaborator
|
@Fabilin ping me when the PR is ready ;) |
zigzago
self-requested a review
August 31, 2026 10:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR resolves #2070 by:
When using a Bedrock model, there are additional options to use the integrated guardrails feature.
Finally, this PR adds a step-by-step guide to setting up RAG on AWS.
AI disclaimer: most of the code implementing the various providers and mappers has been generated using Claude Sonnet 5