Repository navigation
[AINode] Add Timestral as a built-in forecasting model, downloaded on load - #19272
Open
kotselidis wants to merge 1 commit into
Open
kotselidis wants to merge 1 commit into
kotselidis wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds Timestral (Mistral AI) to AINode's built-in forecasting models, next to
timer_xl,sundial,chronos2,moirai2,totoandmoment. Timestral is a patch-based transformer for multivariate probabilistic forecasting. It forecasts 21 quantiles (0.01 to 0.99) from a context of up to 32,768 steps, and it can use past-only and future-known covariates. The model is registered astimestral, with the weights ofmistralai/Timestral-209M. The 209M model is the largest of the four published sizes (5M, 28M, 123M, 209M) and the one Timestral's own documentation uses.Unlike the other built-in models, AINode does not download Timestral's weights at startup. They are downloaded when a user loads the model with
LOAD MODEL. The reason is the weights' licence, below.Weights licence
Timestral's code is under the Apache License 2.0. Its weights are not. Every published size is under a "Modified MIT" licence (licence text), whose second condition reads:
The weights are not part of the AINode distribution in either case. However, AINode downloads the weights of every built-in model when it starts. As an automatic built-in, Timestral would put weights that some users are not licensed to use on every AINode installation, without anyone asking for them. With this PR, the weights reach a machine only when someone runs
LOAD MODEL timestral, which is a deliberate act by that user. The same question applies tomoirai2, whose weights (Salesforce/moirai-2.0-R-small) are under CC BY-NC 4.0 and are downloaded at startup today. The newdownload_on_startflag could be applied to it as well. That is left to the maintainers, and this PR does not changemoirai2.The maintainers may prefer a different policy, such as not shipping models with restricted weights at all. The Timestral integration does not depend on the on-request download, so it can follow whichever policy is chosen.
Usage
Once downloaded, the weights stay in AINode's model directory, and the model is active from the next start on.
Example: the input is 1024 points of
50 + 10·sin(2πt/24) + 0.01·t. The request is aTForecastReqwithoutputLength = 24, sent to AINode's inference manager on CPU with the Timestral-209M checkpoint. The response is 24 values (the median forecast). The first six:The mean absolute error is 0.031 over 24 steps and 0.049 over 96 steps.
On-request download of built-in models
ModelInfogets adownload_on_startflag, which isTrueby default. Every existing built-in model keeps it, so their behaviour does not change.ModelStorageregisters a built-in model withdownload_on_start=Falseasinactiveat startup, unless itsconfig.jsonandmodel.safetensorsare already in its directory.ModelStorage.download_builtin_modeldownloads the two files and marks the modelactive, orinactiveagain if the download fails. The download code is the existing one, moved into a method.LOAD MODELruns the download in the pool controller's background load task, before the inference pools start. TheLOAD MODELstatement returns immediately, as it does today.SHOW MODELSshowsactivatingduring the download, andSHOW LOADED MODELSshows the model once it is loaded.FORECASTandCALL INFERENCEon such a model before the download return error 1504 with the message shown above. Without this check, they would fail while loading the missing weights.Model code
iotdb/ainode/core/model/timestral/contains the inference code from mistralai/timestral 1.0.0 (Apache License 2.0), in the same way aschronos2/contains the code from chronos-forecasting. The model, forecaster, layers, masks, scaler, covariate encoding and configuration are unchanged except for these points:iotdb.ainode.core.model.timestralpackage, and files carry the ASF header followed by Mistral AI's copyright notice.loading.py) andTimestralForecaster.from_pretrainedare left out. AINode downloads and loads the checkpoint itself.AINode integration
modeling_timestral.TimestralForPredictionis the model, loaded throughPyTorchModelHubMixin(hub_mixin_clsinModelInfo), a loading path AINode already supports.config.jsonis read into Timestral's ownTimestralConfig, which rejects unknown keys. The weights load withstrict=True. The rotary-embedding frequencies are recomputed whenever the model is moved to a device, as Timestral's own loader does after moving the model.pipeline_timestral.TimestralPipelineruns Timestral'sTimestralForecasterwith its default settings:past_covariatesbecomes a past covariate. A covariate that also appears infuture_covariatesbecomes a known covariate, built from its past and future values.output_lengththat AINode accepts (up to 2880) works.Dependencies
There are no new dependencies. Timestral needs only PyTorch, NumPy, safetensors and huggingface_hub, which AINode already pins.
pyproject.toml,ainode.specandainode.xmlare unchanged.ainode.specalready collects every module underiotdb.ainode.core.Licences
The code is under the Apache License 2.0 (licence). This PR adds it to
LICENSE,NOTICEand AINode'sNOTICE-binary. See the note above about the licence of the weights.Tests
tests/test_timestral.py(unittest, CPU only, no downloads) uses a two-layer Timestral with random weights. The tests:TimestralPipeline, and check the output shapes for 1-D and 2-D targets over more than one decode block, and that the result is the forecaster's median;tests/test_builtin_model_download.py(unittest, no network:hf_hub_downloadis mocked) checks that:inactiveand not downloaded at startup, while the other built-ins are;download_builtin_modelfetches it and marks itactive, and a second call downloads nothing;activeat the next startup once downloaded;inactive;AINodeTestUtils.ON_REQUEST_BUILTIN_MODEL_MAPliststimestral. TheSHOW MODELStest accepts it in either state, and the newonRequestBuiltInModelLoadInferenceAndForecastTestloads it on the CPU, which downloads it, then runsCALL INFERENCEandFORECAST.SHOW MODELSfirst reportsinactive, andFORECASTreturns 1504.LOAD MODEL timestral TO DEVICES 'cpu'downloads the files and starts one request pool.FORECASTthen returns the 24 values above through that pool.This PR has:
for an unfamiliar reader.
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR
iotdb/ainode/core/model/timestral/(new): the Timestral model and forecaster,TimestralForPrediction,TimestralPipelineiotdb/ainode/core/model/model_info.py:ModelInfo.download_on_start;timestralentry inBUILTIN_HF_TRANSFORMERS_MODEL_MAPiotdb/ainode/core/model/model_storage.py:download_builtin_model; built-ins withdownload_on_start=Falseare not downloaded at startiotdb/ainode/core/inference/pool_controller.py:LOAD MODELdownloads such a model before loading itiotdb/ainode/core/rpc/handler.py: forecasts on a model that is not downloaded yet are rejectediotdb-core/ainode/tests/test_timestral.py,tests/test_builtin_model_download.py(new)integration-test/.../ainode/utils/AINodeTestUtils.java,.../it/AINodeSharedClusterIT.java:ON_REQUEST_BUILTIN_MODEL_MAPand its testsLICENSE,NOTICE,iotdb-core/ainode/src/assembly/resources/NOTICE-binary