Skip to content

fix(opencc): package official release dictionaries - #2231

Open
Bambooin wants to merge 2 commits into
osfans:developfrom
Bambooin:opencc-refactor
Open

Bambooin wants to merge 2 commits into
osfans:developfrom
Bambooin:opencc-refactor

Conversation

@Bambooin

Copy link
Copy Markdown
Collaborator

Pull request

Issue tracker

Fixes will automatically close the related issues

Fixes #2225
Close #2208

Feature

Describe features of this pull request

The dictionaries were generated by OpenCC's scripts and STPhrases_GeneratedFromRegionalPhrases can't be generated without opencc binary, so the s2t/s2tw/s2hk configs were installed without it and silently did no conversion.

Use the official release resource archive instead, it ships all dictionaries and config.

Code of conduct

Code style

Build pass

  • make debug

Manually test

  • Done

Code Review

  1. No wildcards import
  2. Manual build and test pass
  3. GitHub Action CI pass
  4. At least one contributor review and approve
  5. Merged clean without conflicts
  6. PR will be merged by rebase upstream base

Daily build

Login and download artifact at https://github.com/osfans/trime/actions

Additional Info

The dictionaries were generated by OpenCC's scripts and STPhrases_GeneratedFromRegionalPhrases can't be generated without opencc binary, so the s2t/s2tw/s2hk configs were installed without it and silently did no conversion.

Use the official release resource archive instead, it ships all dictionaries and config.

Fix osfans#2225
@WhiredPlanck

WhiredPlanck commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

This PR seems to download OpenCC data from network during building, I think this may not be a good idea:

  1. This may increase the difficulty of building on F-Droid, since F-Droid requires offline build environment.
  2. The data we need actually stay in the submodule, but still requesting them from network looks to be redundant.

@Bambooin

Copy link
Copy Markdown
Collaborator Author

The root cause of the issue is that STPhrases_GeneratedFromRegionalPhrases can't be generated without opencc binary, and it's used in s2t.json.

{
  "name": "Simplified Chinese to Traditional Chinese (OpenCC Standard)",
  "normalization": [
    {
      "dict": {
        "type": "text",
        "file": "CJK_Compatibility_Ideographs.txt"
      }
    }
  ],
  "conversion_chain": [
    {
      "dict": {
        "type": "group",
        "match_policy": "short_circuit",
        "dicts": [
          {
            "type": "group",
            "match_policy": "union",
            "dicts": [
              {
                "type": "text",
                "file": "STPhrases.txt"
              },
              {
                "type": "text",
                "file": "STPhrases_GeneratedFromRegionalPhrases.txt"
              }
            ]
          },
          {
            "type": "text",
            "file": "STCharacters.txt"
          }
        ]
      }
    }
  ]
}

This patch will remove the Python tool and simplify the opencc data logic, it's better solution than #2208 right now, IMO.

The F-Droid concern can be settled by local copy in the build.

@WhiredPlanck

WhiredPlanck commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

The root cause of the issue is that STPhrases_GeneratedFromRegionalPhrases can't be generated without opencc binary, and it's used in s2t.json.

{
  "name": "Simplified Chinese to Traditional Chinese (OpenCC Standard)",
  "normalization": [
    {
      "dict": {
        "type": "text",
        "file": "CJK_Compatibility_Ideographs.txt"
      }
    }
  ],
  "conversion_chain": [
    {
      "dict": {
        "type": "group",
        "match_policy": "short_circuit",
        "dicts": [
          {
            "type": "group",
            "match_policy": "union",
            "dicts": [
              {
                "type": "text",
                "file": "STPhrases.txt"
              },
              {
                "type": "text",
                "file": "STPhrases_GeneratedFromRegionalPhrases.txt"
              }
            ]
          },
          {
            "type": "text",
            "file": "STCharacters.txt"
          }
        ]
      }
    }
  ]
}

This patch will remove the Python tool and simplify the opencc data logic, it's better solution than #2208 right now, IMO.

The F-Droid concern can be settled by local copy in the build.

Hm, I think the mainstream approach may be building the OpenCC binary(-ies) on the host, or just installing it (them) on the host, and using it (them) to generate the data, since the data is supposed to be platform independent.

@Bambooin

Copy link
Copy Markdown
Collaborator Author

IMO, we shouldn't depend on the resource generation algorithm in the opencc submodule, it's will be a nightmare for us. We have to follow the change in the opencc. We don't want to patch our algorithm with opencc upgrade every time.

https://github.com/BYVoid/OpenCC/blob/master/.github/workflows/release-resource.yml

Under the case of new type dictionary is added or deleted in opencc, we don't want to adjust our generation algorithm.

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

we shouldn't depend on the resource generation algorithm in the opencc submodule

Yes, porting the algorithm to our build system is a hard work, but let's try a different approach, i.e. try to utilize OpenCC's build system directly, instead of repeating the same thing in the project. fcitx5-android makes its way to install Fcitx5 components with CMake , I think this may be of reference value.

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

we shouldn't depend on the resource generation algorithm in the opencc submodule

Yes, porting the algorithm to our build system is a hard work, but let's try a different approach, i.e. try to utilize OpenCC's build system directly, instead of repeating the same thing in the project. fcitx5-android makes its way to install Fcitx5 components with CMake , I think this may be of reference value.

@Bambooin The method fcitx5-android used requires building OpenCC's binary(-ies) on host, do you think it's acceptable?

@Bambooin

Copy link
Copy Markdown
Collaborator Author

We should avoid hard code the dictionary name in Trime for low coupling with upstream submodule.

All binary solution have one pain point with upsteam dictionary change issue.

We shouldn't care or reference the detail of the dictionary in Trime within the ideal world, it's the opencc team's concern.

We as the Trime App, only uses the opencc binary(Android) and it's assets to convert the word.

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

We should avoid hard code the dictionary name in Trime for low coupling with upstream submodule.

All binary solution have one pain point with upsteam dictionary change issue.

We shouldn't care or reference the detail of the dictionary in Trime within the ideal world, it's the opencc team's concern.

We as the Trime App, only uses the opencc binary(Android) and it's assets to convert the word.

Anyway, I will new a PR to show you to review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3.3.13版本後簡繁轉換失敗

2 participants