Skip to content

12 min instead of 49 for generation - #66

Open
kimstik wants to merge 4 commits into
CadQuery:arg-fixfrom
kimstik:fast_build
Open

kimstik wants to merge 4 commits into
CadQuery:arg-fixfrom
kimstik:fast_build

Conversation

@kimstik

@kimstik kimstik commented Sep 20, 2026

Copy link
Copy Markdown

While working on OCP I waited four hours for every CI run. Generating OCP with pywrap was the first hour of it, three times over, one per platform. Generating OCP (320 modules, ~7000 headers) on a 4-core runner went from 49 minutes to 12 on Linux, and about the same for Windows. The OSX bindings took 169 minutes on a mac runner and take 15 once they are generated on Linux as well (OCP side in CadQuery/OCP#229). Four commits:

  • PCH of the shared preamble, parsed once per worker: parse 32 -> 10 min per platform, output unchanged.
  • Symbol lookup by bisection instead of a table scan per method: transform 27 min -> 28 s on OSX.
  • Collections chunks of 25 instead of 100: the largest translation unit 5 -> 3.2 GB, which is what lets OCP compile with four jobs.
  • --target=x86_64-apple-darwin and the platform settings taken from the target's section, so the OSX bindings generate on Linux.

Found on the way and fixed:

  • the platform settings (prefix, includes, parsing_header) came from the host section, not the target's.
  • the default args list of parse_tu grew with every call.

kimstik added 4 commits September 19, 2026 09:34
The platform settings (prefix, includes, parsing_header) describe the target, not the host.
A chunk of 100 compiles into a 5 GB translation unit; three of them at once fill a 16 GB runner.
The undefined-symbol filter scanned the whole symbol table once per method. pandas was only used for that table.
Every one of the ~7000 headers re-expanded the same preamble (parsing_header, the platform header, Standard_Handle.hxx and what it pulls in); a precompiled header of it is reused through -include-pch. The preamble text stays in the translation unit as well: its include is guarded, and its forward declarations must remain visible in dummy.cxx. The default argument list is copied instead of appended to, which also keeps the cache key stable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant