Framework: the overlay producers, gcc 8.5, binutils 2.30 and rust by th0ma7 · Pull Request #7707 · SynoCommunity/spksrc

th0ma7 · 2026-11-02T12:43:55Z

Description

The producers of the overlay archives, split out of #7391. Since #7701, master downloads gcc 8.5, binutils 2.30 and rust archives into overlay/, but it can only rebuild the rust built with the vendor gcc: its native/binutils-2.30 still produces the older v3 archive, and there is no gcc producer. This PR brings the producers that made the published archives, so master can rebuild everything it downloads. Nothing a package builds changes.

native/gcc-8.5, with native/gmp, native/mpfr, native/mpc and native/isl, and a patch for the PowerPC SPE soft-float attribute.
native/binutils-2.30 produces the v4 archives the overlays download: profile-guided, a generic arch's archive under the real arch it shares a toolchain with, for the toolchains/dsm<dsm> releases.
native/rustc-1.82 on the same front-end.
A producer is a version, a rev and a component name. The three include mk/spksrc.native-toolchain.mk, which carries arch-<arch>-<dsm>, all-<dsm>, the host optimisation flags and the two profile-guided passes (TOOLCHAIN_PGO, on for gcc and binutils: cc1 8.5% faster at -O2 and 12.5% at -O3, as 10%), and dispatches to mk/spksrc.native/toolchain-{gcc,binutils,rust}.mk. A generic arch is built as the real arch whose archive it fetches (mk/spksrc.overlay/dist-arch.mk), and an archive that exists is not rebuilt, so all-<dsm> builds each one once, TOOLCHAIN_JOBS at a time, and resumes where it stopped. The PGO training sources are generated per arch (toolchain-pgo-sources.sh).
archive.mk writes an archive under a temporary name and renames it when complete, so an interrupted build never leaves a truncated file under the name a producer takes as finished.
Boolean helpers in macros.mk (is_true, is_false, bool, assert_bool), which the producers read their knobs with, documented in Boolean values.
Docs: Building and publishing an overlay, the producers in makefile-system.md, and the changes log.

After review:

native/rustc-1.98 is in, with the gcc decision of overlay.mk (OVERLAY_GCC, OVERLAY_GCC_VERS, TC_OVERLAY_GCC, OVERLAY_GCC_ON), off for the tree (OVERLAY_GCC ?= 0): the producer turns it on for itself. Packages keep the vendor gcc and the same rust.
spksrc.native-toolchain.mk reads top-down: dist-arch.mk and the knobs under the header, spksrc.common.mk before the toolchain is read, the component once it is, the recipes, and spksrc.native-cc.mk last.
PKG_NAME picks the component (toolchain-gcc.mk, toolchain-binutils.mk, toolchain-rustc.mk); NATIVE_TOOLCHAIN goes. dist-arch.mk moves to mk/spksrc.rules/, shared by producers and overlays; toolchain_real and toolchain_archs move to spksrc.common/archs.mk.
makefile_var (macros.mk) reads one variable a Makefile declares, used by the producers, tc-capability.mk, spksrc.toolchain.mk and the overlays.
export LC_ALL := C in spksrc.common.mk, for every build: the vendor gcc of DSM 5.2 aborts under any other locale. A $(shell) running a vendor tool keeps it inline, as make 4.3 (the CI image) does not export to $(shell).
The gcc PGO also trains f951 on a generated train.f90: a Fortran LAPACK build spends a fifth of its time in it. train.c and train.cpp stay byte-identical.
PGO of gmp/mpfr/mpc/isl was measured and left out: they take under 0.1% of a build (scipy, OpenBLAS).

Tests

make -n of native/gcc-8.5, native/binutils-2.30 and native/rustc-1.82 for arch-qoriq-6.2.4 parse cleanly; native/gcc-8.5 all-5.2 finds its 3 archs in overlay/.
native/binutils-2.30 TOOLCHAIN_PGO=0 arch-x86-5.2 builds binutils-2.30-x86_64-pc-linux-gnu-x86-5.2-v4.txz, the name overlay/syno-x86-5.2_binutils-2.30 downloads.
After review: the four producers parse, all-6.2.4 finds 28, 28, 2 and 2 archs, x64-7.1 builds as apollolake-7.1; rustc-1.98 targets gcc-8.5 and the archive its overlay downloads, rustc-1.82 the vendor gcc.
LC_ALL: under a fr_CA.UTF-8 host locale, zlib x86-5.2 (cmake, gcc 4.7.3), file (autotools), liblc3 (meson) and fd (cargo) build.
train.f90 compiles with the overlay's gfortran 8.5 (5 s at -O2, 8 s at -O3).

Checklist

Build rule all-supported completed successfully
New installation of package completed successfully
Package upgrade completed successfully (Manually install the package again)
Package functionality was tested
Any needed documentation is updated/created

Type of change

🤖 Generated with Claude Code

https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

The producers of the archives the overlay/ directories download since SynoCommunity#7701, out of SynoCommunity#7391: native/gcc-8.5 with gmp, mpfr, mpc and isl, native/binutils-2.30 (v4) and native/rustc-1.82 on mk/spksrc.native-toolchain.mk and spksrc.native/toolchain-<component>.mk: arch-<arch>-<dsm>, all-<dsm>, host optimisation flags, profile-guided builds (TOOLCHAIN_PGO), TOOLCHAIN_JOBS, a generic arch built as the real arch whose archive it fetches. archive.mk writes an archive under a temporary name, then renames it; macros.mk gains the boolean helpers the producers read their knobs with. Nothing a package builds changes. native/rustc-1.98, built with gcc 8.5, stays in SynoCommunity#7391 with the switch that uses that gcc. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

The tc_vars section showed TC_CC, TC_AR and the like in tc_vars.mk, which holds the toolchain's identity and versions only: the tools and flags are in tc_vars.autotools.mk and tc_vars.flags.mk, and tc_vars.rust.mk carries the whole cargo and rustup environment. Each file is now shown as generated, for cross/zlib on qoriq-6.2.4, with who reads it; and for CMake and Meson, the per-dependency toolchain and cross files built at configure time from tc_vars.cmake and tc_vars.meson-cross plus the package's flags. Adding New Toolchain Support points at Synology's toolchain archive, the version map and download URL, and shows a toolchain Makefile and digests as they are. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

th0ma7 · 2026-11-03T00:47:51Z

+# Prefix for running a binary of that toolchain: the C locale. The gcc 4.7.3 of x86-5.2
+# (and x64-5.2) aborts on a current host's locale data -- "loadlocale.c:129:
+# _nl_intern_locale_data: Assertion ... failed" -- on -v, -print-sysroot and every compile.
+# Inline, as toolchain-rust.mk does for x.py: an export reaches $(shell) only from make 4.4.
+TC_VENDOR_ENV = LC_ALL=C


Why not setting export LC_ALL := C applicable throughout the framework by default?
Repeated again in spksrc.native/toolchain-rust.mk

th0ma7 · 2026-11-03T00:49:23Z

+ARCHIVE_TARGET = toolchain-pgo-no-archive
 endif

 include ../../mk/spksrc.native-cc.mk


Shouldn't this include be at the absolute end? only keeping what's mandatory above?

th0ma7 · 2026-11-03T00:51:28Z

-# Generic native-toolchain front-end dispatches to mk/spksrc.native/toolchain-binutils.mk.
+# Builds the archive the binutils overlay installs: mk/spksrc.native/toolchain-binutils.mk.
+#   make arch-ppc853x-5.2        make all-6.2.4        make TOOLCHAIN_PGO=0 arch-x86-5.2
 NATIVE_TOOLCHAIN = binutils


why not simply refer to PKG_NAME instead of adding yet another variable? likewise for gcc & rust?

th0ma7 · 2026-11-03T00:51:52Z

+
+# Builds the archive the gcc overlay installs: mk/spksrc.native/toolchain-gcc.mk.
+#   make arch-qoriq-6.2.4        make all-6.2.4        make TOOLCHAIN_PGO=0 arch-x86-5.2
+NATIVE_TOOLCHAIN = gcc


same comment PKG_NAME

th0ma7 · 2026-11-03T00:53:29Z

@@ -1,2 +1,2 @@
 PKG_NAME = rustc
 PKG_VERS = 1.82.0


rust 1.98 is missing and need to be imported from PR #7391

th0ma7 · 2026-11-03T00:54:34Z

@@ -53,31 +53,3 @@
 # Generic native-toolchain front-end dispatches to mk/spksrc.native/toolchain-rust.mk.
 NATIVE_TOOLCHAIN = rust


PKG_NAME could be renamed to rust to accommodate NATIVE_TOOLCHAIN and allowing to remove that variable (or the reverse to accommodate rustc thus renaming the later). similar to previous comments.

th0ma7 · 2026-11-03T01:00:18Z

there already exist a macro $(call tc ... why not reuse that?

th0ma7 · 2026-11-03T01:00:53Z

 #
 # Provides to the component mk and the package:
 #   TC_DIR       toolchain/syno-<arch>-<vers>
 #   _tc_get      $(call _tc_get,VAR) -> VAR as declared in the toolchain Makefile


same as above $(call tc ...

th0ma7 · 2026-11-03T01:01:27Z

@@ -25,6 +25,17 @@
 #   TC_EXTRACT_DIR  where tc-install lands the gcc toolchain (bin/, sysroot, ...)
 #   TC_SYSROOT_DIR  the extracted toolchain's sysroot (from the declared TC_SYSROOT)
 #   tc-install   ensures the gcc toolchain (hence its sysroot) is extracted


why? there already is status-cookies for that

th0ma7 · 2026-11-03T01:08:01Z

+# The arch the archive is named after: TC_ARCH, or the real arch whose toolchain a generic
+# one shares -- the same answer its consumer gets, so the published name is the fetched one.
+include ../../mk/spksrc.overlay/dist-arch.mk


dist-arch.mk serves two purposes: native produce and overlay downloaders.
The only mk level-2 files being included directly falls under the spksrc.rules directory.
Move from overlay to rules.

Also, normally these rule files are included as early as possible in the mk file. move it up right below the header.

th0ma7 · 2026-11-03T01:10:09Z

+# Component-specific logic: configuration, build/install targets, archive variables.
+include ../../mk/spksrc.native/toolchain-$(NATIVE_TOOLCHAIN).mk


likewise this should be moved up under the header. And using PKG_NAME instead of yet another variable such as $(NATIVE_TOOLCHAIN).

th0ma7 · 2026-11-03T01:12:47Z

+# One (arch, DSM), in the sub-make that knows it, hence its archive name: nothing to do when
+# that archive exists -- an underlying arch already built for another generic one, or an
+# all-<vers> resumed after a failure. Remove the archive to rebuild.
+_TC_SELF = TC_ARCH=$(TC_ARCH) TC_VERS=$(TC_VERS)


I get it, it reduces the code repetition for a bit. But this also makes the code complex for no reason. Avoid using _TC_SELF and use TC_ARCH=$(TC_ARCH) TC_VERS=$(TC_VERS) instead.

th0ma7 · 2026-11-03T01:14:11Z

+# The toolchains a component is built for are its consumers (TOOLCHAIN_CONSUMER), each
+# brought back to the arch it builds as (toolchain_real above), once: x64-6.2.4 and
+# x86-6.2.4 are one build of x86-6.2.4, x64-7.1 is apollolake-7.1. An arch whose archive
+# exists is skipped (toolchain-arch), so a batch resumes where it stopped.
+#
+# TOOLCHAIN_KEEP_WORK = 0 drops each work dir once its archive exists; a failed one is
+# kept, so a broken arch stays debuggable.
+#
+# Concurrency via xargs -P, deliberately NOT make's -j: -j goes into MAKEFLAGS and
+# propagates into every sub-make, parallelising toolchain targets not written for it.
+TOOLCHAIN_JOBS      ?= 1
+TOOLCHAIN_KEEP_WORK ?= 1


This should be near the top, close to the header and not hidden down below in the code between recipes.

th0ma7 · 2026-11-03T01:15:42Z

+
+toolchain_archs = $(sort $(foreach d,$(wildcard $(BASEDIR)/overlay/syno-*-$(1)_$(TOOLCHAIN_CONSUMER)),\
+                    $(patsubst %-$(1),%,$(call toolchain_real,$(patsubst syno-%,%,$(firstword $(subst _, ,$(notdir $(d)))))))))
+


don't we have something similar part of spksrc.common.mk (sub-loading spksrc.common/archs.mk)

if not, please avoid hiding variables intertwined between recipes. If needed, this should be around the toolchain_real declaration. ... unless both toolchain_arch and toolchain_real can be moved to spksrc.common/archs.mk and loaded thru spksrc.common.mk?

Just noticed toolchain-rust.mk does include ../../mk/spksrc.common/archs.mk ... method between native toolchain building should be similar throughout.

th0ma7 · 2026-11-03T01:34:00Z

+CONFIGURE_ARGS += --disable-werror
+CONFIGURE_ARGS += --disable-libsanitizer
+
+COMPILE_ARGS  = MAKEINFO=missing


I recall adding to native cmake CONFIGURE_ARGS += -DCMAKE_INSTALL_RPATH='$$ORIGIN/../lib' so binaries can be relocatable. Is this something that should be consider throughout the binutils, gcc, rustc native builds?

- spksrc.native-toolchain.mk reads top-down: dist-arch.mk and the knobs under the header, spksrc.common.mk before the toolchain is read, the component once it is, the recipes, and spksrc.native-cc.mk last. No _TC_SELF. - PKG_NAME picks the component (toolchain-<PKG_NAME>.mk): NATIVE_TOOLCHAIN goes, and toolchain-rust.mk becomes toolchain-rustc.mk, as native/rustc-*. - dist-arch.mk serves producers and overlays: it moves to spksrc.rules/. - toolchain_real and toolchain_archs join spksrc.common/archs.mk, which every component now gets from spksrc.common.mk; toolchain-rustc.mk no longer includes it itself. - makefile_var (macros.mk) reads one variable a Makefile declares: the producers, tc-capability.mk, spksrc.toolchain.mk and the overlays use it. - native/rustc-1.98 comes from SynoCommunity#7391 with the gcc decision of overlay.mk (OVERLAY_GCC, OVERLAY_GCC_VERS, TC_OVERLAY_GCC, OVERLAY_GCC_ON), off for the tree: the producer turns it on for itself. Packages are unchanged. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

spksrc.common.mk exports LC_ALL=C: the vendor gcc of DSM 5.2 aborts under any other locale, which made those builds depend on the caller's shell. A $(shell) running a vendor tool keeps it inline (TC_VENDOR_ENV), as make 4.3 does not export to $(shell); the inline one of the x.py recipe goes. Checked under a fr_CA.UTF-8 host locale: zlib x86-5.2 (cmake, gcc 4.7.3), file (autotools), liblc3 (meson), fd (cargo). tc-install's comment says why it is not a DEPENDS. The docs follow the review: OVERLAY_GCC and OVERLAY_GCC_VERS in the switches, rustc-1.98, toolchain-$(PKG_NAME).mk, spksrc.rules/dist-arch.mk. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

A Fortran LAPACK build spends a fifth of its time in f951 (OpenBLAS 0.3.30, perf: f951 21%, cc1 49%), which train.c and train.cpp never ran. train.f90, generated beside them, is the numerical Fortran such builds compile: loops over real(8) and complex(8) arrays, folded constants, intrinsics, modules and derived types. gcc-pgo-train compiles it at -O2 and -O3; train.c and train.cpp stay byte-identical, so C and C++ train as before. It compiles with the gfortran 8.5 of the overlay, in 5 and 8 s. For the next rebuild of the gcc overlay. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

- macros.mk includes spksrc.common/macros/{version,bool,list,depend,tools,archs,runlog}.mk; every macro documented the same way, define for a multi-line body, _ for internal names. - makefile_var -> read_makefile_var, dedup-files -> dedup_files. - overlay_dist_arch, toolchain_real and toolchain_archs move to macros/archs.mk; spksrc.rules/dist-arch.mk goes. - overlay.mk reads spksrc.common/overlay/{binutils,gcc,rustc,warnings}.mk in that order; the toolchain side moves to spksrc.toolchain/overlay/. - The producer components move to spksrc.native-toolchain/. - toolchain.md: the overlay archives in the caching section. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

- spksrc.common/macros/string.mk: upper and lower, pure make. - TRUE / FALSE replace the internal _TRUE_VALUES / _FALSE_VALUES, in lower and upper case. - The Rust triple suffixes of tc_vars.rust.mk (RUST_TARGET_UENV / _LENV) and kernel-type-lc use them instead of tr in a shell. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

…ions The arch-group patch directory (patch.mk) and DESCRIPTION_<LANG> (spksrc.spk.mk) took their case from tr in a shell, two forks per group or language; same names, no fork. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

th0ma7 added 2 commits November 2, 2026 12:43

docs: the changes log names SynoCommunity#7707

67d8ad8

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VkzQnRy1W48Vg2QGRbwLSd

th0ma7 self-assigned this Nov 3, 2026

th0ma7 commented Nov 3, 2026

View reviewed changes

Comment thread mk/spksrc.native/toolchain-pgo-sources.sh Resolved

th0ma7 commented Nov 3, 2026

View reviewed changes

th0ma7 added 6 commits November 3, 2026 21:47

		@@ -53,31 +53,3 @@
		# Generic native-toolchain front-end dispatches to mk/spksrc.native/toolchain-rust.mk.
		NATIVE_TOOLCHAIN = rust

		# Component-specific logic: configuration, build/install targets, archive variables.
		include ../../mk/spksrc.native/toolchain-$(NATIVE_TOOLCHAIN).mk


		toolchain_archs = $(sort $(foreach d,$(wildcard $(BASEDIR)/overlay/syno-*-$(1)_$(TOOLCHAIN_CONSUMER)),\
		$(patsubst %-$(1),%,$(call toolchain_real,$(patsubst syno-%,%,$(firstword $(subst _, ,$(notdir $(d)))))))))

Conversation

th0ma7 commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests

Checklist

Type of change

Uh oh!

th0ma7 Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

th0ma7 Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

th0ma7 Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

th0ma7 Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

th0ma7 Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

th0ma7 commented Nov 2, 2026 •

edited

Loading

th0ma7 Nov 3, 2026 •

edited

Loading

th0ma7 Nov 3, 2026 •

edited

Loading

th0ma7 Nov 3, 2026 •

edited

Loading

th0ma7 Nov 3, 2026 •

edited

Loading

th0ma7 Nov 3, 2026 •

edited

Loading