perf: use yaml.CSafeLoader/CDumper for 4x faster virtual index merge
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful

Replace yaml.safe_load / yaml.Dumper with the C extension equivalents
(yaml.CSafeLoader, yaml.CDumper) for all YAML parsing and serialisation
in the virtual repo merge path. A module-level try/except falls back to
the pure-Python SafeLoader/Dumper when libyaml is not available.

Measured on 19-member helm-all (real upstream data):
  Before (SafeLoader + Dumper): merge=38,877ms
  After  (CSafeLoader + CDumper): merge=9,625ms  (4.0x faster)

Local microbenchmark (500 charts × 10 versions × 19 members):
  Before: 40.8s  After: 6.1s  (6.7x faster)
This commit is contained in:
2026-05-02 01:50:21 +10:00
parent 7b6c69b70f
commit b7488fce10
2 changed files with 39 additions and 2 deletions
+30
View File
@@ -14,6 +14,8 @@ from artifactapi.artifact.virtual import (
_merge_helm_indexes,
_rewrite_urls,
_VirtualHandler,
_YamlDumperBase,
_YamlLoader,
)
# ---------------------------------------------------------------------------
@@ -82,6 +84,34 @@ _CFG_A = {"base_url": "https://helm.releases.hashicorp.com", "cache": {"mutable_
_CFG_B = {"base_url": "https://charts.example.com", "cache": {"mutable_ttl": 1800}}
# ---------------------------------------------------------------------------
# _YamlLoader / _YamlDumperBase — C extension selection
# ---------------------------------------------------------------------------
class TestYamlExtensionSelection:
def test_loader_is_a_class(self):
assert isinstance(_YamlLoader, type)
def test_dumper_base_is_a_class(self):
assert isinstance(_YamlDumperBase, type)
def test_helm_dumper_uses_selected_base(self):
assert issubclass(_HelmDumper, _YamlDumperBase)
def test_c_extensions_used_when_available(self):
try:
assert _YamlLoader is yaml.CSafeLoader
assert _YamlDumperBase is yaml.CDumper
except AttributeError:
assert _YamlLoader is yaml.SafeLoader
assert _YamlDumperBase is yaml.Dumper
def test_loader_can_parse_yaml(self):
result = yaml.load(b"key: value", Loader=_YamlLoader)
assert result == {"key": "value"}
# ---------------------------------------------------------------------------
# _HelmDumper — datetime/date YAML serialization
# ---------------------------------------------------------------------------