mandelstudio/docs/CI_MULTILINGUAL_AUDIT.md

CI Multilingual Audit

Purpose

The multilingual audit verifies that public content stays locale-correct across all active MandelBlog languages after deploy.

It checks rendered, user-facing text for:

• mixed-language fragments
• foreign UI labels
• weak or generic badge labels flagged by policy
• locale-specific normalization problems

It does not modify content in CI. It only audits and reports.

Jenkins jobs

Main pipeline: mandelstudio

The main pipeline runs a post-deploy multilingual audit after staging deployment completes.

Stages relevant to multilingual quality:

1. Deploy Staging
2. Post-Deploy Multilingual Audit

The audit stage runs remotely on staging:

python manage.py audit_locales --format=json

Artifact archived:

• artifacts/multilingual-audit.json

Nightly pipeline

Pipeline source:

• Jenkinsfile.multilingual-nightly

Schedule:

• H 2 * * *

The nightly job:

• runs the full multilingual audit on staging
• archives the latest JSON artifact
• compares the current artifact against the previous artifact
• prints regressions by locale

Build result policy

The audit summary is interpreted as follows:

• SUCCESS
  • all locales have block=0 and warn=0
• UNSTABLE
  • at least one locale has warn > 0
  • deploy is not blocked
• FAILED
  • at least one locale has block > 0
  • or audit execution itself fails

This keeps deploys safe without making warning-level cleanup a hard blocker.

Jenkins requirements

No dedicated staging SSH credential is required for the multilingual audit stage.

The audit runs from the Jenkins built-in node through the same serverpillar / Salt transport used by staging deployment.

Current implementation uses the following environment defaults:

• STAGING_AUDIT_MINION=welkombij.mandelblog.com
• STAGING_AUDIT_PROJECT_DIR=/home/www-mandelstudio/mandelstudio
• STAGING_AUDIT_MANAGE=/var/lib/virtualenv/mandelstudio/bin/manage.py

Console summary

The Jenkins stage prints a per-locale summary like this:

LOCALE en: issues_found=0 issues_remaining=0 block=0 warn=0 log=0

Nightly runs also print regressions when present:

REGRESSIONS:
- es: remaining=+2 block=+0 warn=+2 log=+0

If no regressions exist:

REGRESSIONS: none

Artifact structure

Archived file:

• artifacts/multilingual-audit.json

Expected clean structure:

• run_id
• total_urls_checked
• issues_found
• summary
• issues

Failure artifacts may also contain:

• error

This happens when the remote audit times out or fails, and is intentional so Jenkins still archives a machine-readable result.

Local rerun

To rerun the same remote audit flow locally:

export STAGING_AUDIT_MINION='welkombij.mandelblog.com'
export STAGING_AUDIT_PROJECT_DIR='/home/www-mandelstudio/mandelstudio'
export STAGING_AUDIT_MANAGE='/var/lib/virtualenv/mandelstudio/bin/manage.py'
./scripts/run_remote_multilingual_audit.sh
python3 scripts/multilingual_audit_ci.py --json artifacts/multilingual-audit.json

To compare against a previous artifact:

python3 scripts/multilingual_audit_ci.py \
  --json artifacts/multilingual-audit.json \
  --previous-json artifacts/previous-multilingual-audit.json

Fixing issues by locale

Recommended response sequence for any locale that returns warnings or blocks:

1. run scoped dry-run audit for the affected locale/pages
2. inspect before/after rewrite candidates
3. apply controlled rewrite only to affected pages
4. rerun post-audit
5. manually verify rendered output

Do not bulk rewrite a locale tree without scoped review first.

Operational notes

• remote audit execution has a timeout
• audit failure still produces JSON output for Jenkins archiving
• missing previous nightly artifact is handled gracefully

Hardening candidates

These are operational follow-ups only. They are not required for current behavior.

• replace root SSH with a dedicated deploy/audit user
• move host/path configuration into Jenkins-managed environment variables or folder-level config
• document SSH credential rotation procedure