diff --git a/docs/CI_MULTILINGUAL_AUDIT.md b/docs/CI_MULTILINGUAL_AUDIT.md new file mode 100644 index 0000000..2de875a --- /dev/null +++ b/docs/CI_MULTILINGUAL_AUDIT.md @@ -0,0 +1,142 @@ +# CI Multilingual Audit + +## Purpose +The multilingual audit verifies that public content stays locale-correct across all active MandelBlog languages after deploy. + +It checks rendered, user-facing text for: +- mixed-language fragments +- foreign UI labels +- weak or generic badge labels flagged by policy +- locale-specific normalization problems + +It does not modify content in CI. It only audits and reports. + +## Jenkins jobs + +### Main pipeline: `mandelstudio` +The main pipeline runs a post-deploy multilingual audit after staging deployment completes. + +Stages relevant to multilingual quality: +1. `Deploy Staging` +2. `Post-Deploy Multilingual Audit` + +The audit stage runs remotely on staging: +```bash +python manage.py audit_locales --format=json +``` + +Artifact archived: +- `artifacts/multilingual-audit.json` + +### Nightly pipeline +Pipeline source: +- `Jenkinsfile.multilingual-nightly` + +Schedule: +- `H 2 * * *` + +The nightly job: +- runs the full multilingual audit on staging +- archives the latest JSON artifact +- compares the current artifact against the previous artifact +- prints regressions by locale + +## Build result policy +The audit summary is interpreted as follows: + +- `SUCCESS` + - all locales have `block=0` and `warn=0` +- `UNSTABLE` + - at least one locale has `warn > 0` + - deploy is not blocked +- `FAILED` + - at least one locale has `block > 0` + - or audit execution itself fails + +This keeps deploys safe without making warning-level cleanup a hard blocker. + +## Required Jenkins credential +Credential location: +- `Manage Jenkins -> Credentials -> System -> Global credentials` + +Credential to add: +- `Kind`: `SSH Username with private key` +- `ID`: `staging-root-ssh` +- `Username`: `root` +- `Private key`: staging SSH key + +Current implementation uses the following environment defaults: +- `STAGING_AUDIT_HOST=root@49.12.204.96` +- `STAGING_AUDIT_PROJECT_DIR=/home/www-mandelstudio/mandelstudio` +- `STAGING_AUDIT_MANAGE=/var/lib/virtualenv/mandelstudio/bin/manage.py` + +## Console summary +The Jenkins stage prints a per-locale summary like this: +```text +LOCALE en: issues_found=0 issues_remaining=0 block=0 warn=0 log=0 +``` + +Nightly runs also print regressions when present: +```text +REGRESSIONS: +- es: remaining=+2 block=+0 warn=+2 log=+0 +``` + +If no regressions exist: +```text +REGRESSIONS: none +``` + +## Artifact structure +Archived file: +- `artifacts/multilingual-audit.json` + +Expected clean structure: +- `run_id` +- `total_urls_checked` +- `issues_found` +- `summary` +- `issues` + +Failure artifacts may also contain: +- `error` + +This happens when the remote audit times out or fails, and is intentional so Jenkins still archives a machine-readable result. + +## Local rerun +To rerun the same remote audit flow locally: +```bash +export STAGING_AUDIT_HOST='root@49.12.204.96' +export STAGING_AUDIT_PROJECT_DIR='/home/www-mandelstudio/mandelstudio' +export STAGING_AUDIT_MANAGE='/var/lib/virtualenv/mandelstudio/bin/manage.py' +./scripts/run_remote_multilingual_audit.sh +python3 scripts/multilingual_audit_ci.py --json artifacts/multilingual-audit.json +``` + +To compare against a previous artifact: +```bash +python3 scripts/multilingual_audit_ci.py \ + --json artifacts/multilingual-audit.json \ + --previous-json artifacts/previous-multilingual-audit.json +``` + +## Fixing issues by locale +Recommended response sequence for any locale that returns warnings or blocks: +1. run scoped dry-run audit for the affected locale/pages +2. inspect before/after rewrite candidates +3. apply controlled rewrite only to affected pages +4. rerun post-audit +5. manually verify rendered output + +Do not bulk rewrite a locale tree without scoped review first. + +## Operational notes +- remote audit execution has a timeout +- audit failure still produces JSON output for Jenkins archiving +- missing previous nightly artifact is handled gracefully + +## Hardening candidates +These are operational follow-ups only. They are not required for current behavior. +- replace `root` SSH with a dedicated deploy/audit user +- move host/path configuration into Jenkins-managed environment variables or folder-level config +- document SSH credential rotation procedure