138 lines
4.0 KiB
Markdown
138 lines
4.0 KiB
Markdown
# CI Multilingual Audit
|
|
|
|
## Purpose
|
|
The multilingual audit verifies that public content stays locale-correct across all active MandelBlog languages after deploy.
|
|
|
|
It checks rendered, user-facing text for:
|
|
- mixed-language fragments
|
|
- foreign UI labels
|
|
- weak or generic badge labels flagged by policy
|
|
- locale-specific normalization problems
|
|
|
|
It does not modify content in CI. It only audits and reports.
|
|
|
|
## Jenkins jobs
|
|
|
|
### Main pipeline: `mandelstudio`
|
|
The main pipeline runs a post-deploy multilingual audit after staging deployment completes.
|
|
|
|
Stages relevant to multilingual quality:
|
|
1. `Deploy Staging`
|
|
2. `Post-Deploy Multilingual Audit`
|
|
|
|
The audit stage runs remotely on staging:
|
|
```bash
|
|
python manage.py audit_locales --format=json
|
|
```
|
|
|
|
Artifact archived:
|
|
- `artifacts/multilingual-audit.json`
|
|
|
|
### Nightly pipeline
|
|
Pipeline source:
|
|
- `Jenkinsfile.multilingual-nightly`
|
|
|
|
Schedule:
|
|
- `H 2 * * *`
|
|
|
|
The nightly job:
|
|
- runs the full multilingual audit on staging
|
|
- archives the latest JSON artifact
|
|
- compares the current artifact against the previous artifact
|
|
- prints regressions by locale
|
|
|
|
## Build result policy
|
|
The audit summary is interpreted as follows:
|
|
|
|
- `SUCCESS`
|
|
- all locales have `block=0` and `warn=0`
|
|
- `UNSTABLE`
|
|
- at least one locale has `warn > 0`
|
|
- deploy is not blocked
|
|
- `FAILED`
|
|
- at least one locale has `block > 0`
|
|
- or audit execution itself fails
|
|
|
|
This keeps deploys safe without making warning-level cleanup a hard blocker.
|
|
|
|
## Jenkins requirements
|
|
No dedicated staging SSH credential is required for the multilingual audit stage.
|
|
|
|
The audit runs through `/srv/apps/mandel-dashboard/bin/deploy_stg_from_jenkins.py --command`, using the same sudo-whitelisted deployment entrypoint as staging deployment.
|
|
|
|
Current implementation uses the following environment defaults:
|
|
- `STAGING_AUDIT_PROJECT_NAME=mandelstudio`
|
|
- `STAGING_AUDIT_PROJECT_DIR=/home/www-mandelstudio/mandelstudio`
|
|
- `STAGING_AUDIT_MANAGE=/var/lib/virtualenv/mandelstudio/bin/manage.py`
|
|
|
|
## Console summary
|
|
The Jenkins stage prints a per-locale summary like this:
|
|
```text
|
|
LOCALE en: issues_found=0 issues_remaining=0 block=0 warn=0 log=0
|
|
```
|
|
|
|
Nightly runs also print regressions when present:
|
|
```text
|
|
REGRESSIONS:
|
|
- es: remaining=+2 block=+0 warn=+2 log=+0
|
|
```
|
|
|
|
If no regressions exist:
|
|
```text
|
|
REGRESSIONS: none
|
|
```
|
|
|
|
## Artifact structure
|
|
Archived file:
|
|
- `artifacts/multilingual-audit.json`
|
|
|
|
Expected clean structure:
|
|
- `run_id`
|
|
- `total_urls_checked`
|
|
- `issues_found`
|
|
- `summary`
|
|
- `issues`
|
|
|
|
Failure artifacts may also contain:
|
|
- `error`
|
|
|
|
This happens when the remote audit times out or fails, and is intentional so Jenkins still archives a machine-readable result.
|
|
|
|
## Local rerun
|
|
To rerun the same remote audit flow locally:
|
|
```bash
|
|
export STAGING_AUDIT_PROJECT_NAME='mandelstudio'
|
|
export STAGING_AUDIT_PROJECT_DIR='/home/www-mandelstudio/mandelstudio'
|
|
export STAGING_AUDIT_MANAGE='/var/lib/virtualenv/mandelstudio/bin/manage.py'
|
|
./scripts/run_remote_multilingual_audit.sh
|
|
python3 scripts/multilingual_audit_ci.py --json artifacts/multilingual-audit.json
|
|
```
|
|
|
|
To compare against a previous artifact:
|
|
```bash
|
|
python3 scripts/multilingual_audit_ci.py \
|
|
--json artifacts/multilingual-audit.json \
|
|
--previous-json artifacts/previous-multilingual-audit.json
|
|
```
|
|
|
|
## Fixing issues by locale
|
|
Recommended response sequence for any locale that returns warnings or blocks:
|
|
1. run scoped dry-run audit for the affected locale/pages
|
|
2. inspect before/after rewrite candidates
|
|
3. apply controlled rewrite only to affected pages
|
|
4. rerun post-audit
|
|
5. manually verify rendered output
|
|
|
|
Do not bulk rewrite a locale tree without scoped review first.
|
|
|
|
## Operational notes
|
|
- remote audit execution has a timeout
|
|
- audit failure still produces JSON output for Jenkins archiving
|
|
- missing previous nightly artifact is handled gracefully
|
|
|
|
## Hardening candidates
|
|
These are operational follow-ups only. They are not required for current behavior.
|
|
- replace `root` SSH with a dedicated deploy/audit user
|
|
- move host/path configuration into Jenkins-managed environment variables or folder-level config
|
|
- document SSH credential rotation procedure
|