Document multilingual audit CI operations
This commit is contained in:
142
docs/CI_MULTILINGUAL_AUDIT.md
Normal file
142
docs/CI_MULTILINGUAL_AUDIT.md
Normal file
@@ -0,0 +1,142 @@
|
||||
# CI Multilingual Audit
|
||||
|
||||
## Purpose
|
||||
The multilingual audit verifies that public content stays locale-correct across all active MandelBlog languages after deploy.
|
||||
|
||||
It checks rendered, user-facing text for:
|
||||
- mixed-language fragments
|
||||
- foreign UI labels
|
||||
- weak or generic badge labels flagged by policy
|
||||
- locale-specific normalization problems
|
||||
|
||||
It does not modify content in CI. It only audits and reports.
|
||||
|
||||
## Jenkins jobs
|
||||
|
||||
### Main pipeline: `mandelstudio`
|
||||
The main pipeline runs a post-deploy multilingual audit after staging deployment completes.
|
||||
|
||||
Stages relevant to multilingual quality:
|
||||
1. `Deploy Staging`
|
||||
2. `Post-Deploy Multilingual Audit`
|
||||
|
||||
The audit stage runs remotely on staging:
|
||||
```bash
|
||||
python manage.py audit_locales --format=json
|
||||
```
|
||||
|
||||
Artifact archived:
|
||||
- `artifacts/multilingual-audit.json`
|
||||
|
||||
### Nightly pipeline
|
||||
Pipeline source:
|
||||
- `Jenkinsfile.multilingual-nightly`
|
||||
|
||||
Schedule:
|
||||
- `H 2 * * *`
|
||||
|
||||
The nightly job:
|
||||
- runs the full multilingual audit on staging
|
||||
- archives the latest JSON artifact
|
||||
- compares the current artifact against the previous artifact
|
||||
- prints regressions by locale
|
||||
|
||||
## Build result policy
|
||||
The audit summary is interpreted as follows:
|
||||
|
||||
- `SUCCESS`
|
||||
- all locales have `block=0` and `warn=0`
|
||||
- `UNSTABLE`
|
||||
- at least one locale has `warn > 0`
|
||||
- deploy is not blocked
|
||||
- `FAILED`
|
||||
- at least one locale has `block > 0`
|
||||
- or audit execution itself fails
|
||||
|
||||
This keeps deploys safe without making warning-level cleanup a hard blocker.
|
||||
|
||||
## Required Jenkins credential
|
||||
Credential location:
|
||||
- `Manage Jenkins -> Credentials -> System -> Global credentials`
|
||||
|
||||
Credential to add:
|
||||
- `Kind`: `SSH Username with private key`
|
||||
- `ID`: `staging-root-ssh`
|
||||
- `Username`: `root`
|
||||
- `Private key`: staging SSH key
|
||||
|
||||
Current implementation uses the following environment defaults:
|
||||
- `STAGING_AUDIT_HOST=root@49.12.204.96`
|
||||
- `STAGING_AUDIT_PROJECT_DIR=/home/www-mandelstudio/mandelstudio`
|
||||
- `STAGING_AUDIT_MANAGE=/var/lib/virtualenv/mandelstudio/bin/manage.py`
|
||||
|
||||
## Console summary
|
||||
The Jenkins stage prints a per-locale summary like this:
|
||||
```text
|
||||
LOCALE en: issues_found=0 issues_remaining=0 block=0 warn=0 log=0
|
||||
```
|
||||
|
||||
Nightly runs also print regressions when present:
|
||||
```text
|
||||
REGRESSIONS:
|
||||
- es: remaining=+2 block=+0 warn=+2 log=+0
|
||||
```
|
||||
|
||||
If no regressions exist:
|
||||
```text
|
||||
REGRESSIONS: none
|
||||
```
|
||||
|
||||
## Artifact structure
|
||||
Archived file:
|
||||
- `artifacts/multilingual-audit.json`
|
||||
|
||||
Expected clean structure:
|
||||
- `run_id`
|
||||
- `total_urls_checked`
|
||||
- `issues_found`
|
||||
- `summary`
|
||||
- `issues`
|
||||
|
||||
Failure artifacts may also contain:
|
||||
- `error`
|
||||
|
||||
This happens when the remote audit times out or fails, and is intentional so Jenkins still archives a machine-readable result.
|
||||
|
||||
## Local rerun
|
||||
To rerun the same remote audit flow locally:
|
||||
```bash
|
||||
export STAGING_AUDIT_HOST='root@49.12.204.96'
|
||||
export STAGING_AUDIT_PROJECT_DIR='/home/www-mandelstudio/mandelstudio'
|
||||
export STAGING_AUDIT_MANAGE='/var/lib/virtualenv/mandelstudio/bin/manage.py'
|
||||
./scripts/run_remote_multilingual_audit.sh
|
||||
python3 scripts/multilingual_audit_ci.py --json artifacts/multilingual-audit.json
|
||||
```
|
||||
|
||||
To compare against a previous artifact:
|
||||
```bash
|
||||
python3 scripts/multilingual_audit_ci.py \
|
||||
--json artifacts/multilingual-audit.json \
|
||||
--previous-json artifacts/previous-multilingual-audit.json
|
||||
```
|
||||
|
||||
## Fixing issues by locale
|
||||
Recommended response sequence for any locale that returns warnings or blocks:
|
||||
1. run scoped dry-run audit for the affected locale/pages
|
||||
2. inspect before/after rewrite candidates
|
||||
3. apply controlled rewrite only to affected pages
|
||||
4. rerun post-audit
|
||||
5. manually verify rendered output
|
||||
|
||||
Do not bulk rewrite a locale tree without scoped review first.
|
||||
|
||||
## Operational notes
|
||||
- remote audit execution has a timeout
|
||||
- audit failure still produces JSON output for Jenkins archiving
|
||||
- missing previous nightly artifact is handled gracefully
|
||||
|
||||
## Hardening candidates
|
||||
These are operational follow-ups only. They are not required for current behavior.
|
||||
- replace `root` SSH with a dedicated deploy/audit user
|
||||
- move host/path configuration into Jenkins-managed environment variables or folder-level config
|
||||
- document SSH credential rotation procedure
|
||||
Reference in New Issue
Block a user