[Data] br_ibge_censo_demografico 2022 public microdata

[JoaoCarabetta]

Template Pull Requests - Pipeline

Nomeação do Pull Request

• [Data]: Para subida de novos dados em produção.

Draft

Opened as draft until official-lake upload + metadata are done.

Dataset

Slug: br_ibge_censo_demografico (already published; this PR only adds tables)
Tables: microdados_pessoa_2022, microdados_domicilio_2022, microdados_familia_2022, microdados_mortalidade_2022, plus 892 2022 rows appended to the shared dicionario
Coverage: 2022
Source: IBGE Censo Demográfico 2022, microdados de acesso público
https://ftp.ibge.gov.br/Censos/Censo_Demografico_2022/Microdados_e_Areas_de_Ponderacao/Microdados_de_acesso_Publico/csv/

Descrição do PR

Extends the existing 1970–2010 sample-microdata series with the 2022 public file. Four record types (pessoa, domicílio, família, mortalidade). No emigração file in the public zip. No Prefect pipeline (decennial). Coded values (1, 2, 3…) stay STRING in the microdata; labels live in the shared dicionario, same as previous censuses.

Motivação/Contexto: IBGE released the public sample on 2026-08-31. Geography is capped at UF; disclosure-risk filter, 50% subsample of 100% sampling-fraction households, age in 5-year groups.

Detalhes Técnicos

• Staging parquet is all-STRING, hive ano=2022/sigla_uf=<UF>/. Types live in dbt safe_cast.
• Local BigQuery for this run used sandbox-507414 only. Official basedosdados-dev / table-approve is a later step.
• set_datalake_project accepts an optional datalake_project var; skip_post_hooks skips the allUsers GRANT (sandbox org policy blocks it).
• Directory relationships tests are in schema.yml but were not run in sandbox (diretorios are not in that project).
• dicionario unions 1970–2010 rows from prod (id_tabela not like '%_2022') with 2022 staging (id_tabela like '%_2022'). Official staging for dicionario can stay 2022-only; do not replace historical prod.

Sandbox row counts (local = BQ)

Table	Rows
microdados_pessoa_2022	21,538,508
microdados_domicilio_2022	7,689,914
microdados_familia_2022	6,550,107
microdados_mortalidade_2022	430,961
dicionario	3,846 (2,954 historical + 892 from 2022)

Teste e Validações

• Testado localmente (sandbox-507414): dbt run 4/4 microdata + dicionario; dbt test 19/19 unique/not-null on microdata + 2/2 on dicionario. Directory FKs skipped.
• Testado na Cloud (basedosdados-dev / table-approve)

test-dev-model was not applied: staging is not in basedosdados-dev yet.

Riscos e Mitigações

• Riscos: table-approve will fail until staging exists in the official lake. Metadata not registered (Data Basis MCP unavailable this session). Public-file suppression leaves many skip-pattern columns sparse.
• Rollback: revert this PR; dataset stays published with 1970–2010 only.

Dependencias

• Official-lake re-upload with BD credentials (this SA must not write basedosdados-dev)
• Metadata dev → approval → prod (under_review at table level; dataset stays published)

Checklist

• DBT models run successfully in sandbox
• DBT tests pass in sandbox (except directory FKs)
• Metadata registered in dev backend
• Metadata promoted to prod
• Verify at: https://basedosdados.org/dataset/br-ibge-censo-demografico

Made with Cursor

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mergify]

@JoaoCarabetta esse pull request tem conflitos 😩