📊 Update WB poverty projections

[paarriagadap]

The WB team has updated the poverty projections replication package, due to an error we found

[owidbot]

Quick links (staging server):

Site Dev	Site Preview	Admin	Wizard	Docs

Login: ssh owid@staging-site-data-wb-poverty-projections

chart-diff: ✅

• 5/5 reviewed charts
• Modified: 5/5
• New: 0/0
• Rejected: 0
• Data changes: 0
• Metadata changes: 0

data-diff: ✅ No differences found

+ Dataset garden/wb/2025-01-15/poverty_projections
+ + Table poverty_projections
+   + Column fgt0
+   + Column poorpop


Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-01-20 10:07:22 UTC
Execution time: 17.43 seconds

[paarriagadap]

Hi @lucasrodes! There is another weird thing with this update: when I run this new snapshot in the same pipeline as before I get these long numbers as dimensions (dataset), when they should be 2.15, 3.65, 6.85. Even weirder is that if I un-archive the steps from version 2024-12-03 I get the same error (dataset). Do you know if this is due to a change in the libraries we use? I haven't been able to fix it anyway, I ran the steps with --grapher and can't see changes (Maybe it takes longer to reflect changes, @Marigold?).

[Marigold]

@paarriagadap floats can sometimes introduce precision errors. I'm not sure why this started happening, perhaps an update to new pandas types? Anyway, rounding to 2 decimals fixed that (another way would be to use decimal.Decimal, but that would be more complex).

A slightly different question for discussion - does it make sense to create a new version of a dataset if there was no change to code and only snapshot has been corrected? I'd personally just update the snapshot and keep all other channels as they are, but others might have a different opinion.

[paarriagadap]

Thanks @Marigold! I only applied changes to the garden step, thinking that it was enough to fix it.

I created a new version to use Anomalist and also check the new values in Indicator Upgrader. If there is another way to do this by just replacing the snapshot please let me know.

[Marigold]

I created a new version to use Anomalist and also check the new values in Indicator Upgrader. If there is another way to do this by just replacing the snapshot please let me know.

If you're doing all quality checks, then you need to bump the version. If you only wanted to compare charts, it'd be enough to re-run the snapshot and then check chart-diff with Show all charts turned on. You'd see all data or metadata changes side by side.