✨ Simplify GBD metadata with Jinja macros

[Marigold]

Motivation

GBD metadata is currently non-manageable. Even simple fix such as adding exception to cause ballooned into this monster. We need a way how to DRY it.

Solution

Add support for shared.meta.yml where user can add definitions: or macros: and use them across other YAML files.

Use macros instead of raw jinja. Macros are very similar to python functions and can be parametrized, which allowed me to DRY a lot of duplicated code. I used o1 and o3 heavily when refactoring & DRYing those functions. There's certainly more space for a cleanup, but I didn't have enough time to polish it.

Alternatives

At first, I thought that registering custom python functions would be the best solution, but it was overly complex and didn't really solve the problem. One of the issues was that python functions would have to be defined in the grapher step, while YAML metadata would live in a garden step.

GBD metadata

Besides DRYing metadata, I fixed a couple of whitespace issues.

[owidbot]

Quick links (staging server):

Site Dev	Site Preview	Admin	Wizard	Docs

Login: ssh owid@staging-site-jinja-macros

chart-diff: ✅

No charts for review.

data-diff: ✅ No differences found

Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-02-02 08:18:37 UTC
Execution time: 16.13 seconds

[lucasrodes]

we should make sure that this doesn't lead to the behavior here.

[lucasrodes]

I just realized in this data page that there is a mysterious whitespace:

The estimated number of current cases of HIV/AIDS⎵, per 100 people.

I am not sure what is the source for this; I tried looking into etl/steps/data/garden/ihme_gbd/2024-05-20/gbd_prevalence.meta.yml but couldn't pinpoint a particular line.

Unsure if related, but when using ifs, I generally try to avoid unexpected linebreaks with the structure:

<% if ... %>
...
<%- elif ... %>
...
<%- else %>
...
<%- endif %>

[spoonerf]

Looks awesome, thanks Mojmir! There is a still a whitespace issue with HIV/AIDs that Lucas noticed, which I've had a go at getting rid of in a separate PR. but I think it requires a sacrifice to the Jinja gods 😅

Strangely it looks okay in the playground notebook, but will have more of a play tomorrow.