Building hldays #12: Adding a new country

hldays covers 6 countries today. Portugal was researched by hand, down to all 308 municipalities. The other five (Brazil, Germany, the UK, Greece and the US) were imported, and Brazil was the most recent.

This post walks through what adding a country actually involves, using the Brazil import as the example. The short version: the national and state layers take about a minute of script time plus reading the reports. The municipal layer is the real work.

What "a country" is in hldays

A country is a folder:

data/<CC>/
  country.yaml        # country metadata (weekend days) + national holidays
  subdivisions.yaml   # regions / districts (ISO 3166-2 ids)
  municipalities.yaml # municipalities (official statistical codes)

Nothing else needs to change. The build validates the YAML, and discovery (/, /v1/countries, /openapi.json) picks the new country up from the data. Coverage stats are computed too, so they can't go stale:

curl "https://api.hldays.dev/v1/countries/BR"

{
  "code": "BR", "name": "Brazil", "weekend": [6, 7],
  "levels": ["country", "subdivision", "municipality"],
  "coverage": {
    "levels": {
      "country": { "jurisdictions": 1, "with_holidays": 1, "verified": 0 },
      "subdivision": { "jurisdictions": 27, "with_holidays": 27, "verified": 0 },
      "municipality": { "jurisdictions": 1, "with_holidays": 1, "verified": 0 }
    },
    "holiday_rules": 71, "verified_rules": 0
  }
}

That verified_rules: 0 is deliberate: imported records are unverified until someone checks them against official sources.

Starting from python-holidays

Writing Brazil's 27 state calendars by hand would take weeks. python-holidays (MIT) already has them, maintained by people who read the laws.

The catch: python-holidays is code, not data. It gives you dates for a year, not rules. hldays stores rules (fixed, easter, nth_weekday...) so the Worker can compute any year without tables. So scripts/import-country.py works backwards, from dates to rules:

# Rules are inferred from the dates: for each holiday the longest run of years one rule explains (fixed, easter,
# orthodox_easter, nth_weekday ± offset) becomes an entry; changes become valid_from/valid_to segments; leftovers `dates`.

For one date, candidates() lists every rule that would produce it, in order of preference:

def candidates(d):
    """Rules that give date d in its year, in order of preference."""
    y = d.year
    out = [("fixed", (d.month, d.day))]
    for kind, method in (("easter", EASTER_WESTERN), ("orthodox_easter", EASTER_ORTHODOX)):
        off = (d - easter(y, method)).days
        if abs(off) <= 120:
            out.append((kind, off))
    ...

Then segments() takes the first year, tries each candidate, and keeps the one that explains the longest run of consecutive years. When a rule stops working (a law changed), a new segment starts. One-offs that no rule explains, like the UK's 2022 Platinum Jubilee, end up as explicit dates. That's how Greek Labour Day became four entries, which I covered in "Observed holidays and edge cases".

The COUNTRIES config

Each country is one entry in a dictionary at the top of the importer. This is Brazil's:

"BR": {
    "name": "Brazil", "aliases": ["Brasil"], "module": "brazil",
    "langs": [("pt", "pt_BR"), ("en", "en_US")], "kind": "state", "kinds": {"DF": "federal_district"},
    # "Ponto facultativo" (Carnival, Ash Wednesday, Corpus Christi, …): offices may close, not a day off by law.
    "categories": {"public": "public", "optional": "optional"},
    "municipalities": {"São Paulo Capital": {"id": "BR-M-3550308", "name": "São Paulo", "parent": "SP",
                                             "codes": {"ibge": "3550308"}}},
    ...
},

A few things worth pointing out:

langs pulls names in Portuguese and English, so name is {"pt": "Dia do Evangélico", "en": "Evangelical Day"}.
categories maps python-holidays categories to hldays types. Brazil's "ponto facultativo" days are real (public offices close), but they aren't a day off by law, so they come in as optional. Business-day maths ignores them by default.
municipalities handles python-holidays subdivisions that are really cities. "São Paulo Capital" becomes a municipality with its IBGE code, under the state of São Paulo.

The generated YAML says at the top not to edit it by hand. The importer overwrites data/BR/ every run, so fixes go in this config or in the importer.

The US needed a different model

For most countries, if a state doesn't observe a national holiday, the importer drops it for that state with excludes. The US broke that. Federal holidays close federal offices and banks everywhere, even in states whose own offices stay open. So the US entry has two extra flags:

# Federal holidays close banks (Federal Reserve) and federal offices in every state, so states never exclude
# them; python-holidays' state lists are state-government closures, imported as `local`.
"keep_inherited": True, "subdiv_type": "local",

Federal = public everywhere, state = local. If you want the private-sector view, ask for types=public. The trade-off is documented in docs/data-review-import.md: California state offices work on Columbus Day, and this model doesn't show that.

Check 1: does the engine agree with the source?

Inferring rules can go wrong quietly. So after writing the YAML, the importer rebuilds the data and runs the real engine through scripts/dump-holidays.ts, which prints every jurisdiction's dates for every year:

for (const j of cal.list({ country: cc })) {
  out[j.id] = {};
  for (let y = Number(from); y <= Number(to); y++)
    out[j.id][y] = [...new Set(cal.holidays(j, y, types.split(",") as HolidayType[]).map((o) => o.date))].sort();
}

The Python side compares that, year by year from 2016 to 2099, with what python-holidays says, substitute days included. The self-check result goes into research/import/<CC>.json, and the goal is an empty one. For Brazil, Germany, the UK and Greece it is empty. For the US there are 5 jurisdictions with differences, all the same kind: two holidays fighting over one substitute day.

"US-WV": { "missing": [], "extra": ["2026-06-18", "2037-06-18", "2043-06-18", ...] }

West Virginia Day on a Saturday moves back to Friday, which is Juneteenth, so the engine moves it on to Thursday. python-holidays lets both share the Friday. I left the engine's behaviour and wrote it down as an open item.

Check 2: do other datasets agree?

Matching python-holidays only proves the import is faithful. It doesn't prove python-holidays is right. So the importer also pulls weekday dates for 2020 to 2030 from Nager.Date (MIT) and OpenHolidays (CC BY 4.0) and diffs them. Only weekdays, because a disagreement about a Sunday doesn't change any business-day answer.

Each source has quirks, and the importer has comments for the ones I ran into:

# Nager sometimes lists a holiday of every subdivision per county instead of as global.
counties = None if h["counties"] and set(sub_ids) <= set(h["counties"]) else h["counties"]

And a fix that came with Brazil: OpenHolidays only has Brazil's national holidays up to 2025, so later years looked like they were missing everywhere. Now the diff only covers years a source actually has data for:

covered = [yr for yr in CROSS if any(yr in by_year for by_year in data.values())]  # OpenHolidays BR stops at 2025

Then I read every difference. A few examples from the reports:

Brazil, Corpus Christi. Nager.Date lists it as a national public holiday. Federally it's only a ponto facultativo; municipalities declare it (São Paulo city does). hldays has it as optional nationally and public in São Paulo.
Scotland, 4 January 2021 and 2027. We list it as the substitute for 2 January; Nager.Date doesn't. gov.uk listed it for 2021, so Nager.Date is wrong here.
Germany. Zero differences with Nager.Date. One with OpenHolidays: it lists Assumption Day for all of Bavaria, when it only applies in municipalities with a Catholic majority.

Each conclusion goes into docs/data-review-import.md. That file is the real output of an import: the YAML is the data, the review doc is what I know is uncertain about it.

What the Brazil commit added

Brazil wasn't only data. The commit "Import Brazil; holiday move rule; python-holidays categories" touched the engine too:

A new move field on holidays, because Acre moves some Tuesday to Thursday holidays to Friday. That needed types.ts, calendar.ts, validate.ts and a test, the same four places every new rule shape goes.
Category support in the importer (for optional).
Move inference with a guard, so a one-off moved holiday doesn't become a permanent rule.
Cross-checks limited to covered years.

So the real answer to "what does a new country take" is: sometimes a new rule shape. Both import commits so far changed the engine: the first one taught substitute days to skip inherited holidays and added excludes, the Brazil one added move.

Names collide

A new country brings new names, and some of them collide. "São Paulo" is both a state (BR-SP) and a city (BR-M-3550308). Name lookup prefers the most specific level:

const candidates = (this.bySlug.get(slugify(ref)) ?? []).filter(
  (j) => !country || j.country === country.toUpperCase(),
);
return candidates.sort((a, b) => LEVEL_RANK[a.level] - LEVEL_RANK[b.level])[0];

So jurisdiction=São Paulo gets you the city, the same way lisboa gets you the municipality of Lisbon and not the district. For the state, use the id. Names get their own post: "Names are harder than dates".

The part you can't import

Everything above covers national and state holidays. The thing that makes hldays different, the municipal layer, isn't in python-holidays, and for most countries it isn't in any dataset.

Brazil: Lei 9.093/1995 lets each of the 5,570 municipalities declare up to four religious holidays. Only São Paulo city is in so far.
Germany: Assumption Day in Bavaria applies in about 1,700 of 2,056 municipalities. Only Augsburg is in, with its Peace Festival on 8 August.

That's the paid layer, and it has to be researched the way Portugal was: one official source per municipality, with a quote, as I described in "A source on every record". It's the slow part, and it's why the roadmap lists municipal layers separately from new countries.

The checklist

For the next country, the steps are:

Add an entry to COUNTRIES in scripts/import-country.py.
Run the importer (about a minute): .venv/bin/python scripts/import-country.py BR DE GB GR US XX.
Check the self-check is empty, or explain every difference.
Read the cross-check and write the open items into docs/data-review-import.md.
If the country has a rule the engine can't express, add it to types.ts, dates.ts, validate.ts and a test.
npm test, push. Cloudflare deploys it and the country shows up in discovery.
Then, separately, research the municipal layer.

So who's number 7?

The roadmap says Turkey and the UAE are next. Both need something new: Islamic holidays, where the actual date is announced each year and can differ by a day from the calculated one. The engine already computes Islamic dates with Intl, but a public holiday needs the announced date, so those countries will need yearly corrections on top.

It isn't decided, though. Turkey and the UAE are the roadmap answer. Italy is the tempting one: every comune has its patron saint's day, which makes it a municipal layer like Portugal's, and that's the part hldays sells.

If you could pick the 7th country, which would it be, and do you know where its local holidays are published? Tell me in the comments.

Next in the series: lessons learned so far.

hldays.dev · Docs