Data & Code

This page is a small hub for datasets, helper functions, and relics of scripts that turned out to be useful more than once (even more so when your favourite AI assistant is offline). Most things here are lightweight: a mix of Python, Stata, and a few project-specific utilities.

Feel free to use anything that helps. If you see something broken, strange, or improvable, send me a note or ping me on GitHub.

Data sources

Country macro

Space reserved for macro panels (GDP, inflation, fiscal, etc.) once I clean up internal scripts and documentation for public sharing.

Trade

This will eventually host links and notes for cleaned trade datasets (Comtrade-based panels, bilateral flows, etc.).

Development indicators

Placeholder for curated subsets of World Bank, UN, and other development indicators I use in projects.

Financial

Placeholder for Financial datasets

Scripts that might help

A few small pieces of code I keep reaching for. This section will grow over time as I clean up old notebooks and do-files.

Country identifier(s)

Things that map between ISO3, IMF country codes, UN M49, and other flavours of "which country is this?".

Python · country lookup (ISO3 ↔ name)

Quick mapping between ISO codes and country names.

Python

1import pandas as pd
2import pycountry
3import re
4
5# ----- dummy input -----
6df = pd.DataFrame({"ISO3": ["HKG", "GBR", "CHE", "ETH"]})
7
8# ----- helper class -----
9class CountryCodeConverter:
10    def __init__(self):
11        self.alpha3_to_num = {}
12        self.num_to_alpha3 = {}
13        for c in pycountry.countries:
14            if hasattr(c, "alpha_3") and hasattr(c, "numeric"):
15                iso3 = c.alpha_3
16                num = c.numeric.zfill(3)
17                self.alpha3_to_num[iso3] = num
18                self.num_to_alpha3[num] = iso3
19
20    def to_un_numeric(self, iso3):
21        return self.alpha3_to_num.get(str(iso3).upper())
22
23cc = CountryCodeConverter()
24
25# ----- Generate complimentary identifiers -----
26df["ISO2"] = df["ISO3"].apply(
27    lambda x: pycountry.countries.get(alpha_3=x).alpha_2
28    if pycountry.countries.get(alpha_3=x) else None
29)
30df["UN_numeric"] = df["ISO3"].apply(cc.to_un_numeric)
31df["country"] = df["ISO3"].apply(
32    lambda x: pycountry.countries.get(alpha_3=x).name
33    if pycountry.countries.get(alpha_3=x) else None
34)

Datetime conversion

For reshaping quarterly/annual series, building year–quarter indices, and fixing date formats that should have been standard but weren't.

Python · datetime parsing & quarters

Convert dates into year, month, and quarter (using crisis dates as example).

Python

1import pandas as pd
2from datetime import datetime
3
4# ----- sample df -----
5df = pd.DataFrame({
6    "label": ["Dot-com", "GFC", "COVID"],
7    "date_str": ["2000-03-10", "2008-09-15", "2020-03-11"]
8})
9
10# ----- convert to datetime -----
11df["date"] = df["date_str"].apply(
12    lambda x: datetime.strptime(x, "%Y-%m-%d")
13)
14# ----- extract components -----
15df["year"] = df["date"].dt.year
16df["month"] = df["date"].dt.month
17df["quarter"] = df["month"].apply(lambda m: (m - 1) // 3 + 1)
18df["year_quarter"] = df.apply(
19    lambda r: f"{r.year}Q{r.quarter}", axis=1
20)

IMF API

Variable keys can be found on data.imf.org under "Indicator", then “Show Timeseries.” Full API documentation is available in the IMF’s API Knowledge Base.

Python · Access to IMF API

Fetch IMF WEO data in a country–year panel.

Python

1#! pip install sdmx1 # I know this looks strange, but it works
2import sdmx
3import pandas as pd
4
5client = sdmx.Client("IMF_DATA")
6indicators = "+".join([
7                        "NGDP", # GDP, in current price, LCU 
8                        "NGDPD" # GDP, in current price, USD 
9                        ])
10resp = client.data(
11    "WEO",
12    key=f"all.{indicators}.A",
13    params={"startPeriod": 1990, "endPeriod": 2024},
14)
15df = sdmx.to_pandas(resp)
16df = df.rename("value").reset_index() if isinstance(df, pd.Series) else df.reset_index()
17df = df.rename(columns={
18    "COUNTRY": "ISO3",
19    "TIME_PERIOD": "year",
20})

UNCTAD API

UNCTAD’s Data Centre provides API examples in R, but not in Python. Below is a minimal Python example to fetch Unit Value Index (UVI) data. You will need to log in (guest sign-in works, e.g. Google), then go to My Home → My API key to obtain your client_id and client_secret.

Python · Access to UNCTAD API

Fetch UNCTAD Unit Value Index data

Python

1# Fetch UNCTAD Unit Value Index data (2005–2023)
2# !
3client_id = "a8bCC.....8bc" <REVISE TO YOUR CODE>
4client_secret = "jtAjARgGDrLVv......vgCc1C0=" <REVISE TO YOUR CODE>
5
6import requests
7import pandas as pd
8from io import BytesIO
9
10headers = {"ClientId": client_id, "ClientSecret": client_secret}
11years_tot = ",".join(str(y) for y in range(2005, 2025))
12
13ECA_UN = {'012', '024', '072'} # "DZA","AGO","BEN" 
14# use country identifier above to convert if needed
15country_list = ",".join(f"'{c}'" for c in ECA_UN)
16
17# Build filter
18filter_tot = (
19    f"Economy/Code in ({country_list}) "
20    f"and Year in ({years_tot}) "
21    "and Index/Code in ('5','6')"
22)
23params_tot = {
24    "$select": "Economy/Code, Year,Index/Label,Index_Base_2015_Value",
25    "$filter": filter_tot,
26    "$orderby": "Year asc,Index/Order asc",
27    "$compute": "round(M6475/Value div 1, 1) as Index_Base_2015_Value",
28    "$format": "csv",
29    "compress": "gz",
30    "culture": "en"
31}
32url_tot = "https://unctadstat-user-api.unctad.org/US.TermsOfTrade/cur/Facts"
33resp2 = requests.get(url_tot, headers=headers, params=params_tot)
34resp2.raise_for_status()
35with open("terms.csv.gz", "wb") as f:
36    f.write(resp2.content)
37
38df_terms = pd.read_csv(
39    "terms.csv.gz",
40    compression="gzip",
41)
42
43# Reshape to UVI_X and UVI_M (adjust the labels if they differ)
44df_terms_wide = (
45    df_terms
46    .pivot(index=['Year', 'Economy_Code'], columns='Index_Label', values='Index_Base_2015_Value')
47    .reset_index()
48    .rename(columns={
49        'Year': 'year',
50        'Economy_Code': 'UN_numeric',
51        'Unit value index of exports': 'UVI_X',
52        'Unit value index of imports': 'UVI_M'
53    })
54)

Comtrade API

Placeholder for the usual "loop over reporter/partner/product and don't get rate-limited" utilities.