From 9ea9f4dc705b881fb69ef02b0f0fa84f9734de25 Mon Sep 17 00:00:00 2001 From: larnsce Date: Wed, 9 Sep 2026 14:11:12 +0200 Subject: [PATCH] feat: catalogue from the owdata harvest, with CI to keep it current The data page read a Google Sheet, and washr 1.1.0 removed the only function that wrote to it, so the catalogue had been frozen at the last hand edit (#136). It now reads the CSVs the weekly owdata harvest publishes. What readers see changes. The sheet listed 56 packages and called 50 of them published; the harvest finds 64 and 32 carry a Zenodo DOI. Rather than relabel 19 packages as unpublished, the table lists every package and shows the DOI as a column, so a missing deposit is visible without the package disappearing. The prose says how many have one. The page also gains a variables table, 2798 rows, which is what makes the catalogue searchable at the level people actually ask about. Both tables are DT. The page used knitr::kable, despite the plan text saying DT; 2798 rows need search and paging. The landing page numbers strip read the same sheet CSV. It now reads the harvest too. "Countries and regions" is replaced by "datasets inside them", because no package carries X-schema.org-spatialCoverage yet, so that count came out as zero. The 14 hand-curated locations in the sheet exist nowhere machine readable; recovering them is cleanup work. refresh-catalogue.yaml re-renders both pages weekly after the harvest and opens a pull request. The site sets freeze: auto and Netlify replays the freeze, so without this a catalogue change reaches readers only when somebody renders locally, which is how the sheet went stale unnoticed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01KniirkTyC9pRpXrCA1JeF3 --- .github/workflows/refresh-catalogue.yaml | 84 +++++++++++ index.qmd | 18 +-- .../tbl-01-openwashdata-datasets.csv | 57 -------- pages/gallery/data/index.qmd | 130 +++++++++++++----- 4 files changed, 188 insertions(+), 101 deletions(-) create mode 100644 .github/workflows/refresh-catalogue.yaml delete mode 100644 pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv diff --git a/.github/workflows/refresh-catalogue.yaml b/.github/workflows/refresh-catalogue.yaml new file mode 100644 index 00000000..7a66079f --- /dev/null +++ b/.github/workflows/refresh-catalogue.yaml @@ -0,0 +1,84 @@ +# Re-render the pages that read the owdata catalogue, and commit the +# refreshed freeze. +# +# The site sets `freeze: auto` and commits `_freeze/`, and Netlify replays +# that freeze rather than running R. So a page only changes when someone +# renders it and commits the result. That is why the old Google Sheet +# catalogue went stale on the site even while the sheet itself was edited. +# +# This job does that render on a schedule instead of by hand. It runs +# after the owdata harvest, which is Monday 03:00 UTC, and opens a pull +# request rather than pushing, so a catalogue change is reviewed like any +# other change to the site. + +name: refresh-catalogue + +on: + schedule: + - cron: '0 9 * * 1' + workflow_dispatch: + +permissions: + contents: write + pull-requests: write + +jobs: + refresh: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: dev + + - uses: quarto-dev/quarto-actions/setup@v2 + + - uses: r-lib/actions/setup-r@v2 + with: + use-public-rspm: true + + - uses: r-lib/actions/setup-r-dependencies@v2 + with: + packages: | + any::readr + any::dplyr + any::DT + any::here + any::rmarkdown + any::pak + + # The landing page loads whichever data package is dataset of the + # month, which changes by hand and is not in Config/Needs. + - name: Install the dataset of the month + run: | + pkg=$(grep -oE '^library\(([a-z0-9]+)\)' index.qmd | sed -n '2s/library(\(.*\))/\1/p') + if [ -n "$pkg" ]; then + Rscript -e "pak::pak(paste0('openwashdata/', '$pkg'))" + fi + + # Only the two pages that read the catalogue. Rendering the whole + # site would need every data package the blog and gallery load. + - name: Render the catalogue pages + run: | + quarto render pages/gallery/data/index.qmd --to html + quarto render index.qmd --to html + + - name: Open a pull request when the catalogue changed + uses: peter-evans/create-pull-request@v6 + with: + branch: refresh-catalogue + base: dev + add-paths: | + _freeze/index/** + _freeze/pages/gallery/data/** + pages/gallery/data/data_data/** + commit-message: "chore: refresh the data catalogue from the owdata harvest" + title: "Refresh the data catalogue" + body: | + The weekly owdata harvest changed the catalogue, so the frozen + render of the data page is out of date. + + This pull request carries the re-executed freeze and the cached + CSVs. Merging it publishes the current catalogue. + + Opened by `.github/workflows/refresh-catalogue.yaml`. + delete-branch: false diff --git a/index.qmd b/index.qmd index 465888c8..93298893 100644 --- a/index.qmd +++ b/index.qmd @@ -24,14 +24,16 @@ listing: ```{r} #| include: false library(dplyr) +# Same source as the data catalogue: the weekly owdata harvest of the +# organisation (openwashdata/owdata#11). data <- readr::read_csv( - "pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv", + "https://raw.githubusercontent.com/openwashdata/owdata/main/inst/extdata/owd_packages.csv", show_col_types = FALSE ) -pub <- data |> filter(status == "published") -n_pub <- nrow(pub) -n_loc <- n_distinct(pub$location) -first_year <- min(as.integer(format(pub$date_published, "%Y")), na.rm = TRUE) +n_pub <- nrow(data) +n_doi <- sum(data$published, na.rm = TRUE) +n_ds <- sum(data$n_datasets, na.rm = TRUE) +first_year <- min(as.integer(format(data$date, "%Y")), na.rm = TRUE) n_years <- as.integer(format(Sys.Date(), "%Y")) - first_year + 1 ``` @@ -63,11 +65,11 @@ n_years <- as.integer(format(Sys.Date(), "%Y")) - first_year + 1 ::: {.owd-numbers} ::: {.owd-number-item}
`r n_pub`
-
published data packages
+
open data packages, `r n_doi` with a DOI
::: ::: {.owd-number-item} -
`r n_loc`
-
countries and regions
+
`r n_ds`
+
datasets inside them
::: ::: {.owd-number-item}
`r n_years`
diff --git a/pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv b/pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv deleted file mode 100644 index 669a1149..00000000 --- a/pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv +++ /dev/null @@ -1,57 +0,0 @@ -id,pkg_name,maintainer,source,difficulty,status,description,location,date_published,link_github,link_pkg_website,doi,temporal_coverage,keywords -1,wasteskipsblantyre,sebastian-loos,academic,low,published,"Data on locations of the publicly accessible waste skips in Blantyre, Malawi collected in 2021.",Malawi,2023-08-02T00:00:00Z,,,10.5281/zenodo.8208417,,"solid waste, waste skip, waste management" -2,fsmglobal,larnsce,academic,low,published,Populations of 175 countries requiring different faecal sludge emptying services.,Global,2023-08-02T00:00:00Z,,,10.5281/zenodo.8208293,,"fecal sludge, pit latrines, solid waste" -3,biogasoutcomesmalawi,larnsce,academic,medium,published,Data for 61 semi-structured interviews with biogas owners in Malawi.,Malawi,2023-08-28T00:00:00Z,,,10.5281/zenodo.8289621,,"biogas, interviews, outcomes" -4,cbssuitabilityhaiti,sebastian-loos,ngo,low,published,Spatial data to support an analysis of suitability of container-based sanitation in flood prone areas of Haiti.,Haiti,2023-09-19T00:00:00Z,,,10.5281/zenodo.8361085,,"solid waste, water access, sanitation zoning" -5,waterpumpkwale,mianzg,academic,low,published,"Weekly volume of water pumped for handpumps monitored with Smart Handpump technology, Kwale County, Kenya",Kenya,2023-09-20T00:00:00Z,,,10.5281/zenodo.8399887,,"upgro, handpumps, geolocation" -6,grdwtrsmpkwale,sebastian-loos,academic,medium,published,"Groundwater analysis from 2016 in Kwale, Kenya",Kenya,2023-09-20T00:00:00Z,,,10.5281/zenodo.10016875,,"solid waste, upgro, groundwater" -7,universityrankingafrica,sskorik,academic,low,published,Data from institutional and national levels was combined into a dataset to provide a comprehensive characterization of the top ranked Universities of every African country.,Africa,2023-12-05T00:00:00Z,,,NA,,"rankings, education, colonial effects" -8,basisghana,sskorik,government,high,published,Open defecation free (ODF) communities in Ghana in 2017.,Ghana,2023-12-14T00:00:00Z,,,NA,,"odf, basis, clts" -9,cleanteamghana,sskorik,private,low,published,"User experience of the Clean Team Ghana’s container-based sanitation service in Kumasi, Ghana.",Ghana,2023-12-28T00:00:00Z,,,NA,,"clean team, publication, csat" -10,chckapmalawi,sskorik,ngo,low,published,Community Health Club Knowledge Attitude Practice Survey Malawi,Malawi,2024-01-04T00:00:00Z,,,NA,,"kap, community health, baseflow" -11,gdho,mianzg,private,low,published,A dataset of global humanitarian organizations collected by Humanitarian Outcomes.,Global,2024-02-29T00:00:00Z,,,10.5281/zenodo.10727977,,"humanitarian organisations, ngos" -12,rwpfunctionality,margauxgo,academic,low,published,"Water point monitoring data associated with the paper ""Rural water point functionality estimates and associations: evidence from nine countries in sub-Saharan Africa and South Asia"".",Global,2024-03-26T00:00:00Z,,,NA,,"water point functionality, sdgs, development" -13,washopenresearch,mianzg,academic,high,published,"Dataset about open research data availability in Water, Sanitation and Hygiene (WASH).",Global,2024-05-13T00:00:00Z,,,10.5281/zenodo.11185700,,"publications, research data, open access" -14,washinvestments,margauxgo,academic,low,published,"Dataset on multilateral development bank (MDB) investment in water supply and sanitation associated with the paper ""Multilateral development banks investment behaviour in water and sanitation: Findings and lessons from 60 years of investment projects in Africa and Asia"" (Heidler et al. 2023).",Global,2024-05-16T00:00:00Z,,,NA,,"development banks, wash, infrastructure" -15,ugabore,margauxgo,private,low,published,Borehole repair data from central Uganda associated with a project report completed by Joseph Lwere for the “data science for openwashdata” course,Uganda,2024-06-24T00:00:00Z,,,NA,,"uganda, borehole, groundwater" -16,boreholefuncmwi,myesaya,NA,low,published,"Data about the survey on borehole functionality in Karonga district, Malawi.",Malawi,2024-07-03T00:00:00Z,,,10.5281/zenodo.12635355,,"borehole, malawi, groundwater" -17,waschoolpiracema,AlexisTabin,academic,low,published,"Data about the water supply, the sewage disposal, the waste collection and the sanitary equipment of the schools in Piracema, Brazil.",Brazil,2024-07-09T00:00:00Z,,,10.5281/zenodo.12701107,,"brazil, schools, wash" -18,saniabidjan,mianzg,academic,low,published,"Data about household survey of managing sanitation facilities in Abidjan, Cote d’Ivoire.",Cote d'lvoire,2024-07-30T00:00:00Z,,,10.5281/zenodo.13134220,,"household survey, sanitation facilities, sanitation practices" -19,portawaterperu,sebastian-loos,government,medium,published,Data about a preliminary review of the portable water system in Peru.,Peru,2024-07-30T00:00:00Z,,,10.5281/zenodo.13134310,,"siasar database, potable water, water quality" -20,watercostaccra,margauxgo,academic,low,published,"Data about the surveys on household water costs, coping mechanisms, and water point estimates in Accra, Ghana.",Ghana,2024-10-23T00:00:00Z,,,10.5281/zenodo.13981225,,"household survey, water costs, adaptation" -21,breathablepitlat,yashdubey132,academic,low,published,"Data on transport of contaminants from two pour flush, twin-pit latrines in which one pit is lined and one is unlined.",India,2024-10-23T00:00:00Z,,,10.5281/zenodo.13960699,,"pit latrines, contamination, soil quality" -22,choleramalawi,yashdubey132,government,medium,published,"Data for the progress of Cholera epidemic in Malawi in 2022-23. Resolved at district scale, includes data on cases and deaths.",Malawi,2024-10-08T00:00:00Z,,,10.5281/zenodo.13920530,,"cholera, epidemic, malawi" -23,unhcrwash,yashdubey132,ngo,low,published,This is a dataset on WASH indicators in refugee camps and settlements. The data is collected from the UNHCR Information Management System (IRIS) and is available on the UNHCR WASH dashboard. This dataset includes data from 191 sites and 29 countries.,Global,2024-11-19T00:00:00Z,,,10.5281/zenodo.14185117,,"refugees, iris, unhcr, wash" -24,worldhdi,yashdubey132,ngo,medium,published,Human Development Index country-wise from 1990-2022,Global,2024-10-29T00:00:00Z,,,10.5281/zenodo.14006110,,"hdi, unsdgs, un" -25,wsabrazil,larnsce,NA,medium,published,Dataset about wastewater management and household infrastructure from various Brazilian regions.,Brazil,2024-11-27T00:00:00Z,,,NA,,"waste management, sanitation infrastructure, wastewater disposal" -26,fslogisticskampala,larnsce,academic,low,published,"Data on faecal sludge transporting logistics in Kampala, Uganda",Uganda,2024-11-27T00:00:00Z,,,NA,,"fecal sludge, logistics, waste disposal" -27,dryingfaecal,mianzg,academic,high,published,Engineering experiments related to drying of faecal sludge from on-site sanitation facilities and fresh faeces.,Global,2024-12-01T00:00:00Z,,,NA,,"fecal sludge, solid waste, onsite sanitation" -NA,climatevulnerabilitykhm,mianzg,academic,low,in-progress,"Data on Effects of Climate Vulnerability on Sanitation Access, Sanitation Infrastructure Functionality and Households’ Fecal Sludge Management Practices in Rural Cambodia.",Cambodia,NA,,,NA,,"climate change, sanitation, fecal sludge" -NA,wasteburningblantyre,larnsce,academic,high,in-progress,NA,Malawi,NA,,,NA,,NA -NA,whatawaste,mianzg,NA,low,in-progress,Data about a global project What a Waste that aggregated data on solid waste management from around the world,Global,NA,,,NA,,NA -NA,swmmakhanda,larnsce,NA,NA,in-progress,NA,NA,NA,,,NA,,NA -NA,hwiserrwanda,yashdubey132,ngo,low,in-progress,"Data from a household water insecurity survey panel conducted in Rwanda over 3 rounds, to understand water insecurity and its impact on household activities and hygiene practices.",Rwanda,NA,NA,NA,NA,,NA -28,planetaryhdi,yashdubey132,ngo,low,published,"Provides data on Planetary Pressures Adjusted Human Development, adjusted by carbon emissions per capita (production related) and materials footprint per capita.",Global,2024-12-10T00:00:00Z,,,10.5281/zenodo.14355748,,"hdi, planetary pressures, emissions per capita, sdgs" -29,ploswater,yashdubey132,academic,low,published,Provides access to publishing trends from PLOS Water Journal.,Global,2025-01-08T00:00:00Z,,,10.5281/zenodo.14616993,,"plos, water, research" -30,glossarywho,yashdubey132,multi-lateral,high,published,This package provides access to a tidy version of the WHO Glossary (https://www.who.int/publications/i/item/9789240105485) and thematic areas for each term.,Global,2025-01-30T00:00:00Z,,,10.5281/zenodo.14754017,,"who, glossary, definitions" -31,ds4owdanalytics,yashdubey132,academic,medium,published,The goal of ds4owdanalytics is to make dark data from the DS4OWD course publicly available,Global,2025-01-31T00:00:00Z,,,10.5281/zenodo.14778963,,"ds4owd, analytics" -32,undpcomposite,yashdubey132,ngo,low,published,Provides tidy data about all UNDP indicators in a composite timeseries.,Global,2025-02-10T00:00:00Z,,,10.5281/zenodo.14845848,,"undp, hdi, composite indicators" -33,glaas,massarin,multi-lateral,medium,in-progress,UN-Water Global Analysis and Assessment of Sanitation and Drinking-water,Global,2025-05-15T00:00:00Z,,,10.5281/zenodo.15497462,,"governance, monitoring, finance" -34,boreholeforensicspumpingtest,emmanuellmhango,ngo,medium,published,"This dataset contains borehole forensic pumping test data, capturing key parameters related to groundwater extraction, borehole performance, and water quality assessments in Malawi.",Malawi,2025-03-05T00:00:00Z,,,10.5281/zenodo.14974957,,"borehole pumping tests, groundwater data, static water level, flow rate, water quality indicators" -35,floodchlorinationsurvey,emmanuellmhango,ngo,low,published,USAID Flood Response Rehabilitation and Chlorination Survey,Malawi,2023-07-11T00:00:00Z,,,10.5281/zenodo.15846578,,"flood response, rehabilitation, chlorination, water point functionality," -36,washmalawi,emmanuellmhango,ngo,medium,published,SDG Household Level Survey (Malawi CJF),Malawi,2025-04-30T00:00:00Z,,,10.5281/zenodo.15799918,,"household survey, sanitation, hygiene, demographic indicators, WASH access" -37,artesianwells,emmanuellmhango,ngo,low,published,Identification and Mapping of Artesian wells,Malawi,2025-03-06T00:00:00Z,,,10.5281/zenodo.15824828,,"artesian wells, aquifer recharge, drilling logs" -38,thyolocbcc,emmanuellmhango,ngo,low,published,Thyolo District WASH in CBCC survey,Malawi,2025-06-24T00:00:00Z,,,10.5281/zenodo.17540499,,"childcare centres, facility survey, hygiene practices, service availability" -39,dowalatrinesurvey,emmanuellmhango,ngo,medium,published,Dowa Household Latrine survey,Malawi,2025-06-25T00:00:00Z,,,10.5281/zenodo.17512719,,"latrine usage, rural households, sanitation coverage, hygiene behaviours, facility types" -40,waterconsumersurvey,emmanuellmhango,ngo,low,published,Household Consumer Satisfaction Survey - Zomba and Mangochi,Malawi,2025-06-27T00:00:00Z,,,10.5281/zenodo.17523095,,"consumer satisfaction, service reliability, household usage" -41,nchalohhwatersurvey,emmanuellmhango,ngo,low,published,Nchalo water project household survey 2021,Malawi,2025-06-27T00:00:00Z,,,10.5281/zenodo.17522183,,"household survey, service infrastructure, consumption patterns" -42,floodresponsemw,emmanuellmhango,ngo,low,published,USAID Flood Response - Water Point Survey 2019 - 2020,Malawi,2025-06-27T00:00:00Z,,,10.5281/zenodo.17432024,,"flood impact, emergency interventions, infrastructure damage, post-disaster assessment, rehabilitation activities" -43,drillingdata,emmanuellmhango,ngo,medium,published,Drilling data,Malawi,2025-04-08T00:00:00Z,,,10.5281/zenodo.17543306,,"borehole construction, drilling records, site characteristics, technical specifications, depth measurements" -44,hydrogeologicalghana,seawaR,academic,low,published,Data on boreholes collected to study the hydrogeological dynamics in the greater Accra and Ashanti regions of Ghana.,Ghana,2025-08-05T00:00:00Z,,,10.5281/zenodo.16743,,"borehole, pumping rate, drawdown, acquifer material" -45,postfloodintervention,emmanuellmhango,ngo,low,published,"USAID Flood Response – Post Intervention Survey (Mulanje, 2019–2020)",Malawi,2025-07-08T00:00:00Z,,,10.5281/zenodo.15837461,,"infrastructure repair, flood recovery, rehabilitation activities, WASH interventions, community needs" -46,boreholelabdata,emmanuellmhango,ngo,high,published,Water Quality Laboratory Analysis and Reporting Dataset – Malawi (2017–2019),Malawi,2025-08-08T00:00:00Z,,,10.5281/zenodo.17433462,,"water quality tests, chemical indicators, lab results, borehole sampling, contamination assessment" -47,dowaodfsurvey,emmanuellmhango,ngo,low,published,ODF Sanitation and Hygiene Household Survey – Ntchisi & Dowa District 2019,Malawi,2025-08-08T00:00:00Z,,,10.5281/zenodo.17492108,,"open defecation, sanitation verification, household survey, behavioural assessment, community coverage" -48,handpumpstatusdata,emmanuellmhango,ngo,medium,published,"Handpump Functionality Verification Survey - Chiradzulu, Malawi 2020",Malawi,2025-08-12T00:00:00Z,,,10.5281/zenodo.17474594,,"handpump functionality, borehole equipment, service reliability, survey records, maintenance status" -49,waterpointfunctionality,emmanuellmhango,ngo,low,published,"Water Point Functionality - Malawi, 2022",Malawi,2025-09-19T00:00:00Z,,,10.5281/zenodo.17433322,,"functional status, non-functional points, community access, repair needs, reliability assessment" -50,mwefloodresponse,emmanuellmhango,ngo,low,published,Malawi Emergency Flood Response Water Point Survey (2019–2020),Malawi,2026-02-25T00:00:00Z,,,10.5281/zenodo.18755616,,"emergency flood data, waterpoints, WASH interventions" -51,mwgroundwaterdata,emmanuellmhango,ngo,low,published,Malawi Groundwater Monitoring Time-Series Dataset (2024–2025),Malawi,2026-02-26T00:00:00Z,,,10.5281/zenodo.18798409,,"groundwater monitoring, monitoring wells" diff --git a/pages/gallery/data/index.qmd b/pages/gallery/data/index.qmd index 6a06d8ae..311e31e7 100644 --- a/pages/gallery/data/index.qmd +++ b/pages/gallery/data/index.qmd @@ -1,63 +1,121 @@ --- pagetitle: "Data" description: | - A table of datasets that have been published through the openwashdata - community or are currently in development. -editor_options: + A searchable catalogue of the datasets published through the openwashdata + community, and of the variables inside them. +editor_options: chunk_output_type: console -execute: +execute: echo: false warning: false --- ```{r} -options(knitr.kable.NA = '') -library(stringr) library(DT) +library(dplyr) ``` +```{r} +# The catalogue comes from the owdata harvest, which reads every repository in +# the organisation once a week. Until washr 1.1.0 this page read a Google Sheet +# that nothing writes to any more (openwashdata/owdata#11). +owd_base <- "https://raw.githubusercontent.com/openwashdata/owdata/main/inst/extdata/" -## Data +packages <- readr::read_csv(paste0(owd_base, "owd_packages.csv"), show_col_types = FALSE) +variables <- readr::read_csv(paste0(owd_base, "owd_variables.csv"), show_col_types = FALSE) -The table on this pages shows data that is published following our R data package workflow. Each data set comes with its own website and rich documentation to increase reusability. +readr::write_csv(packages, here::here("pages/gallery/data/data_data/owd_packages.csv")) +readr::write_csv(variables, here::here("pages/gallery/data/data_data/owd_variables.csv")) -```{r} -data <- googlesheets4::read_sheet("1vtw16vpvJbioDirGTQcy0Ubz01Cz7lcwFVvbxsNPSVM") +n_pkg <- nrow(packages) +n_doi <- sum(packages$published, na.rm = TRUE) +``` -data |> readr::write_csv(here::here("pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv")) +## Data -data <- readr::read_csv(here::here("pages/gallery/data/data_data/tbl-01-openwashdata-datasets.csv")) +The table below lists every R data package in the openwashdata organisation. +Each one comes with its own website and documentation, so the data can be +reused and cited. -``` +A package has a **DOI** once it is deposited on Zenodo, which is what makes it +citable and permanently archived. Of the `r n_pkg` packages listed here, +`r n_doi` have one. The rest are published as packages and usable today, and +are waiting on a deposit. :::{.column-page} ```{r} -tbl_data <- data |> - dplyr::mutate(link_pkg_website = dplyr::case_when( - status == "in-progress" ~ NA_character_, - TRUE ~ link_pkg_website - )) |> - dplyr::mutate(status = dplyr::case_when( - status == "in-progress" ~ paste0("[", status, "]", "(", link_github, ")"), - TRUE ~ status - )) |> - dplyr::select(id, - name = pkg_name, - location, - status, - description, - published = date_published, - website = link_pkg_website) |> - dplyr::mutate(published = format(published, "%d. %B %Y")) |> - dplyr::arrange(id) - -tbl_data |> - dplyr::filter(!stringr::str_detect(status, "in-progress")) |> - knitr::kable(padding = 0) +tbl <- packages |> + mutate( + Package = paste0("[", pkg_name, "](", url_docs, ")"), + DOI = ifelse( + is.na(doi), + "", + paste0("[", doi, "](https://doi.org/", doi, ")") + ), + Updated = format(date, "%d %B %Y") + ) |> + select( + Package, + Description = description, + Datasets = n_datasets, + DOI, + Updated + ) |> + arrange(Package) +datatable( + tbl, + escape = FALSE, + rownames = FALSE, + filter = "top", + options = list( + pageLength = 25, + lengthMenu = c(10, 25, 50, 100), + columnDefs = list(list(width = "45%", targets = 1)) + ) +) ``` - ::: +## Variables +Every variable documented across the packages with a DOI. Search here to find +which dataset holds the measurement you need. +:::{.column-page} +```{r} +var_tbl <- variables |> + left_join(select(packages, pkg_name, url_docs), by = "pkg_name") |> + mutate(Package = paste0("[", pkg_name, "](", url_docs, ")")) |> + select( + Package, + Dataset = dataset_name, + Variable = variable_name, + Type = variable_type, + Description = description + ) |> + arrange(Package, Dataset, Variable) + +datatable( + var_tbl, + escape = FALSE, + rownames = FALSE, + filter = "top", + options = list( + pageLength = 25, + lengthMenu = c(10, 25, 50, 100) + ) +) +``` +::: + +```{r} +#| output: asis +cat( + "\n*Catalogue harvested from the openwashdata organisation by ", + "[owdata](https://github.com/openwashdata/owdata). ", + "Most recent package update: ", + format(max(packages$date, na.rm = TRUE), "%d %B %Y"), ".*\n", + sep = "" +) +```