diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index abb3ba85..eb098f38 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,6 +27,10 @@ A network instance file is read into a graph whose nodes and edges carry the fie - `orlib-pmedcap` reads OR-Library's capacitated p-median file, which holds several problems, as one tasks/v1 document: every problem's customers are nodes `-` with their position and demand and their problem's `medians`, `medianCapacity` and `bestValue`. - `lrp` reads Prodhon's location-routing `.dat` layout (Prins-Prodhon, Tuzun-Burke and Barreto) as tasks/v1: the depots `depot-` with `capacity` and `openingCost`, then the customers `customer-` with `demand`, all with `x` and `y` and a `role`, and the counts, `vehicleCapacity`, `routeCost` and `realCosts` (false where costs are distances times 100 truncated) in `meta`. A coordinate line may carry further fields only if they are zero, as Barreto's Or117 depot lines do. - `dimacs-min` reads a DIMACS minimum-cost flow `.min` file: the nodes `1` to `n` with their `supply` (zero for a node without a descriptor), and the arcs as directed edges with `low`, `capacity` and `cost`, parallel arcs kept and a negative capacity, the specification's mark of an uncapacitated arc, carried as null. The `edge-list` parser reads endpoints only, so it cannot read these files. +- `tntp` reads a Transportation Networks for Research dataset in the TNTP format from a parts archive of its `network` file and its `trips` table (the repository's zip at a commit, which holds both): the nodes `1` to `n`, those up to the zone count of type `zone` and the rest of type `node`, each link a directed edge of type `link` with one field per column the harmonised header line `~ init_node term_node ...` names, in camel case (`free_flow_time` is `freeFlowTime`), and each origin and destination entry of the trip table a commodity node `commodity--` with its `demand`, joined by an `origin` and a `destination` edge to its zones, zero entries kept as the file lists them. `meta` has the zone count, the first through node, the total flow, the original header and any other metadata as text. A link line without its `;`, of another number of values than the header names, a node or zone outside its range, a trip table whose zone count is not the network's and a pair given twice are refused. +- `dimacs-graph` reads an undirected graph in the DIMACS challenge graph format (`layout: "dimacs"`: `p edge N M`, then `e U V` lines, as the clique and colouring benchmarks write it) or the PACE challenge format (`layout: "pace"`: `p PROBLEM N M`, then bare `U V` lines): the nodes `1` to `N`, so a vertex no edge touches is kept, and each edge once in file order. A self-loop, an edge given twice in either direction, a node outside the range and an edge count other than the problem line's are refused. It emits topology only, for the shared `topology.json` crosswalk. +- `steinlib` reads a Steiner tree problem in graphs in the SteinLib STP format: the nodes `1` to `n`, the first Terminals section's nodes of type `terminal` and the rest of type `node`, each `E U V WEIGHT` line an undirected edge with its `weight`, and the Comment section's `Name`, `Date`, `Creator`, `Remark` and `Problem` in `meta`. A presolved instance's Presolve section gives `meta` its `fixed` value, bounds, `presolveTime` and the original's counts; its edge mapping lines, the original instance's Terminals section after it and every Coordinates section are not read. Directed (`Arcs`), obstacle, rooted, prize-collecting and degree-constrained instances are refused, as are an unknown section or key and counts that disagree with the lines. +- `matpower` reads a power system case in the MATPOWER case format, version 2 (the PGLib optimal power flow cases): each bus a node of type `bus`, its id the bus number, with the bus table's columns (`type` as `busType`), each generator a node `generator-` of type `generator` with the generator table's columns (the eleven optional ones null where a case gives ten) and its cost row's `startup`, `shutdown`, `c2`, `c1` and `c0`, joined to its bus by an edge of type `generator`, and each branch an edge of type `branch` with the branch table's columns (`status` as `branchStatus`). `meta` holds the case's name, version and base MVA, and the legacy area table as text. A table other than bus, gen, gencost, branch and areas, a cost other than a quadratic polynomial, a row of another width and a generator or branch at a bus the case does not list are refused. A reference value a source publishes in a file of its own (the OR-Library optima files) is not merged into the instances; the entry lists that file as a source and its description says where the values are. @@ -38,7 +42,7 @@ A text whose records are not a table (a host's directory listing, a netlib index ## Archives and graph file formats -A recipe's `archive` says how the downloaded bytes become the file the parser reads: `none`, a compressed file (`gzip`, `bzip2` or `xz`), a tar archive (`tar`, `tar.gz`, `tar.bz2` or `tar.xz`), a zip archive or a 7z archive (`7z`), from which `member` selects the one file to read (or `members` the files of a collection, or `parts` the named files one document is read from), or an Excel workbook (`xlsx`). A `parts` archive names each part and the one member it is read from, and the parser reads the parts together in the order given; only the parsers that read several files take one (`kg-triples`, whose part names are the splits, and `hgb`, whose parts are `nodes` and `links`), and the entry schema refuses it for any other. A bzip2 file is decoded with every block's CRC and the stream's checked; concatenated streams are read as one file, as `bzip2 -d` reads them, and input that ends inside a stream or holds anything after the last one fails the extract step rather than yielding a prefix of the file. A gzip file (a `gzip` file, the inside of a `tar.gz`, a gzipped member that `inner` unpacks, or a body whose transport encoding is gzip) is read by one platform-neutral reader (`src/gzip.ts`), which reads each member's framing itself and hands only the deflate data to pako's port of zlib, so it knows where each member's trailer is: every member's header is checked, and the CRC-32 and the length modulo 2^32 of what the member decompressed to are compared with the ones its trailer records; concatenated members are read as one file, as `gzip -d` reads them, zero padding after the last member is read past, and input that ends inside a member, fails either check or holds anything else after the last member fails the extract step rather than yielding a prefix of the file. Its tests check it against Node's zlib as the reference, with the data cut at every offset (`src/gzip.unit.test.ts`). An xz file is decoded as it arrives by `xz-decompress`, the XZ Embedded decoder compiled to WebAssembly, which asks for a compressed chunk only when it needs input, so a `tar.xz` far larger than memory is held as its decoder's dictionary and one chunk each way; it reads concatenated streams as one file, as `xz -d` does, and fails on a stream cut short, on bytes after a stream that are neither padding nor another stream, and on a checksum or index that does not hold. The decoder is a port (`XzPort`, `FetchOptions.xz`) that the Node entry point (`keepset/node`) supplies and the browser entry point does not, so a recipe whose archive is xz fails at its extract step in a browser with an error that says where the decoder is, and the browser bundle carries no WebAssembly decoder. A tar archive (plain, gzip, bzip2 or xz) is decompressed and scanned in one pass as it is read back from the kept download, keeping the data of only the members its selector can match, and a zip is read by the ranges its central directory names, so an archive far larger than memory is read in the memory of the members a recipe selects; a tar that ends part of the way through a header, a member or its padding fails the extract step. The readers take a source of bytes (`src/source-bytes.ts`) that the fetch's store keeps: Node's `fileStore` (a temporary file, hashed as it arrives, or ranged requests for a zip of at least `RANGED_MIN_BYTES`) or the browser's in-memory store. A change to a reader is tested against the whole-archive reference (decompress into one array, scan that) with the source cut into chunks of several sizes (`src/archive-streamed.unit.test.ts`), and `src/node/large-source.unit.test.ts` checks that a fetch of a source several times its memory budget retains less than the budget, measured after forced collections as the source is pulled. A 7z archive, solid or not, is read by the platform's 7z reader, a port (`SevenZipPort` in `src/archive.ts`) the fetch is given beside its store: Node's (`src/node/seven-zip.ts`) runs 7-Zip compiled to WebAssembly (the `7z-wasm` package) in a worker thread on the file the store keeps, which 7-Zip reads through Emscripten's NODEFS rather than into memory, decodes a solid block once from its start holding only its dictionary, and writes only the selected members to a temporary folder, from which each is read back when it is reached; the browser entry point gives no 7z reader and refuses the kind. Its tests build their archives with the same 7-Zip, so no system `7z` is needed, and check it against the naive reference of extracting the whole archive in memory (`src/node/seven-zip.unit.test.ts`), and the large-source test checks that a solid 7z several times the budget is decoded with the process and 7-Zip's own heap each retaining less than it. corpus too large to hold as one document has a `stream` recipe with a `files` archive read by a parser that makes one row of each file (`member-text`): `streamDataset` downloads the source whole, then hands on each file's row as soon as the file is read, inflating a zip member that is a tar in its turn and scanning it as it is read (`streamFiles` in `src/archive.ts`), so the corpus is read in the memory of one file; the files of a tar come in the order the tar holds them, since sorting them would mean holding them all, and `fetchDataset` refuses the recipe. +A recipe's `archive` says how the downloaded bytes become the file the parser reads: `none`, a compressed file (`gzip`, `bzip2` or `xz`), a tar archive (`tar`, `tar.gz`, `tar.bz2` or `tar.xz`), a zip archive or a 7z archive (`7z`), from which `member` selects the one file to read (or `members` the files of a collection, or `parts` the named files one document is read from), or an Excel workbook (`xlsx`). A `parts` archive names each part and the one member it is read from, and the parser reads the parts together in the order given; only the parsers that read several files take one (`kg-triples`, whose part names are the splits, `hgb`, whose parts are `nodes` and `links`, and `tntp`, whose parts are `network` and `trips`), and the entry schema refuses it for any other. A bzip2 file is decoded with every block's CRC and the stream's checked; concatenated streams are read as one file, as `bzip2 -d` reads them, and input that ends inside a stream or holds anything after the last one fails the extract step rather than yielding a prefix of the file. A gzip file (a `gzip` file, the inside of a `tar.gz`, a gzipped member that `inner` unpacks, or a body whose transport encoding is gzip) is read by one platform-neutral reader (`src/gzip.ts`), which reads each member's framing itself and hands only the deflate data to pako's port of zlib, so it knows where each member's trailer is: every member's header is checked, and the CRC-32 and the length modulo 2^32 of what the member decompressed to are compared with the ones its trailer records; concatenated members are read as one file, as `gzip -d` reads them, zero padding after the last member is read past, and input that ends inside a member, fails either check or holds anything else after the last member fails the extract step rather than yielding a prefix of the file. Its tests check it against Node's zlib as the reference, with the data cut at every offset (`src/gzip.unit.test.ts`). An xz file is decoded as it arrives by `xz-decompress`, the XZ Embedded decoder compiled to WebAssembly, which asks for a compressed chunk only when it needs input, so a `tar.xz` far larger than memory is held as its decoder's dictionary and one chunk each way; it reads concatenated streams as one file, as `xz -d` does, and fails on a stream cut short, on bytes after a stream that are neither padding nor another stream, and on a checksum or index that does not hold. The decoder is a port (`XzPort`, `FetchOptions.xz`) that the Node entry point (`keepset/node`) supplies and the browser entry point does not, so a recipe whose archive is xz fails at its extract step in a browser with an error that says where the decoder is, and the browser bundle carries no WebAssembly decoder. A tar archive (plain, gzip, bzip2 or xz) is decompressed and scanned in one pass as it is read back from the kept download, keeping the data of only the members its selector can match, and a zip is read by the ranges its central directory names, so an archive far larger than memory is read in the memory of the members a recipe selects; a tar that ends part of the way through a header, a member or its padding fails the extract step. The readers take a source of bytes (`src/source-bytes.ts`) that the fetch's store keeps: Node's `fileStore` (a temporary file, hashed as it arrives, or ranged requests for a zip of at least `RANGED_MIN_BYTES`) or the browser's in-memory store. A change to a reader is tested against the whole-archive reference (decompress into one array, scan that) with the source cut into chunks of several sizes (`src/archive-streamed.unit.test.ts`), and `src/node/large-source.unit.test.ts` checks that a fetch of a source several times its memory budget retains less than the budget, measured after forced collections as the source is pulled. A 7z archive, solid or not, is read by the platform's 7z reader, a port (`SevenZipPort` in `src/archive.ts`) the fetch is given beside its store: Node's (`src/node/seven-zip.ts`) runs 7-Zip compiled to WebAssembly (the `7z-wasm` package) in a worker thread on the file the store keeps, which 7-Zip reads through Emscripten's NODEFS rather than into memory, decodes a solid block once from its start holding only its dictionary, and writes only the selected members to a temporary folder, from which each is read back when it is reached; the browser entry point gives no 7z reader and refuses the kind. Its tests build their archives with the same 7-Zip, so no system `7z` is needed, and check it against the naive reference of extracting the whole archive in memory (`src/node/seven-zip.unit.test.ts`), and the large-source test checks that a solid 7z several times the budget is decoded with the process and 7-Zip's own heap each retaining less than it. corpus too large to hold as one document has a `stream` recipe with a `files` archive read by a parser that makes one row of each file (`member-text`): `streamDataset` downloads the source whole, then hands on each file's row as soon as the file is read, inflating a zip member that is a tar in its turn and scanning it as it is read (`streamFiles` in `src/archive.ts`), so the corpus is read in the memory of one file; the files of a tar come in the order the tar holds them, since sorting them would mean holding them all, and `fetchDataset` refuses the recipe. A ratings file (one rating per line as user, item, rating and time, which is the layout of an Amazon ratings-only file and of each MovieLens release) uses the `user-item-ratings` parser with the recipe stating the delimiter (`comma`, `tab` or `double-colon`), the header lines to skip, what the items are (`movie` or `product`) and the top of the rating scale. It gives a bipartite graph: a user is the node `u:` of type `user` and an item `m:` or `p:` of the item type (so user 1 and movie 1, which a source numbers independently, are two nodes), and each rating is an edge from its user to its item of type `rating`, whose `weight` is the rating over the scale's top and whose `timestamp` is the time field as the file writes it. The entry's `directed` property records whether that direction is meaningful. Its crosswalk is `crosswalks/graph-v1/rated.json`. The file is read a piece of text at a time and each distinct id is held once, so the parser holds the graph it returns and not the text. A fetch holds more than the parser does, because the crosswalk step copies every edge, so a release with more edges than that step can hold is given no recipe, with the failure as its reason, until the step can. diff --git a/README.md b/README.md index da53065b..e8acb320 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ An alias that is a path, such as `foo/graphs/bar`, files the entry in a second g ## The entry format -An entry is a JSON document in `catalogue/entries/`: a descriptor (stable id, name, description, domain, subjects, tags), the category whose normalisation target the entry resolves to, its sources (each with a retrieval date, and for a dead source a pinned Wayback snapshot that travels with the URL because the snapshot timestamp is part of the recipe's reproducibility), its licence and citations, the recipe that turns downloaded bytes into the dataset's native JSON shape, and a crossmap crosswalk mapping that native shape onto the category target. A recipe's archive is no archive, a compressed file (gzip, bzip2 or xz), a tar archive (plain, gzip, bzip2 or xz compressed), a zip archive, a 7z archive (solid or not, read by the Node entry point, which decodes it once with 7-Zip and keeps only the members selected; the browser entry point refuses it) or an Excel workbook, gzip and bzip2 data, whether a compressed file or a compressed tar archive, is checked as it is read (every gzip member's CRC-32 and length, every bzip2 block's CRC and the stream's, and each selected zip member's CRC-32 and length), so a download corrupted or cut short is refused rather than read as a prefix of the file, a member of an archive is selected by path, extension or pattern, and a format that is several files (the splits of a knowledge graph, the node and link files of an HGB dataset) is read from named parts of one archive. The format is defined by Zod schemas, and the generated JSON Schema is published with the package as `keepset/schemas/entry.schema.json`, with the catalogue manifest's beside it. The domain is the entry's topic, its tags are the node fields it carries, its namespace is the source it comes from, and its subjects are the fields of study it is relevant to (for example `operations-research`), as slugs a search filters by (`keepset search --subject operations-research`), a listing page per subject on the site and keywords in the JSON-LD. +An entry is a JSON document in `catalogue/entries/`: a descriptor (stable id, name, description, domain, subjects, tags), the category whose normalisation target the entry resolves to, its sources (each with a retrieval date, and for a dead source a pinned Wayback snapshot that travels with the URL because the snapshot timestamp is part of the recipe's reproducibility), its licence and citations, the recipe that turns downloaded bytes into the dataset's native JSON shape, and a crossmap crosswalk mapping that native shape onto the category target. A recipe's archive is no archive, a compressed file (gzip, bzip2 or xz), a tar archive (plain, gzip, bzip2 or xz compressed), a zip archive, a 7z archive (solid or not, read by the Node entry point, which decodes it once with 7-Zip and keeps only the members selected; the browser entry point refuses it) or an Excel workbook, gzip and bzip2 data, whether a compressed file or a compressed tar archive, is checked as it is read (every gzip member's CRC-32 and length, every bzip2 block's CRC and the stream's, and each selected zip member's CRC-32 and length), so a download corrupted or cut short is refused rather than read as a prefix of the file, a member of an archive is selected by path, extension or pattern, and a format that is several files (the splits of a knowledge graph, the node and link files of an HGB dataset, the network file and trip table of a TNTP dataset) is read from named parts of one archive. The format is defined by Zod schemas, and the generated JSON Schema is published with the package as `keepset/schemas/entry.schema.json`, with the catalogue manifest's beside it. The domain is the entry's topic, its tags are the node fields it carries, its namespace is the source it comes from, and its subjects are the fields of study it is relevant to (for example `operations-research`), as slugs a search filters by (`keepset search --subject operations-research`), a listing page per subject on the site and keywords in the JSON-LD. The site has a listing page for each domain, subject, kind, tag and namespace, each holding the home page's listing with the page's own filter already chosen (without scripts, a static table and a link to the home page's listing filtered to the same datasets), and an index of each attribute's pages that can be filtered by text and ordered by name or by how many datasets each holds. The listing keeps its filters in the page's query string, so a filtered view is a link to share: each attribute is a parameter of comma-separated values (`kind`, `domain`, `subject`, `tag`, `namespace`, with a value holding a comma percent-encoded), an entry matching any value of an attribute and every attribute given, beside `q` for free text and `min-entities` and `max-entities` for the entity total, as in `?domain=facility-location,supply-chain-planning&subject=operations-research&q=depot`. A listing page keeps its own address while its filters are its own; once they change, the address is the home page's listing with the same filters, so a filtered view has one form of link wherever it was reached from. @@ -73,7 +73,7 @@ The catalogue itself is a manifest (`catalogue/manifest.json`: id, version, repo ## Categories and normalisation targets -A category names a normalisation target: the shape every entry in that category resolves to, whatever its native shape. `graph/v1` is `{ meta, nodes, edges }` with unique node ids and edges whose endpoints name nodes; its entries are graph datasets taken from upstream collections such as SuiteSparse, Netzschleuder, SNAP, KONECT and OGB, read by one parser per format (delimited edge lists, which can also be a sheet of a spreadsheet, Matrix Market, GML, Pajek, UCINet DL full matrices, Graph Drawing contest files, Witvliet connectome synapse files, Open Graph Benchmark raw directories read together from the several files of a dataset's zip, KONECT edge files, rating and count files, knowledge graph triples, Heterogeneous Graph Benchmark node and link files, pragmatic similarity rating workbooks, and network instance files: Canad `.dow` multicommodity network design, OR-Library p-median and DIMACS minimum-cost flow problems, whose nodes and edges carry the costs, capacities, demands and supplies the files give), each giving the graph's topology and, where the format states them, node and edge types, weights, times and the split each edge belongs to. `tasks/v1` is `{ meta, nodes }`, a set of tasks as the places or cases it offers to visit or serve (a routing benchmark, or an open data register of permits and complaints), each flat with optional coordinates, prize, prize probability, duration, travel time, skill, territory, lifecycle status and times, and deadline, read from tabular sources by the `csv-table` parser (a delimited file, a member of an archive, a sheet of an Excel workbook, a legacy binary xls workbook or an OpenDocument spreadsheet, or an open data portal read a page at a time) and from ArcGIS feature layers by the `geojson` parser. Its entries come from a survey of open operations-research datasets, imported with a curation that records, for each source, what its download gives and how its columns map onto the node; a dataset whose source has no usable table is imported without a recipe and says why. Any category can be fetched as a collection: a recipe that selects every matching archive member with `members` (by path, by the extensions its name ends with, or by a regular expression its whole path matches) resolves to `{ meta, instances }`, where each instance is a document of the entry's category under the path of the member it came from, the schema derived from the category's own, so a category needs nothing of its own to be collected. A matched member that is itself an archive is opened by the recipe's `inner` layout, a tar, tar.gz, tar.bz2 or zip with its own `members` selector, and each file that selector matches is one instance, filed under the member's path and its own joined by `/`, so a download whose transcripts sit in one tarball inside another is fetched once and every other member of the outer archive is skipped as it streams past. The parser reads each member, the crosswalk is applied to each, and a member that does not parse or does not validate fails the fetch and is named. A `grouped-member-path` parser groups related archive paths by an identifier and representation, retaining their names as fields without loading their payloads; each group must contain exactly the representations its recipe names. Instance files are read by one parser per format (Solomon and TOPTW time window files, Chao team orienteering files and their dynamic form, TSPLIB style orienteering files, VRP-REP XML, VeRoLog 2019 files, customer point lists, Prodhon location-routing files and the OR-Library capacitated p-median file), which gives each node the columns the file has and each instance a header of its own; a curation says which of them fill the `tasks/v1` node's fields, which pass through under their own names and which are dropped, so what an entry leaves out is written down, and a source whose time quantities are not seconds keeps them under the source's names with the unit recorded as the entry's `timeUnit`. `texts/v1` is `{ meta, nodes }`, a set of texts as the works a collection catalogues (a library's catalogue feed, say), each flat with optional title, language, authors, subjects, issue date, type, classification, collections and the addresses of its files as plain text, a web page and an EPUB, read from tabular sources by the `csv-table` parser (which can build a column from a template over a row's other columns, so an address a source only implies by a scheme over the record's id is one more column to map); it describes the works and where their files are and carries none of their contents. `resources/v1` is `{ meta, nodes }`, one downloadable resource (a speech corpus, a lexicon, a toolkit) as its `meta`, which says what it is (name, identifier, summary, category, licence, description, external address), and the files it is distributed as in `nodes`, each with optional url, size and description; a host's whole listing is a collection with one instance per resource, read page by page by the `openslr-resource` parser; a set of model files is read from its host's listing by the `text-pattern` parser (each match of a pattern a file) or from an archive's member paths by the `member-path` parser, and it describes the resources and carries none of their files. `speech/v1` is `{ meta, nodes }`, a speech corpus as the utterances it transcribes, each flat with optional transcript, speaker, gender, language, audio file name, duration in seconds and split, read from tabular sources by the `csv-table` parser (a transcript table may have no header row, in which case the recipe names its columns, use comma-space as its delimiter so unquoted commas remain in fields, or hold double quotes in its text, in which case the recipe says it is unquoted, or hold line breaks inside its cells without quoting them, in which case the recipe names the columns so a line break ends a record only once it has them all), from files of one utterance per line, as Kaldi, LibriSpeech, Festival and Sphinx write them (the id first, the id last, in Festival's parentheses or after a Sphinx transcript's sentence markers, as the recipe's layout says), by the `utterance-lines` parser, from JSON manifests (one record per line, or one array of records) by the `json-table` parser, and from corpora with one small text file per recording by the `member-text` parser, or one XML file per recording by the `nxt-words` and `eaf` parsers (NXT word files and ELAN annotation files), or one Praat TextGrid per recording by the `textgrid` parser (the long or short text form, in UTF-8 or UTF-16 by the file's byte order mark, the tier named, or every tier whose name a pattern matches, or with neither every tier, read, each row carrying the tier it is on), or one CHAT file per conversation by the `chat` parser (each turn a row with its speaker, text and time marks), or the line-aligned `segments`, YAML and transcript files of each split of a Multilingual TEDx language by the `mtedx` parser (a row for each segment with its speaker, audio file, start, end and duration), or one NIST STM file per recording by the `stm` parser (each segment line of a speaker a row with its channel, times in seconds, label field and text, the reserved gap and excluded-region lines left out), or the `ANNOTATIONS` files of the MUSAN corpus by the `musan-annotations` parser (each line of a music, speech or noise list a row, labelled with the kind of sound and the directory it is in, with the genre, vocals, artist and composer of music and the gender and language of speech), which read the files a `files` archive selects together into one table, a member that is an archive in its turn (annotations shipped as a zip inside a tarball) being unpacked by the archive's `inner` selector, and which a collection of downloads (one archive per split, say) reads each file into one instance; when a file is distributed as consecutive downloads, its recipe names the parts in order and provenance records each part separately; a files archive that sets `concatenate` instead joins the files it selects, in path order, into the one file a parser that reads one file takes (a transcript table split into a file for each chapter); it carries the transcripts and none of the audio, which is in the distribution a `resources/v1` entry lists. `lexicon/v1`, `speakers/v1`, `splits/v1` and `audio/v1` are `{ meta, nodes }` as well: a word list or pronunciation dictionary as one entry per line (word, pronunciation, count), a speaker list as its speakers (gender, nationality, language, set, name), a corpus's division into training, validation and test data as the items each part holds, and a collection of sounds that are not transcribed speech as the clips it annotates (label, speaker, duration, split); PCM WAV files can be read by the `wav-metadata` parser, which records their channel count, sample rate, sample width, duration and any invalid format parameters without carrying the audio; a lexicon is read by the `lexicon-lines` parser, or by the `json-word-counts` parser when it is one JSON object of words and their counts (read member by member in the file's order, a word listed twice being an error), the annotations of a sound collection by the `json-table` parser (an object nested by sound class and then by file name, each record named by its keys) or the `option-list` parser (lists of lines of the form `--name value path`, read together from the files a `files` archive selects), or the `simulated-rir` parser (response lists joined to their room dimensions, receiver position and absorption coefficient), or the `air-mat` parser (recording metadata from the Aachen database's MATLAB files), or the `member-path` parser (recording identifiers and named fields encoded in selected archive paths), and each carries the lists and annotations and none of the words' texts, speakers' recordings or audio. `corpus/v1` is `{ meta, nodes }` as well, a text corpus as the documents it holds, each with its whole text as decoded from the source and an optional title, language, format and encoding; it carries the texts themselves, where `texts/v1` only describes them. Its Project Gutenberg entry reads the zipped tar of every plain-text ebook that Project Gutenberg publishes for bulk use, one book per document named by its ebook number, with the `member-text` parser set to `verbatim`, so each text is the file exactly as distributed, licence header and footer included; the library is far larger than one document can hold, so the recipe is streamed and `keepset fetch` writes it a book at a time. A corpus of Word documents is read by the `word-text` parser, one document per file, each a `.docx` package or a legacy `.doc` file (Word 97 to 2003) told apart by its bytes and read as the plain text of its main story. A new category starts by defining its target schema in `src/targets/` and registering it in the category registry; an entry cannot name a category that does not exist. The statically generated site renders every entry with schema.org Dataset JSON-LD, so the catalogue surfaces in the dataset discovery layers. +A category names a normalisation target: the shape every entry in that category resolves to, whatever its native shape. `graph/v1` is `{ meta, nodes, edges }` with unique node ids and edges whose endpoints name nodes; its entries are graph datasets taken from upstream collections such as SuiteSparse, Netzschleuder, SNAP, KONECT and OGB, read by one parser per format (delimited edge lists, which can also be a sheet of a spreadsheet, Matrix Market, GML, Pajek, UCINet DL full matrices, Graph Drawing contest files, Witvliet connectome synapse files, Open Graph Benchmark raw directories read together from the several files of a dataset's zip, KONECT edge files, rating and count files, knowledge graph triples, Heterogeneous Graph Benchmark node and link files, pragmatic similarity rating workbooks, and network instance files: Canad `.dow` multicommodity network design, OR-Library p-median and DIMACS minimum-cost flow problems and TNTP traffic assignment networks with their trip tables, whose nodes and edges carry the costs, capacities, demands and supplies the files give, DIMACS and PACE challenge graphs, SteinLib Steiner tree problems and MATPOWER power system cases), each giving the graph's topology and, where the format states them, node and edge types, weights, times and the split each edge belongs to. `tasks/v1` is `{ meta, nodes }`, a set of tasks as the places or cases it offers to visit or serve (a routing benchmark, or an open data register of permits and complaints), each flat with optional coordinates, prize, prize probability, duration, travel time, skill, territory, lifecycle status and times, and deadline, read from tabular sources by the `csv-table` parser (a delimited file, a member of an archive, a sheet of an Excel workbook, a legacy binary xls workbook or an OpenDocument spreadsheet, or an open data portal read a page at a time) and from ArcGIS feature layers by the `geojson` parser. Its entries come from a survey of open operations-research datasets, imported with a curation that records, for each source, what its download gives and how its columns map onto the node; a dataset whose source has no usable table is imported without a recipe and says why. Any category can be fetched as a collection: a recipe that selects every matching archive member with `members` (by path, by the extensions its name ends with, or by a regular expression its whole path matches) resolves to `{ meta, instances }`, where each instance is a document of the entry's category under the path of the member it came from, the schema derived from the category's own, so a category needs nothing of its own to be collected. A matched member that is itself an archive is opened by the recipe's `inner` layout, a tar, tar.gz, tar.bz2 or zip with its own `members` selector, and each file that selector matches is one instance, filed under the member's path and its own joined by `/`, so a download whose transcripts sit in one tarball inside another is fetched once and every other member of the outer archive is skipped as it streams past. The parser reads each member, the crosswalk is applied to each, and a member that does not parse or does not validate fails the fetch and is named. A `grouped-member-path` parser groups related archive paths by an identifier and representation, retaining their names as fields without loading their payloads; each group must contain exactly the representations its recipe names. Instance files are read by one parser per format (Solomon and TOPTW time window files, Chao team orienteering files and their dynamic form, TSPLIB style orienteering files, VRP-REP XML, VeRoLog 2019 files, customer point lists, Prodhon location-routing files and the OR-Library capacitated p-median file), which gives each node the columns the file has and each instance a header of its own; a curation says which of them fill the `tasks/v1` node's fields, which pass through under their own names and which are dropped, so what an entry leaves out is written down, and a source whose time quantities are not seconds keeps them under the source's names with the unit recorded as the entry's `timeUnit`. `texts/v1` is `{ meta, nodes }`, a set of texts as the works a collection catalogues (a library's catalogue feed, say), each flat with optional title, language, authors, subjects, issue date, type, classification, collections and the addresses of its files as plain text, a web page and an EPUB, read from tabular sources by the `csv-table` parser (which can build a column from a template over a row's other columns, so an address a source only implies by a scheme over the record's id is one more column to map); it describes the works and where their files are and carries none of their contents. `resources/v1` is `{ meta, nodes }`, one downloadable resource (a speech corpus, a lexicon, a toolkit) as its `meta`, which says what it is (name, identifier, summary, category, licence, description, external address), and the files it is distributed as in `nodes`, each with optional url, size and description; a host's whole listing is a collection with one instance per resource, read page by page by the `openslr-resource` parser; a set of model files is read from its host's listing by the `text-pattern` parser (each match of a pattern a file) or from an archive's member paths by the `member-path` parser, and it describes the resources and carries none of their files. `speech/v1` is `{ meta, nodes }`, a speech corpus as the utterances it transcribes, each flat with optional transcript, speaker, gender, language, audio file name, duration in seconds and split, read from tabular sources by the `csv-table` parser (a transcript table may have no header row, in which case the recipe names its columns, use comma-space as its delimiter so unquoted commas remain in fields, or hold double quotes in its text, in which case the recipe says it is unquoted, or hold line breaks inside its cells without quoting them, in which case the recipe names the columns so a line break ends a record only once it has them all), from files of one utterance per line, as Kaldi, LibriSpeech, Festival and Sphinx write them (the id first, the id last, in Festival's parentheses or after a Sphinx transcript's sentence markers, as the recipe's layout says), by the `utterance-lines` parser, from JSON manifests (one record per line, or one array of records) by the `json-table` parser, and from corpora with one small text file per recording by the `member-text` parser, or one XML file per recording by the `nxt-words` and `eaf` parsers (NXT word files and ELAN annotation files), or one Praat TextGrid per recording by the `textgrid` parser (the long or short text form, in UTF-8 or UTF-16 by the file's byte order mark, the tier named, or every tier whose name a pattern matches, or with neither every tier, read, each row carrying the tier it is on), or one CHAT file per conversation by the `chat` parser (each turn a row with its speaker, text and time marks), or the line-aligned `segments`, YAML and transcript files of each split of a Multilingual TEDx language by the `mtedx` parser (a row for each segment with its speaker, audio file, start, end and duration), or one NIST STM file per recording by the `stm` parser (each segment line of a speaker a row with its channel, times in seconds, label field and text, the reserved gap and excluded-region lines left out), or the `ANNOTATIONS` files of the MUSAN corpus by the `musan-annotations` parser (each line of a music, speech or noise list a row, labelled with the kind of sound and the directory it is in, with the genre, vocals, artist and composer of music and the gender and language of speech), which read the files a `files` archive selects together into one table, a member that is an archive in its turn (annotations shipped as a zip inside a tarball) being unpacked by the archive's `inner` selector, and which a collection of downloads (one archive per split, say) reads each file into one instance; when a file is distributed as consecutive downloads, its recipe names the parts in order and provenance records each part separately; a files archive that sets `concatenate` instead joins the files it selects, in path order, into the one file a parser that reads one file takes (a transcript table split into a file for each chapter); it carries the transcripts and none of the audio, which is in the distribution a `resources/v1` entry lists. `lexicon/v1`, `speakers/v1`, `splits/v1` and `audio/v1` are `{ meta, nodes }` as well: a word list or pronunciation dictionary as one entry per line (word, pronunciation, count), a speaker list as its speakers (gender, nationality, language, set, name), a corpus's division into training, validation and test data as the items each part holds, and a collection of sounds that are not transcribed speech as the clips it annotates (label, speaker, duration, split); PCM WAV files can be read by the `wav-metadata` parser, which records their channel count, sample rate, sample width, duration and any invalid format parameters without carrying the audio; a lexicon is read by the `lexicon-lines` parser, or by the `json-word-counts` parser when it is one JSON object of words and their counts (read member by member in the file's order, a word listed twice being an error), the annotations of a sound collection by the `json-table` parser (an object nested by sound class and then by file name, each record named by its keys) or the `option-list` parser (lists of lines of the form `--name value path`, read together from the files a `files` archive selects), or the `simulated-rir` parser (response lists joined to their room dimensions, receiver position and absorption coefficient), or the `air-mat` parser (recording metadata from the Aachen database's MATLAB files), or the `member-path` parser (recording identifiers and named fields encoded in selected archive paths), and each carries the lists and annotations and none of the words' texts, speakers' recordings or audio. `corpus/v1` is `{ meta, nodes }` as well, a text corpus as the documents it holds, each with its whole text as decoded from the source and an optional title, language, format and encoding; it carries the texts themselves, where `texts/v1` only describes them. Its Project Gutenberg entry reads the zipped tar of every plain-text ebook that Project Gutenberg publishes for bulk use, one book per document named by its ebook number, with the `member-text` parser set to `verbatim`, so each text is the file exactly as distributed, licence header and footer included; the library is far larger than one document can hold, so the recipe is streamed and `keepset fetch` writes it a book at a time. A corpus of Word documents is read by the `word-text` parser, one document per file, each a `.docx` package or a legacy `.doc` file (Word 97 to 2003) told apart by its bytes and read as the plain text of its main story. A new category starts by defining its target schema in `src/targets/` and registering it in the category registry; an entry cannot name a category that does not exist. The statically generated site renders every entry with schema.org Dataset JSON-LD, so the catalogue surfaces in the dataset discovery layers. The Nicolingua speech parser reads the corpus's recording metadata and vocabulary tables together, joining each recording's prompt ID and language to its prompt text. Its `text` is the prompt supplied to the speaker, rather than a verified transcription; the recording's source metadata is retained alongside it. diff --git a/catalogue/entries/huggingface-frontierco-mds.json b/catalogue/entries/huggingface-frontierco-mds.json new file mode 100644 index 00000000..b9d277fe --- /dev/null +++ b/catalogue/entries/huggingface-frontierco-mds.json @@ -0,0 +1,358 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Feng, W. Sun, S. Li, A. Talwalkar, Y. Yang (2025). FrontierCO: real-world and large-scale evaluation of machine learning solvers for combinatorial optimization. arXiv:2505.16952.", + "url": "https://arxiv.org/abs/2505.16952" + }, + { + "text": "PACE 2025: the Parameterized Algorithms and Computational Experiments Challenge, Dominating Set.", + "url": "https://pacechallenge.org/2025/ds/" + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "The minimum dominating set track of FrontierCO (Feng et al., 2025), 60 undirected graphs in the PACE 2025 graph format (`p ds n m`, then one edge per line): 20 easy test instances named after the exact track of the PACE 2025 dominating set challenge, 20 hard test instances named after its heuristic track, and 20 generated Barabasi-Albert validation graphs. Each instance is a graph whose nodes are 1 to n, isolated ones included; FrontierCO's reference value for each instance is in the track's config.py.", + "domain": "Minimum dominating set", + "expectedCounts": {}, + "id": "huggingface-frontierco-mds", + "licence": { + "id": "MIT", + "kind": "spdx" + }, + "name": "FrontierCO minimum dominating set", + "properties": { + "directed": false, + "downloadBytes": 2671331407, + "referenceValueKind": "optimal_scores in norm_score: the domination number where it is known, else the best known value", + "referenceValues": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/config.py" + }, + "recipe": { + "reason": "Its 20 hard test instances hold 164,677,007 edges between them by their problem lines (the largest 12,885,369), and a collection is held whole: the easy and validation instances alone hold 10,239,865 edges, read and checked with the dimacs-graph parser in its pace layout, which is the recipe this entry would use. It becomes fetchable when graph/v1 results can be streamed (https://github.com/Mearman/keepset/issues/305).", + "status": "not-fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260527024426", + "url": "https://web.archive.org/web/20260527024426/https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/config.py" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_033.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_034.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_051.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_056.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_057.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_058.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_063.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_066.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_067.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_069.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_071.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_072.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_075.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_076.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_081.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_083.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_088.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_092.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_093.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/easy_test_instances/exact_097.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_003.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_015.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_016.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_017.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_019.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_026.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_036.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_037.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_042.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_044.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_049.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_059.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_060.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_065.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_066.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_067.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_074.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_077.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_078.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/hard_test_instances/heuristic_097.gr" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_0.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_1.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_10.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_11.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_12.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_13.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_14.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_15.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_16.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_17.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_18.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_19.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_2.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_3.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_4.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_5.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_6.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_7.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_8.txt" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MDS/valid_instances/ba_graph_large_train_9.txt" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "minimum-dominating-set" + ] +} diff --git a/catalogue/entries/huggingface-frontierco-mis.json b/catalogue/entries/huggingface-frontierco-mis.json new file mode 100644 index 00000000..4a61f52c --- /dev/null +++ b/catalogue/entries/huggingface-frontierco-mis.json @@ -0,0 +1,946 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Feng, W. Sun, S. Li, A. Talwalkar, Y. Yang (2025). FrontierCO: real-world and large-scale evaluation of machine learning solvers for combinatorial optimization. arXiv:2505.16952.", + "url": "https://arxiv.org/abs/2505.16952" + }, + { + "doi": "10.1090/dimacs/026", + "text": "D. S. Johnson, M. A. Trick (eds.) (1996). Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge. DIMACS Series in Discrete Mathematics and Theoretical Computer Science 26." + }, + { + "doi": "10.1016/j.artint.2007.04.001", + "text": "K. Xu, F. Boussemart, F. Hemery, C. Lecoutre (2007). Random constraint satisfaction: easy generation of hard (satisfiable) instances. Artificial Intelligence 171(8-9), 514-534." + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "The maximum independent set track of FrontierCO (Feng et al., 2025), 73 undirected graphs in the DIMACS graph format (`p edge`, then `e` lines): 37 easy test instances that are the complements of graphs of the Second DIMACS Implementation Challenge's clique benchmark (brock, C, DSJC, gen, hamming, keller, MANN and p_hat), so that an independent set here is a clique there, 16 hard test instances from BHOSLIB (the frb graphs of Xu et al.'s model RB, with hidden optimum) and 20 validation instances generated from the same RB model. Each instance is a graph whose nodes are 1 to n, isolated ones included, with one edge per line; FrontierCO's reference value for each instance is in the track's config.py.", + "domain": "Maximum independent set", + "expectedCounts": { + "instances": 73 + }, + "id": "huggingface-frontierco-mis", + "licence": { + "id": "MIT", + "kind": "spdx" + }, + "name": "FrontierCO maximum independent set", + "properties": { + "directed": false, + "downloadBytes": 142244252, + "referenceValueKind": "optimal_scores in norm_score: the independence number where it is known, else the best known value", + "referenceValues": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/config.py" + }, + "recipe": { + "files": [ + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C1000.9.mis", + "source": 2 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C125.9.mis", + "source": 3 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C2000.5.mis", + "source": 4 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C2000.9.mis", + "source": 5 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C250.9.mis", + "source": 6 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C4000.5.mis", + "source": 7 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/C500.9.mis", + "source": 8 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/DSJC1000.5.mis", + "source": 9 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/DSJC500.5.mis", + "source": 10 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/MANN_a27.mis", + "source": 11 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/MANN_a45.mis", + "source": 12 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/MANN_a81.mis", + "source": 13 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock200_2.mis", + "source": 14 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock200_4.mis", + "source": 15 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock400_2.mis", + "source": 16 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock400_4.mis", + "source": 17 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock800_2.mis", + "source": 18 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/brock800_4.mis", + "source": 19 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/gen200_p0.9_44.mis", + "source": 20 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/gen200_p0.9_55.mis", + "source": 21 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/gen400_p0.9_55.mis", + "source": 22 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/gen400_p0.9_65.mis", + "source": 23 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/gen400_p0.9_75.mis", + "source": 24 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/hamming10-4.mis", + "source": 25 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/hamming8-4.mis", + "source": 26 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/keller4.mis", + "source": 27 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/keller5.mis", + "source": 28 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/keller6.mis", + "source": 29 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat1500-1.mis", + "source": 30 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat1500-2.mis", + "source": 31 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat1500-3.mis", + "source": 32 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat300-1.mis", + "source": 33 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat300-2.mis", + "source": 34 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat300-3.mis", + "source": 35 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat700-1.mis", + "source": 36 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat700-2.mis", + "source": 37 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/p_hat700-3.mis", + "source": 38 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb100-40.mis", + "source": 39 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb50-23-1.mis", + "source": 40 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb50-23-2.mis", + "source": 41 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb50-23-3.mis", + "source": 42 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb50-23-4.mis", + "source": 43 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb50-23-5.mis", + "source": 44 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb53-24-1.mis", + "source": 45 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb53-24-2.mis", + "source": 46 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb53-24-3.mis", + "source": 47 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb53-24-4.mis", + "source": 48 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb53-24-5.mis", + "source": 49 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb59-26-1.mis", + "source": 50 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb59-26-2.mis", + "source": 51 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb59-26-3.mis", + "source": 52 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb59-26-4.mis", + "source": 53 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/frb59-26-5.mis", + "source": 54 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_0.mis", + "source": 55 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_1.mis", + "source": 56 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_10.mis", + "source": 57 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_11.mis", + "source": 58 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_12.mis", + "source": 59 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_13.mis", + "source": 60 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_14.mis", + "source": 61 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_15.mis", + "source": 62 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_16.mis", + "source": 63 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_17.mis", + "source": 64 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_18.mis", + "source": 65 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_19.mis", + "source": 66 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_2.mis", + "source": 67 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_3.mis", + "source": 68 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_4.mis", + "source": 69 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_5.mis", + "source": 70 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_6.mis", + "source": 71 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_7.mis", + "source": 72 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_8.mis", + "source": 73 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/RB_800_1200_9.mis", + "source": 74 + } + ], + "parser": { + "encoding": "utf-8", + "format": "dimacs-graph", + "layout": "dimacs" + }, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260527024426", + "url": "https://web.archive.org/web/20260527024426/https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/config.py" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C1000.9.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C125.9.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C2000.5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C2000.9.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C250.9.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C4000.5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/C500.9.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/DSJC1000.5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/DSJC500.5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/MANN_a27.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/MANN_a45.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/MANN_a81.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock200_2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock200_4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock400_2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock400_4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock800_2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/brock800_4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/gen200_p0.9_44.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/gen200_p0.9_55.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/gen400_p0.9_55.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/gen400_p0.9_65.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/gen400_p0.9_75.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/hamming10-4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/hamming8-4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/keller4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/keller5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/keller6.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat1500-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat1500-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat1500-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat300-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat300-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat300-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat700-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat700-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/easy_test_instances/p_hat700-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb100-40.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb50-23-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb50-23-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb50-23-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb50-23-4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb50-23-5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb53-24-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb53-24-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb53-24-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb53-24-4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb53-24-5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb59-26-1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb59-26-2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb59-26-3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb59-26-4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/hard_test_instances/frb59-26-5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_0.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_1.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_10.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_11.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_12.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_13.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_14.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_15.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_16.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_17.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_18.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_19.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_2.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_3.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_4.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_5.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_6.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_7.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_8.mis" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/MIS/valid_instances/RB_800_1200_9.mis" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "maximum-independent-set" + ] +} diff --git a/catalogue/entries/huggingface-frontierco-stp.json b/catalogue/entries/huggingface-frontierco-stp.json new file mode 100644 index 00000000..0409c72e --- /dev/null +++ b/catalogue/entries/huggingface-frontierco-stp.json @@ -0,0 +1,1132 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Feng, W. Sun, S. Li, A. Talwalkar, Y. Yang (2025). FrontierCO: real-world and large-scale evaluation of machine learning solvers for combinatorial optimization. arXiv:2505.16952.", + "url": "https://arxiv.org/abs/2505.16952" + }, + { + "doi": "10.1007/978-1-4613-0255-1_9", + "text": "T. Koch, A. Martin, S. Voss (2001). SteinLib: an updated library on Steiner tree problems in graphs. In Steiner Trees in Industry, Combinatorial Optimization 11, 285-325." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "edges[].source", + "edges[].target", + "edges[].weight" + ], + "source": "steinlib", + "target": "graph/v1" + }, + "description": "The Steiner tree problem in graphs track of FrontierCO (Feng et al., 2025), 88 instances in the SteinLib STP format: 23 easy test instances (the G instances, each the presolved reduction of a larger original whose fixed value, bounds and original counts the instance's meta carries), 50 hard test instances from SteinLib's bipartite (bip), code covering (cc) and hypercube (hc) sets, and 15 validation instances (generated hypercube graphs and GeoSteiner's reduced full Steiner tree graphs of random points, R25K). In each, the nodes are 1 to n, those that must be connected of type terminal and the rest of type node, and every edge is undirected with its weight; the Comment section's name, creator and remark are in the meta. FrontierCO's reference value for each instance is in the track's config.py.", + "domain": "Steiner tree problem in graphs", + "expectedCounts": { + "instances": 88 + }, + "id": "huggingface-frontierco-stp", + "licence": { + "id": "MIT", + "kind": "spdx" + }, + "name": "FrontierCO Steiner tree problem in graphs", + "properties": { + "directed": false, + "downloadBytes": 180648556, + "referenceValueKind": "optimal_scores in norm_score: the optimal or best known tree weight", + "referenceValues": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/config.py" + }, + "recipe": { + "files": [ + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G101a.stp", + "source": 2 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G102a.stp", + "source": 3 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G103a.stp", + "source": 4 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G104a.stp", + "source": 5 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G105a.stp", + "source": 6 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G106a.stp", + "source": 7 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G107a.stp", + "source": 8 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G201a.stp", + "source": 9 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G202a.stp", + "source": 10 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G203a.stp", + "source": 11 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G204a.stp", + "source": 12 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G205a.stp", + "source": 13 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G206a.stp", + "source": 14 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G207a.stp", + "source": 15 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G301a.stp", + "source": 16 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G302a.stp", + "source": 17 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G303a.stp", + "source": 18 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G304a.stp", + "source": 19 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G305a.stp", + "source": 20 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G306a.stp", + "source": 21 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G307a.stp", + "source": 22 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G308a.stp", + "source": 23 + }, + { + "archive": { + "kind": "none" + }, + "path": "easy_test_instances/G309a.stp", + "source": 24 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip42p.stp", + "source": 25 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip42u.stp", + "source": 26 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip52p.stp", + "source": 27 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip52u.stp", + "source": 28 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip62p.stp", + "source": 29 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bip62u.stp", + "source": 30 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bipa2p.stp", + "source": 31 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bipa2u.stp", + "source": 32 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bipe2p.stp", + "source": 33 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/bipe2u.stp", + "source": 34 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc10-2p.stp", + "source": 35 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc10-2u.stp", + "source": 36 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc11-2p.stp", + "source": 37 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc11-2u.stp", + "source": 38 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc12-2p.stp", + "source": 39 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc12-2u.stp", + "source": 40 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-10p.stp", + "source": 41 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-10u.stp", + "source": 42 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-11p.stp", + "source": 43 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-11u.stp", + "source": 44 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-12p.stp", + "source": 45 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-12u.stp", + "source": 46 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-4p.stp", + "source": 47 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-4u.stp", + "source": 48 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-5p.stp", + "source": 49 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc3-5u.stp", + "source": 50 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc5-3p.stp", + "source": 51 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc5-3u.stp", + "source": 52 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc6-2p.stp", + "source": 53 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc6-2u.stp", + "source": 54 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc6-3p.stp", + "source": 55 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc6-3u.stp", + "source": 56 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc7-3p.stp", + "source": 57 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc7-3u.stp", + "source": 58 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc9-2p.stp", + "source": 59 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/cc9-2u.stp", + "source": 60 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc10p.stp", + "source": 61 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc10u.stp", + "source": 62 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc11p.stp", + "source": 63 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc11u.stp", + "source": 64 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc12p.stp", + "source": 65 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc12u.stp", + "source": 66 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc6p.stp", + "source": 67 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc6u.stp", + "source": 68 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc7p.stp", + "source": 69 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc7u.stp", + "source": 70 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc8p.stp", + "source": 71 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc8u.stp", + "source": 72 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc9p.stp", + "source": 73 + }, + { + "archive": { + "kind": "none" + }, + "path": "hard_test_instances/hc9u.stp", + "source": 74 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/R25K01EFST.stp", + "source": 75 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/R25K02EFST.stp", + "source": 76 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/R25K03EFST.stp", + "source": 77 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/R25K04EFST.stp", + "source": 78 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/R25K05EFST.stp", + "source": 79 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc10_valid_0p.stp", + "source": 80 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc10_valid_1p.stp", + "source": 81 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc6_valid_0p.stp", + "source": 82 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc6_valid_1p.stp", + "source": 83 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc7_valid_0p.stp", + "source": 84 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc7_valid_1p.stp", + "source": 85 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc8_valid_0p.stp", + "source": 86 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc8_valid_1p.stp", + "source": 87 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc9_valid_0p.stp", + "source": 88 + }, + { + "archive": { + "kind": "none" + }, + "path": "valid_instances/hc9_valid_1p.stp", + "source": 89 + } + ], + "parser": { + "encoding": "utf-8", + "format": "steinlib" + }, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260527024426", + "url": "https://web.archive.org/web/20260527024426/https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/config.py" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G101a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G102a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G103a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G104a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G105a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G106a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G107a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G201a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G202a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G203a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G204a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G205a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G206a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G207a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G301a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G302a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G303a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G304a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G305a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G306a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G307a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G308a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/easy_test_instances/G309a.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip42p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip42u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip52p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip52u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip62p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bip62u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bipa2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bipa2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bipe2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/bipe2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc10-2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc10-2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc11-2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc11-2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc12-2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc12-2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-10p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-10u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-11p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-11u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-12p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-12u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-4p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-4u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-5p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc3-5u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc5-3p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc5-3u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc6-2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc6-2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc6-3p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc6-3u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc7-3p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc7-3u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc9-2p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/cc9-2u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc10p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc10u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc11p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc11u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc12p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc12u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc6p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc6u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc7p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc7u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc8p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc8u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc9p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/hard_test_instances/hc9u.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/R25K01EFST.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/R25K02EFST.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/R25K03EFST.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/R25K04EFST.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/R25K05EFST.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc10_valid_0p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc10_valid_1p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc6_valid_0p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc6_valid_1p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc7_valid_0p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc7_valid_1p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc8_valid_0p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc8_valid_1p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc9_valid_0p.stp" + }, + { + "retrieved": "2026-11-11", + "status": "live", + "url": "https://huggingface.co/datasets/CO-Bench/FrontierCO/resolve/e92b9d60757ac4a6274c4b27f40e03fc70c60061/STP/valid_instances/hc9_valid_1p.stp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "steiner-tree", + "typed", + "weight" + ] +} diff --git a/catalogue/entries/pglib-opf-api.json b/catalogue/entries/pglib-opf-api.json new file mode 100644 index 00000000..073d04d1 --- /dev/null +++ b/catalogue/entries/pglib-opf-api.json @@ -0,0 +1,149 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. DeMarco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, C. Josz, R. Korab, B. Lesieutre, J. Maeght, T. W. K. Mak, D. K. Molzahn, T. J. Overbye, P. Panciatici, B. Park, J. Snodgrass, A. Tbaileh, P. Van Hentenryck, R. Zimmerman (2019). The Power Grid Library for benchmarking AC optimal power flow algorithms. arXiv:1908.02788.", + "url": "https://arxiv.org/abs/1908.02788" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].busType", + "nodes[].Pd", + "nodes[].Qd", + "nodes[].Gs", + "nodes[].Bs", + "nodes[].area", + "nodes[].Vm", + "nodes[].Va", + "nodes[].baseKV", + "nodes[].zone", + "nodes[].Vmax", + "nodes[].Vmin", + "nodes[].Pg", + "nodes[].Qg", + "nodes[].Qmax", + "nodes[].Qmin", + "nodes[].Vg", + "nodes[].mBase", + "nodes[].status", + "nodes[].Pmax", + "nodes[].Pmin", + "nodes[].Pc1", + "nodes[].Pc2", + "nodes[].Qc1min", + "nodes[].Qc1max", + "nodes[].Qc2min", + "nodes[].Qc2max", + "nodes[].ramp_agc", + "nodes[].ramp_10", + "nodes[].ramp_30", + "nodes[].ramp_q", + "nodes[].apf", + "nodes[].startup", + "nodes[].shutdown", + "nodes[].c2", + "nodes[].c1", + "nodes[].c0", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].r", + "edges[].x", + "edges[].b", + "edges[].rateA", + "edges[].rateB", + "edges[].rateC", + "edges[].ratio", + "edges[].angle", + "edges[].branchStatus", + "edges[].angmin", + "edges[].angmax" + ], + "source": "matpower", + "target": "graph/v1" + }, + "description": "The heavily loaded (API) group of PGLib-OPF v23.07, the IEEE PES Power Grid Library's benchmark cases for AC optimal power flow: 66 power system cases, heavily loaded cases, in which thermal limits bind, each a MATPOWER case file of the repository at commit dc6be4b2f85c. In each case the buses are nodes of type bus with the bus table's columns, the generators nodes of type generator with the generator table's columns and their quadratic costs, joined to their buses by edges of type generator, and the branches edges of type branch with their impedances, ratings, tap ratios and angle limits; the case's name, base MVA and any legacy area table are in its meta. Each file's header names the source documents of its network. Reference objective values (the AC and DC optima PowerModels.jl reached and the QC and SOC relaxation gaps) are in the repository's BASELINE.md, section Congested Operating Conditions (API), which this entry lists as a source and does not merge.", + "domain": "Optimal power flow", + "expectedCounts": { + "instances": 66 + }, + "id": "pglib-opf-api", + "licence": { + "id": "CC-BY-4.0", + "kind": "spdx" + }, + "name": "PGLib-OPF heavily loaded (API) cases", + "namespace": [ + "pglib" + ], + "properties": { + "directed": false, + "downloadBytes": 49765204, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "referenceValueKind": "AC and DC optimal objective values (dollars per hour) reached by PowerModels.jl with IPOPT, and QC and SOC relaxation gaps", + "referenceValues": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "recipe": { + "archive": { + "kind": "zip", + "members": { + "pattern": "pglib-opf-dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/api/pglib_opf_case[^/]*\\.m" + } + }, + "parser": { + "encoding": "utf-8", + "format": "matpower" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 57104, + "sha256": "2c5db8843d6ea2d55ebaa6945d421d29e6e95272e7000f9643ce4a3f18604288", + "timestamp": "20260607092944", + "url": "https://web.archive.org/web/20260607092944/https://github.com/power-grid-lib/pglib-opf" + }, + "status": "live", + "url": "https://github.com/power-grid-lib/pglib-opf" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 49765204, + "sha256": "281ad37cc859ff453b28f1a363b549b693af0c12b6f0228ae5627b2aa978386b", + "timestamp": "20261111210642", + "url": "https://web.archive.org/web/20261111210642/https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + "status": "live", + "url": "https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 5957, + "sha256": "41f7358e8b0eeb2dd05833f15feab648475cfca5a53c37529530e3ec8f613827", + "timestamp": "20261111210813", + "url": "https://web.archive.org/web/20261111210813/https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "power-flow", + "typed" + ] +} diff --git a/catalogue/entries/pglib-opf-sad.json b/catalogue/entries/pglib-opf-sad.json new file mode 100644 index 00000000..540fc287 --- /dev/null +++ b/catalogue/entries/pglib-opf-sad.json @@ -0,0 +1,149 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. DeMarco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, C. Josz, R. Korab, B. Lesieutre, J. Maeght, T. W. K. Mak, D. K. Molzahn, T. J. Overbye, P. Panciatici, B. Park, J. Snodgrass, A. Tbaileh, P. Van Hentenryck, R. Zimmerman (2019). The Power Grid Library for benchmarking AC optimal power flow algorithms. arXiv:1908.02788.", + "url": "https://arxiv.org/abs/1908.02788" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].busType", + "nodes[].Pd", + "nodes[].Qd", + "nodes[].Gs", + "nodes[].Bs", + "nodes[].area", + "nodes[].Vm", + "nodes[].Va", + "nodes[].baseKV", + "nodes[].zone", + "nodes[].Vmax", + "nodes[].Vmin", + "nodes[].Pg", + "nodes[].Qg", + "nodes[].Qmax", + "nodes[].Qmin", + "nodes[].Vg", + "nodes[].mBase", + "nodes[].status", + "nodes[].Pmax", + "nodes[].Pmin", + "nodes[].Pc1", + "nodes[].Pc2", + "nodes[].Qc1min", + "nodes[].Qc1max", + "nodes[].Qc2min", + "nodes[].Qc2max", + "nodes[].ramp_agc", + "nodes[].ramp_10", + "nodes[].ramp_30", + "nodes[].ramp_q", + "nodes[].apf", + "nodes[].startup", + "nodes[].shutdown", + "nodes[].c2", + "nodes[].c1", + "nodes[].c0", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].r", + "edges[].x", + "edges[].b", + "edges[].rateA", + "edges[].rateB", + "edges[].rateC", + "edges[].ratio", + "edges[].angle", + "edges[].branchStatus", + "edges[].angmin", + "edges[].angmax" + ], + "source": "matpower", + "target": "graph/v1" + }, + "description": "The small phase angle difference (SAD) group of PGLib-OPF v23.07, the IEEE PES Power Grid Library's benchmark cases for AC optimal power flow: 66 power system cases, cases with small phase angle difference limits, which bind, each a MATPOWER case file of the repository at commit dc6be4b2f85c. In each case the buses are nodes of type bus with the bus table's columns, the generators nodes of type generator with the generator table's columns and their quadratic costs, joined to their buses by edges of type generator, and the branches edges of type branch with their impedances, ratings, tap ratios and angle limits; the case's name, base MVA and any legacy area table are in its meta. Each file's header names the source documents of its network. Reference objective values (the AC and DC optima PowerModels.jl reached and the QC and SOC relaxation gaps) are in the repository's BASELINE.md, section Small Angle Difference Conditions (SAD), which this entry lists as a source and does not merge.", + "domain": "Optimal power flow", + "expectedCounts": { + "instances": 66 + }, + "id": "pglib-opf-sad", + "licence": { + "id": "CC-BY-4.0", + "kind": "spdx" + }, + "name": "PGLib-OPF small phase angle difference (SAD) cases", + "namespace": [ + "pglib" + ], + "properties": { + "directed": false, + "downloadBytes": 49765204, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "referenceValueKind": "AC and DC optimal objective values (dollars per hour) reached by PowerModels.jl with IPOPT, and QC and SOC relaxation gaps", + "referenceValues": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "recipe": { + "archive": { + "kind": "zip", + "members": { + "pattern": "pglib-opf-dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/sad/pglib_opf_case[^/]*\\.m" + } + }, + "parser": { + "encoding": "utf-8", + "format": "matpower" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 57104, + "sha256": "2c5db8843d6ea2d55ebaa6945d421d29e6e95272e7000f9643ce4a3f18604288", + "timestamp": "20260607092944", + "url": "https://web.archive.org/web/20260607092944/https://github.com/power-grid-lib/pglib-opf" + }, + "status": "live", + "url": "https://github.com/power-grid-lib/pglib-opf" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 49765204, + "sha256": "281ad37cc859ff453b28f1a363b549b693af0c12b6f0228ae5627b2aa978386b", + "timestamp": "20261111210642", + "url": "https://web.archive.org/web/20261111210642/https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + "status": "live", + "url": "https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 5957, + "sha256": "41f7358e8b0eeb2dd05833f15feab648475cfca5a53c37529530e3ec8f613827", + "timestamp": "20261111210813", + "url": "https://web.archive.org/web/20261111210813/https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "power-flow", + "typed" + ] +} diff --git a/catalogue/entries/pglib-opf-typical.json b/catalogue/entries/pglib-opf-typical.json new file mode 100644 index 00000000..eb376c6f --- /dev/null +++ b/catalogue/entries/pglib-opf-typical.json @@ -0,0 +1,149 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. DeMarco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, C. Josz, R. Korab, B. Lesieutre, J. Maeght, T. W. K. Mak, D. K. Molzahn, T. J. Overbye, P. Panciatici, B. Park, J. Snodgrass, A. Tbaileh, P. Van Hentenryck, R. Zimmerman (2019). The Power Grid Library for benchmarking AC optimal power flow algorithms. arXiv:1908.02788.", + "url": "https://arxiv.org/abs/1908.02788" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].busType", + "nodes[].Pd", + "nodes[].Qd", + "nodes[].Gs", + "nodes[].Bs", + "nodes[].area", + "nodes[].Vm", + "nodes[].Va", + "nodes[].baseKV", + "nodes[].zone", + "nodes[].Vmax", + "nodes[].Vmin", + "nodes[].Pg", + "nodes[].Qg", + "nodes[].Qmax", + "nodes[].Qmin", + "nodes[].Vg", + "nodes[].mBase", + "nodes[].status", + "nodes[].Pmax", + "nodes[].Pmin", + "nodes[].Pc1", + "nodes[].Pc2", + "nodes[].Qc1min", + "nodes[].Qc1max", + "nodes[].Qc2min", + "nodes[].Qc2max", + "nodes[].ramp_agc", + "nodes[].ramp_10", + "nodes[].ramp_30", + "nodes[].ramp_q", + "nodes[].apf", + "nodes[].startup", + "nodes[].shutdown", + "nodes[].c2", + "nodes[].c1", + "nodes[].c0", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].r", + "edges[].x", + "edges[].b", + "edges[].rateA", + "edges[].rateB", + "edges[].rateC", + "edges[].ratio", + "edges[].angle", + "edges[].branchStatus", + "edges[].angmin", + "edges[].angmax" + ], + "source": "matpower", + "target": "graph/v1" + }, + "description": "The typical operating conditions group of PGLib-OPF v23.07, the IEEE PES Power Grid Library's benchmark cases for AC optimal power flow: 66 power system cases, the base cases as originally specified, each a MATPOWER case file of the repository at commit dc6be4b2f85c. In each case the buses are nodes of type bus with the bus table's columns, the generators nodes of type generator with the generator table's columns and their quadratic costs, joined to their buses by edges of type generator, and the branches edges of type branch with their impedances, ratings, tap ratios and angle limits; the case's name, base MVA and any legacy area table are in its meta. Each file's header names the source documents of its network. Reference objective values (the AC and DC optima PowerModels.jl reached and the QC and SOC relaxation gaps) are in the repository's BASELINE.md, section Typical Operating Conditions (TYP), which this entry lists as a source and does not merge.", + "domain": "Optimal power flow", + "expectedCounts": { + "instances": 66 + }, + "id": "pglib-opf-typical", + "licence": { + "id": "CC-BY-4.0", + "kind": "spdx" + }, + "name": "PGLib-OPF typical operating conditions cases", + "namespace": [ + "pglib" + ], + "properties": { + "directed": false, + "downloadBytes": 49765204, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "referenceValueKind": "AC and DC optimal objective values (dollars per hour) reached by PowerModels.jl with IPOPT, and QC and SOC relaxation gaps", + "referenceValues": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "recipe": { + "archive": { + "kind": "zip", + "members": { + "pattern": "pglib-opf-dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/pglib_opf_case[^/]*\\.m" + } + }, + "parser": { + "encoding": "utf-8", + "format": "matpower" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 57104, + "sha256": "2c5db8843d6ea2d55ebaa6945d421d29e6e95272e7000f9643ce4a3f18604288", + "timestamp": "20260607092944", + "url": "https://web.archive.org/web/20260607092944/https://github.com/power-grid-lib/pglib-opf" + }, + "status": "live", + "url": "https://github.com/power-grid-lib/pglib-opf" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 49765204, + "sha256": "281ad37cc859ff453b28f1a363b549b693af0c12b6f0228ae5627b2aa978386b", + "timestamp": "20261111210642", + "url": "https://web.archive.org/web/20261111210642/https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + "status": "live", + "url": "https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 5957, + "sha256": "41f7358e8b0eeb2dd05833f15feab648475cfca5a53c37529530e3ec8f613827", + "timestamp": "20261111210813", + "url": "https://web.archive.org/web/20261111210813/https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "power-flow", + "typed" + ] +} diff --git a/catalogue/entries/tntp-anaheim.json b/catalogue/entries/tntp-anaheim.json new file mode 100644 index 00000000..b25db83d --- /dev/null +++ b/catalogue/entries/tntp-anaheim.json @@ -0,0 +1,131 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The 1992 Anaheim, California network provided by J. Ban and R. Jayakrishnan (times in minutes, lengths in feet). From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 38 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's Anaheim_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 3726, + "nodes": 1822 + }, + "id": "tntp-anaheim", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Anaheim traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_flow.tntp", + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 16059, + "sha256": "1bb18564598271b60859a01875bdbd8581f3d04967b47fb9feeb4eb2e9ddc616", + "timestamp": "20261111203038", + "url": "https://web.archive.org/web/20261111203038/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-austin.json b/catalogue/entries/tntp-austin.json new file mode 100644 index 00000000..9e25f2cb --- /dev/null +++ b/catalogue/entries/tntp-austin.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The 2005 Austin, Texas network provided by C. Xie, with its AM peak hour demand; the network keeps the five duplicate links the repository's readme lists. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and every node is a zone; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 2514339, + "nodes": 1255077 + }, + "id": "tntp-austin", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Austin traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Austin/Austin_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Austin/Austin_trips_am.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-barcelona.json b/catalogue/entries/tntp-barcelona.json new file mode 100644 index 00000000..ace0b111 --- /dev/null +++ b/catalogue/entries/tntp-barcelona.json @@ -0,0 +1,131 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Barcelona network. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 110 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's Barcelona_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 18366, + "nodes": 8942 + }, + "id": "tntp-barcelona", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Barcelona traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_flow.tntp", + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 39724, + "sha256": "1b2e2aea1e5df4807fc625ce427eb058d884bc866b223b74bb2e9b583599e657", + "timestamp": "20261111203053", + "url": "https://web.archive.org/web/20261111203053/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-center.json b/catalogue/entries/tntp-berlin-center.json new file mode 100644 index 00000000..fec13c66 --- /dev/null +++ b/catalogue/entries/tntp-berlin-center.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Berlin Center network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version; the network keeps the six duplicate links the repository's readme lists. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 865 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 127752, + "nodes": 62669 + }, + "id": "tntp-berlin-center", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Center traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Center/berlin-center_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Center/berlin-center_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-friedrichshain.json b/catalogue/entries/tntp-berlin-friedrichshain.json new file mode 100644 index 00000000..b07bf14c --- /dev/null +++ b/catalogue/entries/tntp-berlin-friedrichshain.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Berlin Friedrichshain network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 23 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 1535, + "nodes": 730 + }, + "id": "tntp-berlin-friedrichshain", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Friedrichshain traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Friedrichshain/friedrichshain-center_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Friedrichshain/friedrichshain-center_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-mitte-center.json b/catalogue/entries/tntp-berlin-mitte-center.json new file mode 100644 index 00000000..2fa50b31 --- /dev/null +++ b/catalogue/entries/tntp-berlin-mitte-center.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Berlin Mitte Center network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 36 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 3391, + "nodes": 1658 + }, + "id": "tntp-berlin-mitte-center", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Mitte Center traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Mitte-Center/berlin-mitte-center_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Mitte-Center/berlin-mitte-center_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-mitte-prenzlauerberg-friedrichshain-center.json b/catalogue/entries/tntp-berlin-mitte-prenzlauerberg-friedrichshain-center.json new file mode 100644 index 00000000..d72967dc --- /dev/null +++ b/catalogue/entries/tntp-berlin-mitte-prenzlauerberg-friedrichshain-center.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The combined Berlin Mitte, Prenzlauerberg and Friedrichshain Center network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 98 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 21194, + "nodes": 10480 + }, + "id": "tntp-berlin-mitte-prenzlauerberg-friedrichshain-center", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Mitte, Prenzlauerberg and Friedrichshain Center traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Mitte-Prenzlauerberg-Friedrichshain-Center/berlin-mitte-prenzlauerberg-friedrichshain-center_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Mitte-Prenzlauerberg-Friedrichshain-Center/berlin-mitte-prenzlauerberg-friedrichshain-center_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-prenzlauerberg-center.json b/catalogue/entries/tntp-berlin-prenzlauerberg-center.json new file mode 100644 index 00000000..a632df69 --- /dev/null +++ b/catalogue/entries/tntp-berlin-prenzlauerberg-center.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Berlin Prenzlauerberg Center network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 38 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 3561, + "nodes": 1758 + }, + "id": "tntp-berlin-prenzlauerberg-center", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Prenzlauerberg Center traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Prenzlauerberg-Center/berlin-prenzlauerberg-center_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Prenzlauerberg-Center/berlin-prenzlauerberg-center_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-berlin-tiergarten.json b/catalogue/entries/tntp-berlin-tiergarten.json new file mode 100644 index 00000000..b582d612 --- /dev/null +++ b/catalogue/entries/tntp-berlin-tiergarten.json @@ -0,0 +1,122 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1287/opre.1040.0197", + "text": "O. Jahn, R. H. Moehring, A. S. Schulz, N. E. Stier-Moses (2005). System-optimal routing of traffic flows with user constraints in networks with congestion. Operations Research 53(4), 600-616." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Berlin Tiergarten network, one of the Berlin networks provided by R. Moehring, A. Schulz and N. Stier-Moses and used by Jahn et al. (2005), in H. Bar-Gera's TNTP version. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 26 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 2054, + "nodes": 1005 + }, + "id": "tntp-berlin-tiergarten", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Berlin Tiergarten traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Tiergarten/berlin-tiergarten_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Berlin-Tiergarten/berlin-tiergarten_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-birmingham.json b/catalogue/entries/tntp-birmingham.json new file mode 100644 index 00000000..db52baf5 --- /dev/null +++ b/catalogue/entries/tntp-birmingham.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "A modified version of the PRISM model of the Birmingham city region in England, converted by J. Xie and Y. Nie. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 898 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 977211, + "nodes": 486276 + }, + "id": "tntp-birmingham", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Birmingham traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Birmingham-England/Birmingham_Net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Birmingham-England/Birmingham_Trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-braess.json b/catalogue/entries/tntp-braess.json new file mode 100644 index 00000000..9a9e76ab --- /dev/null +++ b/catalogue/entries/tntp-braess.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The four-node network of Braess's paradox, prepared by A. Koh. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 2 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 9, + "nodes": 6 + }, + "id": "tntp-braess", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Braess example traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Braess-Example/Braess_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Braess-Example/Braess_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-chicago-regional.json b/catalogue/entries/tntp-chicago-regional.json new file mode 100644 index 00000000..a224a686 --- /dev/null +++ b/catalogue/entries/tntp-chicago-regional.json @@ -0,0 +1,89 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "A large, detailed network of the Chicago region developed by the Chicago Area Transportation Study, with its basic trip table (times in minutes, lengths in miles, tolls in cents). From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 1790 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's ChicagoRegional_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": {}, + "id": "tntp-chicago-regional", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Chicago Regional traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/chicago-regional/ChicagoRegional_flow.tntp", + "selfLoops": false + }, + "recipe": { + "reason": "Its trip table lists 3,136,441 origin and destination pairs, so the graph holds 3,149,423 nodes and 6,311,900 edges, and a round trip of it read whole ran out of the default Node heap; the largest network of this collection round-tripped whole is tntp-austin, with 2,514,339 edges. The recipe it would use is the parts archive of the other entries, reading chicago-regional/ChicagoRegional_net.tntp and chicago-regional/ChicagoRegional_trips.tntp. It becomes fetchable when graph/v1 results can be streamed (https://github.com/Mearman/keepset/issues/305).", + "status": "not-fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 799918, + "sha256": "62376a794eaf917427ba88c419d3331fee568e335a0a83707b24c1d825aaab9b", + "timestamp": "20261111203211", + "url": "https://web.archive.org/web/20261111203211/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/chicago-regional/ChicagoRegional_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/chicago-regional/ChicagoRegional_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-chicago-sketch.json b/catalogue/entries/tntp-chicago-sketch.json new file mode 100644 index 00000000..f831fa7a --- /dev/null +++ b/catalogue/entries/tntp-chicago-sketch.json @@ -0,0 +1,134 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "text": "R. W. Eash, K. S. Chon, Y. J. Lee, D. E. Boyce (1983). Equilibrium traffic assignment on an aggregated highway network for sketch planning. Transportation Research Record 994, 30-37." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "An aggregated sketch planning network of the Chicago region developed by the Chicago Area Transportation Study (times in minutes, lengths in miles, tolls in cents); the repository's readme recommends doubling the trip table for algorithm testing, which this entry does not do. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 387 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's ChicagoSketch_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 288730, + "nodes": 143823 + }, + "id": "tntp-chicago-sketch", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Chicago Sketch traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_flow.tntp", + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 56398, + "sha256": "d6fea5ad22063b42bf0fd74a80b9f576e52657204ca8b49595eff36229746aae", + "timestamp": "20261111203113", + "url": "https://web.archive.org/web/20261111203113/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-eastern-massachusetts.json b/catalogue/entries/tntp-eastern-massachusetts.json new file mode 100644 index 00000000..38b12cb3 --- /dev/null +++ b/catalogue/entries/tntp-eastern-massachusetts.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "A highway subnetwork of the Eastern Massachusetts network for the PM period of April 2012, from the InverseVIsTraffic project, with origin-destination demand estimated from traffic data. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and every node is a zone; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 11210, + "nodes": 5550 + }, + "id": "tntp-eastern-massachusetts", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Eastern Massachusetts traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Eastern-Massachusetts/EMA_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Eastern-Massachusetts/EMA_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-gold-coast.json b/catalogue/entries/tntp-gold-coast.json new file mode 100644 index 00000000..8fe5b0d8 --- /dev/null +++ b/catalogue/entries/tntp-gold-coast.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].criticalSpeed", + "edges[].lanes" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Gold Coast, Australia network from Veitch Lister Consultancy, provided by M. Bliemer and converted by D. Rey, as revised in January 2016; its links carry critical speed and lane count columns in place of toll and link type. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, criticalSpeed, lanes, and nodes 1 to 1068 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 2292388, + "nodes": 1145431 + }, + "id": "tntp-gold-coast", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Gold Coast traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/GoldCoast/Goldcoast_network_2016_01.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/GoldCoast/Goldcoast_trips_2016_01.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "criticalSpeed", + "lanes" + ] +} diff --git a/catalogue/entries/tntp-hessen-asymmetric.json b/catalogue/entries/tntp-hessen-asymmetric.json new file mode 100644 index 00000000..a6119ffc --- /dev/null +++ b/catalogue/entries/tntp-hessen-asymmetric.json @@ -0,0 +1,62 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "The Hessen network provided by E. Codina Sancho of the Universitat Politecnica de Catalunya, whose asymmetric link cost functions are described in the repository's AsymmetricCostFunctions_Codina.pdf; the links here carry the symmetric BPR columns only. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection: a directed road network with link capacities and free-flow times and an origin-destination trip table, in the files Hessen-Asym_net.tntp and Hessen-Asym_trips.tntp of the repository at commit d1639b4ef218.", + "domain": "Traffic assignment", + "expectedCounts": {}, + "id": "tntp-hessen-asymmetric", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Hessen asymmetric traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409 + }, + "recipe": { + "reason": "Its network file's column header names nine columns (init_node to toll) but every link line holds ten values, so which column each value belongs to is not stated and the tntp parser refuses the file. It becomes fetchable when the source's header names every column.", + "status": "not-fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment" + ] +} diff --git a/catalogue/entries/tntp-munich.json b/catalogue/entries/tntp-munich.json new file mode 100644 index 00000000..fd1edc55 --- /dev/null +++ b/catalogue/entries/tntp-munich.json @@ -0,0 +1,62 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "The Munich network. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection: a directed road network with link capacities and free-flow times and an origin-destination trip table, in the files munich_net.tntp and munich_trips.tntp of the repository at commit d1639b4ef218.", + "domain": "Traffic assignment", + "expectedCounts": {}, + "id": "tntp-munich", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Munich traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409 + }, + "recipe": { + "reason": "Its files do not follow the TNTP numbering: the network has no , one link line has another number of values than the header, and the trip table names destination zones such as 2146237932 outside 1 to its 742 zones, so the tntp parser refuses them. It becomes fetchable when the source renumbers its zones and nodes from 1.", + "status": "not-fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment" + ] +} diff --git a/catalogue/entries/tntp-philadelphia.json b/catalogue/entries/tntp-philadelphia.json new file mode 100644 index 00000000..2eba73af --- /dev/null +++ b/catalogue/entries/tntp-philadelphia.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "A large network of the Delaware Valley region provided by the Delaware Valley Regional Planning Commission, with 24 hour demand between its zones (capacities per day, times in minutes, lengths in miles, tolls in cents). From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 1525 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 2342335, + "nodes": 1164555 + }, + "id": "tntp-philadelphia", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Philadelphia traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Philadelphia/Philadelphia_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Philadelphia/Philadelphia_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-sioux-falls.json b/catalogue/entries/tntp-sioux-falls.json new file mode 100644 index 00000000..9cbea515 --- /dev/null +++ b/catalogue/entries/tntp-sioux-falls.json @@ -0,0 +1,135 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "doi": "10.1016/0041-1647(75)90030-1", + "text": "L. J. LeBlanc, E. K. Morlok, W. P. Pierskalla (1975). An efficient approach to solving the road network equilibrium traffic assignment problem. Transportation Research 9(5), 309-318." + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Sioux Falls network of LeBlanc, Morlok and Pierskalla (1975), with the paper's daily flows multiplied by 100 (so 0.1 of the daily flows) and capacities computed for B = 0.15; the repository's readme notes that it is not a realistic network. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and every node is a zone; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's SiouxFalls_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 1228, + "nodes": 600 + }, + "id": "tntp-sioux-falls", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Sioux Falls traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 2, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_flow.tntp", + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 1734, + "sha256": "09ebb1d4ed86e70af531710a18859454b192ae669df01f4b3f8b9e1ce0360301", + "timestamp": "20261111203131", + "url": "https://web.archive.org/web/20261111203131/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-sydney.json b/catalogue/entries/tntp-sydney.json new file mode 100644 index 00000000..73be9cfc --- /dev/null +++ b/catalogue/entries/tntp-sydney.json @@ -0,0 +1,62 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "file": "crosswalks/graph-v1/topology.json" + }, + "description": "The Sydney, Australia network from Veitch Lister Consultancy, provided by M. Bliemer and converted by D. Rey. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection: a directed road network with link capacities and free-flow times and an origin-destination trip table, in the files Sydney_net.tntp and Sydney_trips.tntp.zip of the repository at commit d1639b4ef218.", + "domain": "Traffic assignment", + "expectedCounts": {}, + "id": "tntp-sydney", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Sydney traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409 + }, + "recipe": { + "reason": "Its link lines are not ended by the ; the TNTP format requires, and its trip table is distributed only as a zip inside the repository (Sydney_trips.tntp.zip), which a parts archive does not unpack. It becomes fetchable when the source terminates its link lines and keepset reads a part from an archive inside the download.", + "status": "not-fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment" + ] +} diff --git a/catalogue/entries/tntp-terrassa-asymmetric.json b/catalogue/entries/tntp-terrassa-asymmetric.json new file mode 100644 index 00000000..3b64f132 --- /dev/null +++ b/catalogue/entries/tntp-terrassa-asymmetric.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Terrassa network provided by E. Codina Sancho of the Universitat Politecnica de Catalunya, whose asymmetric link cost functions are described in the repository's AsymmetricCostFunctions_Codina.pdf; the links here carry the symmetric BPR columns only. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 55 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 7694, + "nodes": 3824 + }, + "id": "tntp-terrassa-asymmetric", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Terrassa asymmetric traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Terrassa-Asymmetric/Terrassa-Asym_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Terrassa-Asymmetric/Terrassa-Asym_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-winnipeg-asymmetric.json b/catalogue/entries/tntp-winnipeg-asymmetric.json new file mode 100644 index 00000000..d481da34 --- /dev/null +++ b/catalogue/entries/tntp-winnipeg-asymmetric.json @@ -0,0 +1,118 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "The Winnipeg network provided by E. Codina Sancho of the Universitat Politecnica de Catalunya, whose asymmetric link cost functions are described in the repository's AsymmetricCostFunctions_Codina.pdf; the links here carry the symmetric BPR columns only. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 154 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 11225, + "nodes": 5402 + }, + "id": "tntp-winnipeg-asymmetric", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Winnipeg asymmetric traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": false, + "nodeTypes": 3, + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg-Asymmetric/Winnipeg-Asym_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg-Asymmetric/Winnipeg-Asym_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/catalogue/entries/tntp-winnipeg.json b/catalogue/entries/tntp-winnipeg.json new file mode 100644 index 00000000..ddd44dcb --- /dev/null +++ b/catalogue/entries/tntp-winnipeg.json @@ -0,0 +1,131 @@ +{ + "$schema": "https://cdn.jsdelivr.net/npm/keepset/schemas/entry.schema.json", + "category": "graph/v1", + "citations": [ + { + "text": "Transportation Networks for Research Core Team. Transportation Networks for Research.", + "url": "https://github.com/bstabler/TransportationNetworks" + } + ], + "crosswalk": { + "keep": [ + "meta", + "nodes[].id", + "nodes[].type", + "nodes[].demand", + "edges[].source", + "edges[].target", + "edges[].type", + "edges[].capacity", + "edges[].length", + "edges[].freeFlowTime", + "edges[].b", + "edges[].power", + "edges[].speed", + "edges[].toll", + "edges[].linkType" + ], + "source": "tntp", + "target": "graph/v1" + }, + "description": "One version of the Winnipeg network, in which capacity is set to 1 and B holds B divided by capacity to the power. From Transportation Networks for Research, the traffic assignment test networks that continue H. Bar-Gera's TNTP collection, read from the repository at commit d1639b4ef218 in the TNTP format: the network file's links are directed edges of type link carrying capacity, length, freeFlowTime, b, power, speed, toll, linkType, and nodes 1 to 147 are zones (type zone) and the rest of type node; each origin-destination entry of the trip table is a node of type commodity carrying its demand, joined by an origin edge and a destination edge to its two zones, zero entries included as the file lists them. The node coordinate files are not read. The best known equilibrium link flows are in the repository's Winnipeg_flow.tntp, which this entry lists as a source and does not merge.", + "domain": "Traffic assignment", + "expectedCounts": { + "edges": 11526, + "nodes": 5397 + }, + "id": "tntp-winnipeg", + "licence": { + "kind": "custom", + "name": "Transportation Networks for Research terms: academic research purposes only, with the source indicated in any publication", + "url": "https://github.com/bstabler/TransportationNetworks/blob/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/README.md#license" + }, + "name": "TNTP Winnipeg traffic assignment network", + "namespace": [ + "tntp" + ], + "properties": { + "directed": true, + "downloadBytes": 58755409, + "heterogeneous": true, + "multigraph": true, + "nodeTypes": 3, + "referenceValueKind": "best known user-equilibrium link flows, with the optimal objective value in the network's readme", + "referenceValues": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_flow.tntp", + "selfLoops": false + }, + "recipe": { + "archive": { + "kind": "zip", + "parts": [ + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_net.tntp" + }, + "name": "network" + }, + { + "member": { + "path": "TransportationNetworks-d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_trips.tntp" + }, + "name": "trips" + } + ] + }, + "parser": { + "encoding": "utf-8", + "format": "tntp" + }, + "source": 1, + "status": "fetchable" + }, + "sources": [ + { + "retrieved": "2026-11-11", + "snapshot": { + "timestamp": "20260530135004", + "url": "https://web.archive.org/web/20260530135004/https://github.com/bstabler/TransportationNetworks" + }, + "status": "live", + "url": "https://github.com/bstabler/TransportationNetworks" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 58755409, + "sha256": "0bfb1f9593fccc23fd07c174ac5c20da31fc759081f9d2b89e48ec07846138e7", + "timestamp": "20261111203011", + "url": "https://web.archive.org/web/20261111203011/https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + "status": "live", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "retrieved": "2026-11-11", + "snapshot": { + "byteLength": 38859, + "sha256": "30b79e11c51f1a1c0d367c8389ba707b7ae6e007ee466a7054437bbe9044380f", + "timestamp": "20261111203232", + "url": "https://web.archive.org/web/20261111203232/https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_flow.tntp" + }, + "status": "live", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_flow.tntp" + } + ], + "subjects": [ + "operations-research" + ], + "tags": [ + "traffic-assignment", + "typed", + "demand", + "capacity", + "length", + "freeFlowTime", + "b", + "power", + "speed", + "toll", + "linkType" + ] +} diff --git a/scripts/archive-sources.report.json b/scripts/archive-sources.report.json index ce1c5aa5..e26f1fda 100644 --- a/scripts/archive-sources.report.json +++ b/scripts/archive-sources.report.json @@ -14750,6 +14750,14 @@ "timestamp": "20261013045429", "url": "https://brunel.figshare.com/articles/dataset/Supply_Chain_Logistics_Problem_Dataset/7558679" }, + { + "timestamp": "20261111203011", + "url": "https://codeload.github.com/bstabler/TransportationNetworks/zip/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d" + }, + { + "timestamp": "20261111210642", + "url": "https://codeload.github.com/power-grid-lib/pglib-opf/zip/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3" + }, { "timestamp": "20261011035221", "url": "https://commalab.di.unipi.it/datasets/mmcf/" @@ -14906,6 +14914,10 @@ "timestamp": "20260228133049", "url": "https://github.com/benmaier/BFMaierFBnetwork" }, + { + "timestamp": "20260530135004", + "url": "https://github.com/bstabler/TransportationNetworks" + }, { "timestamp": "20250425205803", "url": "https://github.com/ciol-researchlab/SupplyGraph" @@ -14914,6 +14926,10 @@ "timestamp": "20261031113231", "url": "https://github.com/karolpiczak/ESC-50" }, + { + "timestamp": "20260607092944", + "url": "https://github.com/power-grid-lib/pglib-opf" + }, { "timestamp": "20261022021217", "url": "https://github.com/rionbr/DDIBlumenau/tree/master/csv" @@ -16330,6 +16346,30 @@ "timestamp": "20261022040211", "url": "https://raw.githubusercontent.com/afkummer/hhcrsp-dataset-2021/262c7b89938ca817b927d4287e709252844b63fd/instances/instances-75/HHCRSP_75_15_98_1.6_R_C.txt" }, + { + "timestamp": "20261111203038", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Anaheim/Anaheim_flow.tntp" + }, + { + "timestamp": "20261111203053", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Barcelona/Barcelona_flow.tntp" + }, + { + "timestamp": "20261111203113", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Chicago-Sketch/ChicagoSketch_flow.tntp" + }, + { + "timestamp": "20261111203131", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/SiouxFalls/SiouxFalls_flow.tntp" + }, + { + "timestamp": "20261111203232", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/Winnipeg/Winnipeg_flow.tntp" + }, + { + "timestamp": "20261111203211", + "url": "https://raw.githubusercontent.com/bstabler/TransportationNetworks/d1639b4ef218c17928ba573e806ddf8ba5e7ae6d/chicago-regional/ChicagoRegional_flow.tntp" + }, { "timestamp": "20261110084325", "url": "https://raw.githubusercontent.com/ciol-researchlab/SupplyGraph/0e6d010e256ee08e34ab6bef5a111ef1c758b32d/Raw%20Dataset/Homogenoeus/Edges/Edges%20%28Plant%29.csv" @@ -16362,6 +16402,10 @@ "timestamp": "20261031113059", "url": "https://raw.githubusercontent.com/karolpiczak/ESC-50/33c8ce9eb2cf0b1c2f8bcf322eb349b6be34dbb6/meta/esc50.csv" }, + { + "timestamp": "20261111210813", + "url": "https://raw.githubusercontent.com/power-grid-lib/pglib-opf/dc6be4b2f85ca0e776952ec22cbd4c22396ea5a3/BASELINE.md" + }, { "timestamp": "20261022040339", "url": "https://raw.githubusercontent.com/psxjpc/MOVRPTW-Generator/master/dataset/movrptw-instances.zip" diff --git a/scripts/seed/convert.ts b/scripts/seed/convert.ts index 1e563029..4a724db9 100644 --- a/scripts/seed/convert.ts +++ b/scripts/seed/convert.ts @@ -312,7 +312,7 @@ function archiveKindOf(url: string): Exclude; +type GraphFormat = Exclude; /** The keepset parser each catalogue format reads with, by the pipeline's own dispatch (the format branches of experiments/pipeline/scripts/download_benchmark_graph.py in Mearman/Thesis): formats whose pipeline parser is the generic edge list, GML, Pajek, Matrix Market, UCINet DL or GD contest reader. */ const PARSERS: ReadonlyMap = new Map([ diff --git a/src/entry.ts b/src/entry.ts index 99d4f8c9..05607b93 100644 --- a/src/entry.ts +++ b/src/entry.ts @@ -177,7 +177,7 @@ export type SheetArchive = z.infer; export const InnerArchiveSchema = z.union([z.object({ kind: z.literal("gzip") }).strict(), z.object({ kind: FileArchiveKindSchema, member: ArchiveMemberSchema }).strict(), SheetArchiveSchema]); export type InnerArchive = z.infer; -/** The name of a part of a document read from several files, which the parser gives its meaning: the split a triples file holds (`train`, `valid`, `test`), or which of an HGB dataset's files it is (`nodes`, `links`). */ +/** The name of a part of a document read from several files, which the parser gives its meaning: the split a triples file holds (`train`, `valid`, `test`), which of an HGB dataset's files it is (`nodes`, `links`), or which of a TNTP dataset's files it is (`network`, `trips`). */ export const PartNameSchema = z.string().regex(/^[a-z][a-z0-9]*$/, { message: "must be a lowercase name of letters and digits, such as train" }); /** An archive one document is read from several files of: each part names the one member it is read from, and the parser reads the parts together in the order given. */ @@ -192,7 +192,7 @@ export const PartsArchiveSchema = z .strict(); export type PartsArchive = z.infer; -/** How the downloaded bytes unpack into the file the parser reads, the named files a parser of a parts format reads together into one document (`parts`, such as the split files of a triples download or an HGB dataset's nodes and links), the files a parser of a multi-file format reads together into one document (`files`, such as an OGB raw directory), or for a collection the files it reads one by one. A `members` selector gives every member it matches, at least one, in path order. `gzip`, `bzip2` and `xz` are one compressed file, `tar.gz`, `tar.bz2` and `tar.xz` a tar archive compressed with them, and `7z` a 7z archive, solid or not, which is decoded once from its start for every member selected; xz and 7z are read only where the platform gives a decoder or reader (Node does, the browser does not). A member that is an archive in its turn (a contest bundle that ships its problems as a zip inside the zip, a spreadsheet inside a zip) is unpacked by `inner`. A spreadsheet (`xlsx`, `xls` or `ods`) is read as the CSV of its named sheet, with dates as ISO 8601 text, for the `csv-table` or `edge-list` parser. A point shapefile (`shapefile`) is a zip whose `member` selects the `.shp`, read with the `.dbf`, `.prj` and `.cpg` of the same name beside it as the CSV of its attribute table with each point's stored coordinates and its WGS84 position, for the `csv-table` parser. */ +/** How the downloaded bytes unpack into the file the parser reads, the named files a parser of a parts format reads together into one document (`parts`, such as the split files of a triples download, an HGB dataset's nodes and links, or a TNTP dataset's network and trips), the files a parser of a multi-file format reads together into one document (`files`, such as an OGB raw directory), or for a collection the files it reads one by one. A `members` selector gives every member it matches, at least one, in path order. `gzip`, `bzip2` and `xz` are one compressed file, `tar.gz`, `tar.bz2` and `tar.xz` a tar archive compressed with them, and `7z` a 7z archive, solid or not, which is decoded once from its start for every member selected; xz and 7z are read only where the platform gives a decoder or reader (Node does, the browser does not). A member that is an archive in its turn (a contest bundle that ships its problems as a zip inside the zip, a spreadsheet inside a zip) is unpacked by `inner`. A spreadsheet (`xlsx`, `xls` or `ods`) is read as the CSV of its named sheet, with dates as ISO 8601 text, for the `csv-table` or `edge-list` parser. A point shapefile (`shapefile`) is a zip whose `member` selects the `.shp`, read with the `.dbf`, `.prj` and `.cpg` of the same name beside it as the CSV of its attribute table with each point's stored coordinates and its WGS84 position, for the `csv-table` parser. */ export const ArchiveSchema = z.union([ z.object({ kind: z.literal("none") }).strict(), z.object({ kind: z.literal("gzip") }).strict(), @@ -512,6 +512,14 @@ export const ParserSchema = z.discriminatedUnion("format", [ z.object({ format: z.literal("lrp"), encoding: EncodingSchema }).strict(), /** A DIMACS minimum-cost flow problem (`.min`): nodes with their supply, arcs with their lower bound, capacity and cost. */ z.object({ format: z.literal("dimacs-min"), encoding: EncodingSchema }).strict(), + /** A Transportation Networks for Research dataset in the TNTP format, read from the parts `network` (the `_net.tntp` file) and `trips` (the trip table): the road network's zones and nodes, its links with their columns as fields, and each origin and destination pair of the trip table as a commodity node with its demand, joined to its origin and destination zones. */ + z.object({ format: z.literal("tntp"), encoding: EncodingSchema }).strict(), + /** An undirected graph in the DIMACS challenge graph format (`p edge N M`, then `e U V` lines) or the PACE challenge format (`p PROBLEM N M`, then `U V` lines), as `layout` says: the nodes `1` to `N`, isolated ones included, and each edge once. */ + z.object({ format: z.literal("dimacs-graph"), encoding: EncodingSchema, layout: z.enum(["dimacs", "pace"]) }).strict(), + /** A Steiner tree problem in graphs in the SteinLib STP format: the nodes, terminals typed as such, the weighted undirected edges, and the Comment and Presolve values in `meta`. */ + z.object({ format: z.literal("steinlib"), encoding: EncodingSchema }).strict(), + /** A power system case in the MATPOWER case format, version 2: buses and generators as nodes with their tables' columns and each generator's quadratic cost, generators joined to their buses, branches as edges, and the case's name, version and base MVA in `meta`. */ + z.object({ format: z.literal("matpower"), encoding: EncodingSchema }).strict(), /** A resource page of OpenSLR (openslr.org): the resource's title, identifier, summary, category, licence, description and external address as the instance's meta, and its download links as nodes. */ z.object({ format: z.literal("openslr-resource"), encoding: z.literal("utf-8") }).strict(), /** A text read as the records a regular expression matches in it (a host's directory listing, a reference-value file): every match, in text order, is a row of the pattern's named captures, the `id` capture being the row's id, and each derived column is built from the captures its template names. */ @@ -656,6 +664,7 @@ export const STREAMED_FILE_FORMATS: ReadonlySet = new Set = new Map([ ["kg-triples", undefined], ["hgb", ["nodes", "links"]], + ["tntp", ["network", "trips"]], ]); /** diff --git a/src/entry.unit.test.ts b/src/entry.unit.test.ts index 2e040a62..f7de30ae 100644 --- a/src/entry.unit.test.ts +++ b/src/entry.unit.test.ts @@ -201,6 +201,20 @@ describe("an entry's sameDataAs", () => { }); }); +describe("a tntp entry's parts", () => { + const part = (name: string): unknown => ({ name, member: { path: `${name}.tntp` } }); + const tntpEntry = (parts: readonly unknown[]): unknown => ({ ...liveEntry, recipe: { status: "fetchable", source: 0, archive: { kind: "zip", parts }, parser: { format: "tntp", encoding: "utf-8" } } }); + + it("requires both the network and the trips part, and no other", () => { + expect(EntrySchema.is(tntpEntry([part("network"), part("trips")]))).toBe(true); + expect(EntrySchema.is(tntpEntry([part("network")]))).toBe(false); + expect(EntrySchema.is(tntpEntry([part("trips")]))).toBe(false); + expect(EntrySchema.is(tntpEntry([part("network"), part("nodes")]))).toBe(false); + expect(EntrySchema.is(tntpEntry([part("network"), part("trips"), part("extra")]))).toBe(false); + expect(EntrySchema.is({ ...liveEntry, recipe: { ...liveEntry.recipe, archive: { kind: "zip", member: { path: "net.tntp" } }, parser: { format: "tntp", encoding: "utf-8" } } })).toBe(false); + }); +}); + describe("an archive that selects one file or every file", () => { const withArchive = (base: typeof taskEntry | typeof taskCollectionEntry, archive: unknown): unknown => ({ ...base, recipe: { ...base.recipe, archive } }); const issues = (doc: unknown): string => { diff --git a/src/fetch.ts b/src/fetch.ts index 42132010..9c407c47 100644 --- a/src/fetch.ts +++ b/src/fetch.ts @@ -337,7 +337,7 @@ async function runToNative(dataset: Dataset, options: FetchOptions): Promise unpackParts(stored, archive, unpackOptionsOf(options))); const parsed = await step(entry.id, "parse", () => parseNativeParts(parts, recipe.parser)); - return { collection: false, documents: [{ id: undefined, native: { ...parsed, meta: metaOf(entry) } }], provenance: await provenanceOf({ members: parts.map((part) => part.path) }) }; + return { collection: false, documents: [{ id: undefined, native: { ...parsed, meta: { ...("meta" in parsed ? parsed.meta : {}), ...metaOf(entry) } } }], provenance: await provenanceOf({ members: parts.map((part) => part.path) }) }; } if ("files" in archive) { if (recipe.parser.format === "member-path" || recipe.parser.format === "grouped-member-path") { diff --git a/src/fetch.unit.test.ts b/src/fetch.unit.test.ts index 511b4e59..a3db0559 100644 --- a/src/fetch.unit.test.ts +++ b/src/fetch.unit.test.ts @@ -530,6 +530,33 @@ describe("a graph/v1 entry that is a collection", () => { }); }); +describe("a graph/v1 entry read from named parts", () => { + const network = [" 1", " 2", " 1", " 1", " Parser name", " parser-id", "", "~\tinit_node\tterm_node\tcapacity\t;", "\t1\t2\t10\t;"]; + const trips = [" 1", " 5", "", "Origin 1", " 1 : 5;"]; + const archive = zipSync({ "road/net.tntp": new TextEncoder().encode(`${network.join("\n")}\n`), "road/trips.tntp": new TextEncoder().encode(`${trips.join("\n")}\n`) }); + const entry = { + ...liveEntry, + id: "road-network", + name: "Road network", + sources: [{ url: "https://example.org/data/road.zip", retrieved: "2026-10-21", status: "live" }], + recipe: { + status: "fetchable", + source: 0, + archive: { kind: "zip", parts: [{ name: "network", member: { path: "road/net.tntp" } }, { name: "trips", member: { path: "road/trips.tntp" } }] }, + parser: { format: "tntp", encoding: "utf-8" }, + }, + crosswalk: { file: "crosswalks/graph-v1/topology.json" }, + expectedCounts: {}, + properties: {}, + }; + + it("keeps the meta the parser gives, and lets the entry's id and name take precedence over a parser key of the same name", async () => { + const { fetch } = servedBy({ "https://example.org/data/road.zip": archive }); + const fetched = await fetchEntry(datasetWith(entry), { ...storePort, fetch, now }); + expect(fetched.data.meta).toEqual({ id: "road-network", name: "Road network", zones: 1, firstThruNode: 1, totalOdFlow: 5 }); + }); +}); + describe("a graph/v1 entry of a multi-file format", () => { const archive = zipSync({ "dataset/raw/num-node-dict.csv.gz": gzip("paper,author\n2,1\n"), diff --git a/src/parsers/dimacs-graph.ts b/src/parsers/dimacs-graph.ts new file mode 100644 index 00000000..6f316d1e --- /dev/null +++ b/src/parsers/dimacs-graph.ts @@ -0,0 +1,79 @@ +import { NativeParseError } from "../errors"; +import type { NativeEdge, NativeGraph, NativeNode } from "./native"; +import { fieldLines, numberReader } from "./number-lines"; + +const FORMAT = "dimacs-graph"; + +const read = numberReader(FORMAT); + +/** How a file writes its edges: the DIMACS challenge graph format, `e U V` under a problem line `p edge N M` (the clique and colouring benchmarks), or the PACE challenge format, a bare `U V` under a problem line `p N M` naming the challenge's problem (`ds`, `td`, `vc`). */ +export type DimacsGraphLayout = "dimacs" | "pace"; + +/** + * Parses an undirected graph in the DIMACS challenge graph format or the PACE challenge format that derives from it: `c` comment lines anywhere, one problem line `p PROBLEM N M` before any edge (`p edge N M` in the DIMACS layout), then exactly M edge lines, `e U V` in the DIMACS layout or `U V` in the PACE layout. Nodes are `1` to `N`, so a vertex no edge touches is kept, which a maximum independent set or a minimum dominating set depends on; edges are emitted in file order, each once in the direction it is written. + * @param text - The file text. + * @param layout - Which of the two edge line forms the file uses. + * @returns The native graph. + * @throws NativeParseError naming the line for a missing, repeated or misplaced problem line, a DIMACS problem other than `edge`, a line of another form, a node outside 1 to N, a self-loop, an edge given twice (in either direction), or an edge count that differs from the problem line's. + */ +export function parseDimacsGraph(text: string, layout: DimacsGraphLayout): NativeGraph { + let problem: { readonly nodes: number; readonly edges: number } | undefined; + const edges: NativeEdge[] = []; + const seen = new Set(); + let last = 1; + for (const line of fieldLines(text)) { + last = line.line; + const [designator] = line.fields; + if (designator === "c") { + continue; + } + if (designator === "p") { + if (problem !== undefined) { + throw new NativeParseError(FORMAT, line.line, "a second problem line"); + } + const field = read.fields(line, ["designator", "problem", "nodes", "edges"], "the problem line p PROBLEM N M"); + if (layout === "dimacs" && field("problem") !== "edge") { + throw new NativeParseError(FORMAT, line.line, `expected the problem line p edge N M, found ${JSON.stringify(line.fields.join(" "))}`); + } + problem = { nodes: read.count(field("nodes"), line.line, "the node count", 0), edges: read.count(field("edges"), line.line, "the edge count", 0) }; + continue; + } + const ends = layout === "dimacs" ? (designator === "e" ? line.fields.slice(1) : undefined) : line.fields; + if (ends?.length !== 2) { + throw new NativeParseError(FORMAT, line.line, `expected ${layout === "dimacs" ? "an edge line e U V" : "an edge line U V"} or a line starting c or p, found ${JSON.stringify(line.fields.join(" "))}`); + } + if (problem === undefined) { + throw new NativeParseError(FORMAT, line.line, "an edge line before the problem line"); + } + const count = problem.nodes; + const node = (field: string): number => { + const value = read.integer(field, line.line, "a node number"); + if (value < 1 || value > count) { + throw new NativeParseError(FORMAT, line.line, `expected a node from 1 to ${String(count)}, found ${field}`); + } + + return value; + }; + const [first, second] = ends; + const u = node(first ?? ""); + const v = node(second ?? ""); + if (u === v) { + throw new NativeParseError(FORMAT, line.line, `an edge from node ${String(u)} to itself`); + } + const key = u < v ? `${String(u)} ${String(v)}` : `${String(v)} ${String(u)}`; + if (seen.has(key)) { + throw new NativeParseError(FORMAT, line.line, `the edge between nodes ${String(u)} and ${String(v)} is given twice`); + } + seen.add(key); + edges.push({ source: String(u), target: String(v) }); + } + if (problem === undefined) { + throw new NativeParseError(FORMAT, last, "the file has no problem line"); + } + if (edges.length !== problem.edges) { + throw new NativeParseError(FORMAT, last, `expected ${String(problem.edges)} edges, found ${String(edges.length)}`); + } + const nodes: NativeNode[] = Array.from({ length: problem.nodes }, (_, index) => ({ id: String(index + 1) })); + + return { nodes, edges }; +} diff --git a/src/parsers/dimacs-graph.unit.test.ts b/src/parsers/dimacs-graph.unit.test.ts new file mode 100644 index 00000000..669a10d8 --- /dev/null +++ b/src/parsers/dimacs-graph.unit.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, it } from "vitest"; +import { parseNative } from "./parse-native"; + +const dimacs = { format: "dimacs-graph", encoding: "utf-8", layout: "dimacs" } as const; +const pace = { format: "dimacs-graph", encoding: "utf-8", layout: "pace" } as const; + +const bytes = (...lines: readonly string[]): Uint8Array => new TextEncoder().encode(`${lines.join("\n")}\n`); + +describe("the DIMACS graph parser", () => { + it("reads the nodes 1 to N, an isolated one included, and each edge line once in file order", () => { + const expected = { + nodes: [{ id: "1" }, { id: "2" }, { id: "3" }, { id: "4" }], + edges: [ + { source: "1", target: "2" }, + { source: "3", target: "1" }, + ], + }; + expect(parseNative(bytes("c a comment", "p edge 4 2 ", "e 1 2", "c another", "e 3 1"), dimacs)).toEqual(expected); + expect(parseNative(bytes("p ds 4 2", "1 2", "3 1"), pace)).toEqual(expected); + expect(parseNative(new TextEncoder().encode("p ds 4 2\r\n1 2\r\n3 1\r\n\r\n"), pace)).toEqual(expected); + }); + + it("refuses a file whose problem line or edges disagree with the format", () => { + expect(() => parseNative(bytes("e 1 2"), dimacs)).toThrow(/line 1: an edge line before the problem line/); + expect(() => parseNative(bytes("p col 2 1", "e 1 2"), dimacs)).toThrow(/line 1: expected the problem line p edge N M/); + expect(() => parseNative(bytes("p edge 2 1", "p edge 2 1", "e 1 2"), dimacs)).toThrow(/line 2: a second problem line/); + expect(() => parseNative(bytes("p edge 2 1", "1 2"), dimacs)).toThrow(/line 2: expected an edge line e U V/); + expect(() => parseNative(bytes("p ds 2 1", "e 1 2"), pace)).toThrow(/line 2: expected an edge line U V/); + expect(() => parseNative(bytes("p edge 2 1", "e 1 3"), dimacs)).toThrow(/line 2: expected a node from 1 to 2, found 3/); + expect(() => parseNative(bytes("p edge 2 1", "e 2 2"), dimacs)).toThrow(/line 2: an edge from node 2 to itself/); + expect(() => parseNative(bytes("p edge 2 2", "e 1 2", "e 2 1"), dimacs)).toThrow(/line 3: the edge between nodes 2 and 1 is given twice/); + expect(() => parseNative(bytes("p edge 3 2", "e 1 2"), dimacs)).toThrow(/expected 2 edges, found 1/); + expect(() => parseNative(bytes("c nothing"), dimacs)).toThrow(/the file has no problem line/); + }); + + it("refuses an edge count that differs from the problem line's under the PACE layout", () => { + expect(() => parseNative(bytes("p ds 3 2", "1 2"), pace)).toThrow(/expected 2 edges, found 1/); + expect(() => parseNative(bytes("p ds 3 1", "1 2", "2 3"), pace)).toThrow(/expected 1 edges, found 2/); + }); + + it("reads a file with no edges as isolated nodes only, and refuses one that promises edges it lacks", () => { + expect(parseNative(bytes("p ds 3 0"), pace)).toEqual({ nodes: [{ id: "1" }, { id: "2" }, { id: "3" }], edges: [] }); + expect(parseNative(bytes("p edge 2 0"), dimacs)).toEqual({ nodes: [{ id: "1" }, { id: "2" }], edges: [] }); + expect(() => parseNative(bytes("p ds 3 1"), pace)).toThrow(/expected 1 edges, found 0/); + }); +}); diff --git a/src/parsers/matpower.ts b/src/parsers/matpower.ts new file mode 100644 index 00000000..ce66c794 --- /dev/null +++ b/src/parsers/matpower.ts @@ -0,0 +1,233 @@ +import { NativeParseError } from "../errors"; +import type { NativeAttributedEdge, NativeAttributedGraph, NativeRow } from "./native"; +import { lines } from "./native"; +import { numberReader } from "./number-lines"; + +const FORMAT = "matpower"; + +const read = numberReader(FORMAT); + +/** The bus table's columns in MATPOWER's order (the MATPOWER manual, appendix B, "caseformat"). `type` is read as `busType`, since a node's `type` says what the node is. */ +const BUS_COLUMNS = ["bus_i", "busType", "Pd", "Qd", "Gs", "Bs", "area", "Vm", "Va", "baseKV", "zone", "Vmax", "Vmin"] as const; +/** The generator table's columns: the ten every case has, then the eleven of the capability curve, ramp rates and participation factor that a case may add. `bus` is the edge to the generator's bus. */ +const GEN_COLUMNS = ["bus", "Pg", "Qg", "Qmax", "Qmin", "Vg", "mBase", "status", "Pmax", "Pmin", "Pc1", "Pc2", "Qc1min", "Qc1max", "Qc2min", "Qc2max", "ramp_agc", "ramp_10", "ramp_30", "ramp_q", "apf"] as const; +/** How many generator columns every case has; the rest are optional as a block. */ +const GEN_REQUIRED = 10; +/** The branch table's columns. `status` is read as `branchStatus`, as the generator's `status` is a node field and a crossmap path names one field. */ +const BRANCH_COLUMNS = ["fbus", "tbus", "r", "x", "b", "rateA", "rateB", "rateC", "ratio", "angle", "branchStatus", "angmin", "angmax"] as const; +/** The generator cost table's columns for the one form read: a polynomial (model 2) of three coefficients, highest order first. */ +const GENCOST_COLUMNS = ["model", "startup", "shutdown", "n", "c2", "c1", "c0"] as const; +/** MATPOWER's code for a polynomial cost. */ +const POLYNOMIAL = 2; +/** The number of polynomial coefficients the parser accepts, a quadratic cost: a cost row of any other model or coefficient count is refused. */ +const COEFFICIENTS = 3; + +/** A matrix assignment's opening line, `mpc.NAME = [`. */ +const MATRIX_START = /^mpc\.(\w+)\s*=\s*\[\s*$/; +/** A scalar assignment, `mpc.NAME = VALUE;`, the value a number or a quoted string. */ +const SCALAR = /^mpc\.(\w+)\s*=\s*(?:'([^']*)'|([^;\s]+))\s*;$/; +/** The function line that names the case. */ +const FUNCTION = /^function\s+mpc\s*=\s*(\w+)$/; +/** The tables the parser reads. */ +const TABLES: ReadonlySet = new Set(["bus", "gen", "gencost", "branch", "areas"]); + +/** A table's rows, each with its 1-based line. */ +interface Table { + readonly rows: readonly { readonly line: number; readonly fields: readonly string[] }[]; + readonly line: number; +} + +/** + * Splits a case file into its scalar assignments, the case's function name and its matrix tables, dropping `%` comments. + * @param text - The file text. + * @returns The scalars by name, the function name and the tables by name. + * @throws NativeParseError for a line outside a table that is not a comment, the function line or a scalar assignment, a table the parser does not read, a value given twice, or a table that is not closed. + */ +function split(text: string): { scalars: Map; name: string | undefined; tables: Map } { + const scalars = new Map(); + const tables = new Map(); + let name: string | undefined; + let open: { readonly name: string; readonly line: number; readonly rows: { readonly line: number; readonly fields: readonly string[] }[] } | undefined; + for (const [index, raw] of lines(text).entries()) { + const line = index + 1; + const content = raw.replace(/%.*$/, "").trim(); + if (content === "") { + continue; + } + if (open !== undefined) { + if (content === "];") { + tables.set(open.name, { rows: open.rows, line: open.line }); + open = undefined; + continue; + } + if (!content.endsWith(";")) { + throw new NativeParseError(FORMAT, line, `expected a row of mpc.${open.name} ending with ;, found ${JSON.stringify(content)}`); + } + open.rows.push({ line, fields: content.slice(0, -1).trim().split(/\s+/) }); + continue; + } + const start = MATRIX_START.exec(content); + if (start !== null) { + const table = start[1] ?? ""; + if (!TABLES.has(table)) { + throw new NativeParseError(FORMAT, line, `the table mpc.${table} is not one the parser reads (bus, gen, gencost, branch, areas)`); + } + if (tables.has(table)) { + throw new NativeParseError(FORMAT, line, `the table mpc.${table} is given twice`); + } + open = { name: table, line, rows: [] }; + continue; + } + const scalar = SCALAR.exec(content); + if (scalar !== null) { + const key = scalar[1] ?? ""; + if (scalars.has(key)) { + throw new NativeParseError(FORMAT, line, `mpc.${key} is given twice`); + } + scalars.set(key, { value: scalar[2] ?? scalar[3] ?? "", line }); + continue; + } + const fn = FUNCTION.exec(content); + if (fn !== null && name === undefined) { + name = fn[1]; + continue; + } + throw new NativeParseError(FORMAT, line, `expected a comment, the function line, mpc.NAME = VALUE; or mpc.NAME = [, found ${JSON.stringify(content)}`); + } + if (open !== undefined) { + throw new NativeParseError(FORMAT, open.line, `the table mpc.${open.name} is not closed with ];`); + } + + return { scalars, name, tables }; +} + +/** + * Reads a table's rows as numbers, each row of one of the widths allowed. + * @param tables - The case's tables. + * @param table - The table's name. + * @param widths - The numbers of columns a row may have. + * @returns The rows, each its numbers with its line. + * @throws NativeParseError when the table is missing, a row has another width or the rows differ in width, or a field is not a number. + */ +function numbers(tables: ReadonlyMap, table: string, widths: readonly number[]): { readonly line: number; readonly values: readonly number[] }[] { + return numbersOf(table, requireTable(tables, table), widths); +} + +/** + * Finds a table the case must have. + * @param tables - The case's tables. + * @param table - The table's name. + * @returns The table. + * @throws NativeParseError when the table is missing. + */ +function requireTable(tables: ReadonlyMap, table: string): Table { + const found = tables.get(table); + if (found === undefined) { + throw new NativeParseError(FORMAT, 1, `the case has no mpc.${table} table`); + } + + return found; +} + +/** + * Reads a found table's rows as numbers, each row of one of the widths allowed. + * @param table - The table's name, for errors. + * @param found - The table. + * @param widths - The numbers of columns a row may have. + * @returns The rows, each its numbers with its line. + * @throws NativeParseError when a row has another width or the rows differ in width, or a field is not a number. + */ +function numbersOf(table: string, found: Table, widths: readonly number[]): { readonly line: number; readonly values: readonly number[] }[] { + const width = found.rows[0]?.fields.length; + + return found.rows.map(({ line, fields }) => { + if (!widths.includes(fields.length) || fields.length !== width) { + throw new NativeParseError(FORMAT, line, `expected a row of mpc.${table} of ${widths.join(" or ")} columns, the same in every row, found ${String(fields.length)}`); + } + + return { line, values: fields.map((field) => read.scientific(field, line, `a value of mpc.${table}`)) }; + }); +} + +/** + * Parses a power system case in the MATPOWER case format, version 2 (the MATPOWER manual, appendix B), as the PGLib optimal power flow library distributes it: `%` comments, the `function mpc = NAME` line, `mpc.version = '2';`, `mpc.baseMVA = N;`, and the matrices `mpc.bus`, `mpc.gen`, `mpc.gencost` and `mpc.branch` (and the legacy `mpc.areas`), one row per line ended by `;`. + * + * Each bus is a node of type `bus`, its id the bus number, with the bus table's columns as fields (`type` as `busType`). Each generator is a node `generator-` (its 1-based row) of type `generator` with the generator table's columns (the eleven optional ones null when the case gives ten) and its cost row's `startup`, `shutdown`, `c2`, `c1` and `c0`, joined to its bus by an edge of type `generator`. Each branch is an undirected edge of type `branch` from its `fbus` to its `tbus` with the branch table's columns (`status` as `branchStatus`), parallel branches kept. Every node carries every node field and every edge every edge field, null where it has none. `meta` holds the case's `name`, `version` and `baseMVA`, and the area table, where there is one, as `areas`: each row's area and reference bus, joined by `;`. + * @param text - The file text. + * @returns The native graph. + * @throws NativeParseError naming the line for a line of another form, a table or value given twice or missing, a version other than 2, a row of another width, a value that is not a number, a bus number given twice or not a whole number, a generator or branch at a bus the case does not list, a cost row count other than the generator count, or a cost other than a quadratic polynomial. + */ +export function parseMatpower(text: string): NativeAttributedGraph { + const { scalars, name, tables } = split(text); + const version = scalars.get("version"); + if (version?.value !== "2") { + throw new NativeParseError(FORMAT, version?.line ?? 1, `expected mpc.version = '2', found ${JSON.stringify(version?.value ?? "none")}`); + } + const base = scalars.get("baseMVA"); + if (base === undefined) { + throw new NativeParseError(FORMAT, 1, "the case has no mpc.baseMVA"); + } + for (const [key, { line }] of scalars) { + if (key !== "version" && key !== "baseMVA") { + throw new NativeParseError(FORMAT, line, `mpc.${key} is not a value the parser reads (version, baseMVA)`); + } + } + const areas = tables.get("areas"); + const meta: Record = { + ...(name === undefined ? {} : { name }), + version: version.value, + baseMVA: read.scientific(base.value, base.line, "mpc.baseMVA"), + ...(areas === undefined ? {} : { areas: numbers(tables, "areas", [2]).map(({ values }) => values.join(" ")).join("; ") }), + }; + + const busNulls = Object.fromEntries(BUS_COLUMNS.slice(1).map((column) => [column, null])); + const genNulls = Object.fromEntries([...GEN_COLUMNS.slice(1), ...GENCOST_COLUMNS.filter((column) => column !== "model" && column !== "n")].map((column) => [column, null])); + const edgeNulls = Object.fromEntries(BRANCH_COLUMNS.slice(2).map((column) => [column, null])); + + const nodes: NativeRow[] = []; + const buses = new Set(); + for (const { line, values } of numbers(tables, "bus", [BUS_COLUMNS.length])) { + const [id, ...rest] = values; + if (id === undefined || !Number.isSafeInteger(id) || buses.has(id)) { + throw new NativeParseError(FORMAT, line, `the bus number ${String(id)} is not a whole number or is given twice`); + } + buses.add(id); + nodes.push({ id: String(id), type: "bus", ...Object.fromEntries(BUS_COLUMNS.slice(1).map((column, position) => [column, rest[position] ?? null])), ...genNulls }); + } + const atBus = (bus: number | undefined, line: number, what: string): string => { + if (bus === undefined || !buses.has(bus)) { + throw new NativeParseError(FORMAT, line, `${what} at bus ${String(bus)}, which the case does not list`); + } + + return String(bus); + }; + + const gens = numbers(tables, "gen", [GEN_REQUIRED, GEN_COLUMNS.length]); + const costTable = requireTable(tables, "gencost"); + const costs = numbersOf("gencost", costTable, [GENCOST_COLUMNS.length]); + if (costs.length !== gens.length) { + throw new NativeParseError(FORMAT, costTable.line, `expected one cost row per generator (${String(gens.length)}), found ${String(costs.length)}`); + } + const edges: NativeAttributedEdge[] = []; + for (const [index, { line, values }] of gens.entries()) { + const cost = costs[index]; + if (cost === undefined) { + throw new NativeParseError(FORMAT, line, "the generator has no cost row"); + } + const [model, startup, shutdown, n, ...coefficients] = cost.values; + if (model !== POLYNOMIAL || n !== COEFFICIENTS) { + throw new NativeParseError(FORMAT, cost.line, `expected a quadratic polynomial cost (model ${String(POLYNOMIAL)}, n ${String(COEFFICIENTS)}), found model ${String(model)} with n ${String(n)}`); + } + const [bus, ...rest] = values; + const id = `generator-${String(index + 1)}`; + const [c2 = null, c1 = null, c0 = null] = coefficients; + nodes.push({ id, type: "generator", ...busNulls, ...Object.fromEntries(GEN_COLUMNS.slice(1).map((column, position) => [column, rest[position] ?? null])), startup: startup ?? null, shutdown: shutdown ?? null, c2, c1, c0 }); + edges.push({ source: id, target: atBus(bus, line, "a generator"), type: "generator", ...edgeNulls }); + } + for (const { line, values } of numbers(tables, "branch", [BRANCH_COLUMNS.length])) { + const [fbus, tbus, ...rest] = values; + edges.push({ source: atBus(fbus, line, "a branch"), target: atBus(tbus, line, "a branch"), type: "branch", ...Object.fromEntries(BRANCH_COLUMNS.slice(2).map((column, position) => [column, rest[position] ?? null])) }); + } + + return { meta, nodes, edges }; +} diff --git a/src/parsers/matpower.unit.test.ts b/src/parsers/matpower.unit.test.ts new file mode 100644 index 00000000..521e2f47 --- /dev/null +++ b/src/parsers/matpower.unit.test.ts @@ -0,0 +1,92 @@ +import { describe, expect, it } from "vitest"; +import { parseNative } from "./parse-native"; + +const matpower = { format: "matpower", encoding: "utf-8" } as const; + +const bytes = (...lines: readonly string[]): Uint8Array => new TextEncoder().encode(`${lines.join("\n")}\n`); + +/** A two-bus case laid out as the PGLib files are: a comment header, the function line, the scalars, and each table under its column comment, with a trailing comment on a cost row. */ +const SMALL = [ + "%% a two-bus case", + "function mpc = small_case", + "mpc.version = '2';", + "mpc.baseMVA = 100.0;", + "", + "%% bus data", + "%\tbus_i\ttype\tPd\tQd\tGs\tBs\tarea\tVm\tVa\tbaseKV\tzone\tVmax\tVmin", + "mpc.bus = [", + "\t1\t 3\t 0.0\t 0.0\t 0.0\t 0.0\t 1\t 1.06\t 0.0\t 0.0\t 1\t 1.06\t 0.94;", + "\t4\t 1\t 21.7\t 12.7\t 0.0\t 5.02e-05\t 1\t 1.0\t 0.0\t 0.0\t 1\t 1.06\t 0.94;", + "];", + "", + "%% generator data", + "mpc.gen = [", + "\t1\t 232.4\t -16.9\t 10.0\t 0.0\t 1.06\t 100.0\t 1\t 340\t 0.0;", + "];", + "", + "%% generator cost data", + "mpc.gencost = [", + "\t2\t 0.0\t 0.0\t 3\t 0.000000\t 7.920951\t 0.000000; % NG", + "];", + "", + "%% branch data", + "mpc.branch = [", + "\t1\t 4\t 0.01938\t 0.05917\t 0.0528\t 472\t 472\t 472\t 0.0\t 0.0\t 1\t -30.0\t 30.0;", + "];", +]; + +const genNulls = { Pc1: null, Pc2: null, Qc1min: null, Qc1max: null, Qc2min: null, Qc2max: null, ramp_agc: null, ramp_10: null, ramp_30: null, ramp_q: null, apf: null }; +const busNulls = { busType: null, Pd: null, Qd: null, Gs: null, Bs: null, area: null, Vm: null, Va: null, baseKV: null, zone: null, Vmax: null, Vmin: null }; +const noGen = { Pg: null, Qg: null, Qmax: null, Qmin: null, Vg: null, mBase: null, status: null, Pmax: null, Pmin: null, ...genNulls, startup: null, shutdown: null, c2: null, c1: null, c0: null }; +const noBranch = { r: null, x: null, b: null, rateA: null, rateB: null, rateC: null, ratio: null, angle: null, branchStatus: null, angmin: null, angmax: null }; + +describe("the MATPOWER parser", () => { + it("reads buses and generators as nodes, each generator joined to its bus, and branches as edges", () => { + expect(parseNative(bytes(...SMALL), matpower)).toEqual({ + meta: { name: "small_case", version: "2", baseMVA: 100 }, + nodes: [ + { id: "1", type: "bus", busType: 3, Pd: 0, Qd: 0, Gs: 0, Bs: 0, area: 1, Vm: 1.06, Va: 0, baseKV: 0, zone: 1, Vmax: 1.06, Vmin: 0.94, ...noGen }, + { id: "4", type: "bus", busType: 1, Pd: 21.7, Qd: 12.7, Gs: 0, Bs: 5.02e-5, area: 1, Vm: 1, Va: 0, baseKV: 0, zone: 1, Vmax: 1.06, Vmin: 0.94, ...noGen }, + { id: "generator-1", type: "generator", ...busNulls, Pg: 232.4, Qg: -16.9, Qmax: 10, Qmin: 0, Vg: 1.06, mBase: 100, status: 1, Pmax: 340, Pmin: 0, ...genNulls, startup: 0, shutdown: 0, c2: 0, c1: 7.920951, c0: 0 }, + ], + edges: [ + { source: "generator-1", target: "1", type: "generator", ...noBranch }, + { source: "1", target: "4", type: "branch", r: 0.01938, x: 0.05917, b: 0.0528, rateA: 472, rateB: 472, rateC: 472, ratio: 0, angle: 0, branchStatus: 1, angmin: -30, angmax: 30 }, + ], + }); + }); + + it("carries the legacy area table in the meta", () => { + expect(parseNative(bytes(...SMALL, "mpc.areas = [", "\t1\t 1;", "];"), matpower).meta).toEqual({ name: "small_case", version: "2", baseMVA: 100, areas: "1 1" }); + }); + + it("refuses a case that disagrees with the format", () => { + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("'2'", "'1'"))), matpower)).toThrow(/expected mpc.version = '2'/); + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("\t4\t 1\t", "\t1\t 1\t"))), matpower)).toThrow(/line 10: the bus number 1 is not a whole number or is given twice/); + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("\t1\t 4\t 0.01938", "\t1\t 5\t 0.01938"))), matpower)).toThrow(/line 25: a branch at bus 5, which the case does not list/); + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("\t2\t 0.0\t 0.0\t 3\t 0.000000\t", "\t2\t 0.0\t 0.0\t 2\t"))), matpower)).toThrow(/line 20: expected a row of mpc.gencost of 7 columns/); + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("\t2\t 0.0\t 0.0\t 3\t", "\t1\t 0.0\t 0.0\t 3\t"))), matpower)).toThrow(/line 20: expected a quadratic polynomial cost/); + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace(" 0.94;", " 0.94"))), matpower)).toThrow(/line 9: expected a row of mpc.bus ending with ;/); + expect(() => parseNative(bytes(...SMALL, "mpc.dcline = ["), matpower)).toThrow(/the table mpc.dcline is not one the parser reads/); + expect(() => parseNative(bytes(...SMALL.slice(0, -1)), matpower)).toThrow(/line 24: the table mpc.branch is not closed/); + }); + + it("reads a value written with an exponent", () => { + expect(parseNative(bytes(...SMALL.map((line) => line.replace("baseMVA = 100.0", "baseMVA = 1.5e+2"))), matpower).meta).toMatchObject({ baseMVA: 150 }); + }); + + it("refuses a case with no gencost table", () => { + const start = SMALL.indexOf("mpc.gencost = ["); + const without = [...SMALL.slice(0, start), ...SMALL.slice(SMALL.indexOf("];", start) + 1)]; + expect(() => parseNative(bytes(...without), matpower)).toThrow(/matpower line 1: the case has no mpc.gencost table/); + }); + + it("refuses Inf as a value that is not a number", () => { + expect(() => parseNative(bytes(...SMALL.map((line) => line.replace("\t 472\t 472\t 472\t", "\t Inf\t 472\t 472\t"))), matpower)).toThrow(/matpower line 25: expected a value of mpc.branch as a number, found "Inf"/); + }); + + it("refuses a gencost table of twice as many rows as generators", () => { + const cost = SMALL.find((line) => line.includes("% NG")) ?? ""; + expect(() => parseNative(bytes(...SMALL.flatMap((line) => (line === cost ? [line, line] : [line]))), matpower)).toThrow(/matpower line 19: expected one cost row per generator \(1\), found 2/); + }); +}); diff --git a/src/parsers/number-lines.ts b/src/parsers/number-lines.ts index 0e91f224..bd7efbdc 100644 --- a/src/parsers/number-lines.ts +++ b/src/parsers/number-lines.ts @@ -10,6 +10,9 @@ export interface FieldLine { /** A decimal as the numeric instance formats write one: optionally signed, with digits on either side of an optional point (`5`, `6.000000`, `.0`). */ const DECIMAL = /^[+-]?(?:\d+\.?\d*|\.\d+)$/; +/** A decimal that may carry an exponent, as the formats written by numeric software do (`25900.20064`, `7.12506e+007`, `5.02e-05`). */ +const SCIENTIFIC = /^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$/; + /** A whole number as the numeric instance formats write one, optionally signed. */ const INTEGER = /^[+-]?\d+$/; @@ -41,6 +44,15 @@ export interface NumberReader { * @throws NativeParseError when the field is not a decimal number. */ readonly decimal: (field: string, line: number, what: string) => number; + /** + * Reads a field as a finite number that may carry an exponent (`1.5e+3`, `5.02e-05`). + * @param field - The field as written. + * @param line - Its 1-based line. + * @param what - What the field should be. + * @returns The number. + * @throws NativeParseError when the field is not a decimal number, with or without an exponent. + */ + readonly scientific: (field: string, line: number, what: string) => number; /** * Reads a field as a whole number that a double holds exactly. * @param field - The field as written. @@ -94,6 +106,13 @@ export function numberReader(format: string): NumberReader { return Number(field); }, + scientific(field, line, what) { + if (!SCIENTIFIC.test(field)) { + throw new NativeParseError(format, line, `expected ${what} as a number, found ${JSON.stringify(field)}`); + } + + return Number(field); + }, integer, count(field, line, what, least) { const value = integer(field, line, what); diff --git a/src/parsers/number-lines.unit.test.ts b/src/parsers/number-lines.unit.test.ts new file mode 100644 index 00000000..c73a0f9b --- /dev/null +++ b/src/parsers/number-lines.unit.test.ts @@ -0,0 +1,25 @@ +import { describe, expect, it } from "vitest"; +import { NativeParseError } from "../errors"; +import { numberReader } from "./number-lines"; + +const read = numberReader("example"); +const LINE = 4; + +describe("the shared number reader", () => { + it("reads a decimal with or without an exponent as a number", () => { + const expected: Record = { "1.5e+3": 1500, "7.12506E+007": 71250600, "5.02e-05": 0.0000502, "-.5": -0.5, "12": 12 }; + for (const [field, value] of Object.entries(expected)) { + expect(read.scientific(field, LINE, "a weight")).toBe(value); + } + }); + + it("refuses what is not a number, naming the format, line and field", () => { + for (const field of ["", "e5", "1e", "1e+", "1.5x", "NaN", "Infinity", "0x10"]) { + expect(() => read.scientific(field, LINE, "a weight")).toThrow(new NativeParseError("example", LINE, `expected a weight as a number, found ${JSON.stringify(field)}`)); + } + }); + + it("leaves decimal refusing an exponent", () => { + expect(() => read.decimal("1.5e+3", LINE, "a weight")).toThrow(/example line 4: expected a weight as a number/); + }); +}); diff --git a/src/parsers/parse-native.ts b/src/parsers/parse-native.ts index c01bf9ad..8fd307c4 100644 --- a/src/parsers/parse-native.ts +++ b/src/parsers/parse-native.ts @@ -45,6 +45,10 @@ import { parseOrlibPmed } from "./orlib-pmed"; import { parseOrlibPmedcap } from "./orlib-pmedcap"; import { parseLrp } from "./lrp"; import { parseDimacsMin } from "./dimacs-min"; +import { parseTntp } from "./tntp"; +import { parseDimacsGraph } from "./dimacs-graph"; +import { parseSteinlib } from "./steinlib"; +import { parseMatpower } from "./matpower"; import type { NativeAttributedGraph, NativeDocument, NativeGraph, NativeInstance, NativeTable } from "./native"; import { parsePajek } from "./pajek"; import { parseCustomerPoints } from "./customer-points"; @@ -163,7 +167,7 @@ export function* decodePieces(bytes: Uint8Array, parser: Parser, pieceBytes: num type TableParser = Extract; /** The parsers of network instance formats, whose nodes and edges carry the fields the file gives them. */ -type AttributedGraphParser = Extract; +type AttributedGraphParser = Extract; /** * Reads a file into the native shape of the recipe's parser: a graph for the graph formats, a table of rows for the table formats. @@ -296,6 +300,15 @@ export function parseNative(bytes: Uint8Array, parser: Parser): NativeDocument { if (parser.format === "dimacs-min") { return parseDimacsMin(text); } + if (parser.format === "dimacs-graph") { + return parseDimacsGraph(text, parser.layout); + } + if (parser.format === "steinlib") { + return parseSteinlib(text); + } + if (parser.format === "matpower") { + return parseMatpower(text); + } return parsePajek(text); } @@ -323,13 +336,13 @@ export interface NamedBytes { } /** - * Reads the named parts of a parts archive into the one native graph of the recipe's parser: the split files of a triples download, each edge carrying its part's name as its split, or an HGB dataset's `nodes` and `links` files. + * Reads the named parts of a parts archive into the one native graph of the recipe's parser: the split files of a triples download, each edge carrying its part's name as its split, an HGB dataset's `nodes` and `links` files, or a TNTP dataset's `network` file and `trips` table, whose graph carries a `meta` of its own. * @param parts - The parts, in the order the recipe names them. * @param parser - The recipe's parser, one of the formats that read parts. * @returns The native graph. - * @throws NativeParseError when a part is not valid in the stated encoding, its text is not in the stated format, the parser reads one file, or an HGB recipe lacks its `nodes` or `links` part. + * @throws NativeParseError when a part is not valid in the stated encoding, its text is not in the stated format, the parser reads one file, or an HGB or TNTP recipe lacks one of its parts. */ -export function parseNativeParts(parts: readonly NamedBytes[], parser: Parser): NativeGraph { +export function parseNativeParts(parts: readonly NamedBytes[], parser: Parser): NativeGraph | NativeAttributedGraph { if (parser.format === "kg-triples") { return parseKgTriples( parts.map((part) => ({ split: part.name, pieces: decodePieces(part.bytes, parser) })), @@ -345,6 +358,15 @@ export function parseNativeParts(parts: readonly NamedBytes[], parser: Parser): return parseHgb({ nodes: decodePieces(nodes.bytes, parser), links: decodePieces(links.bytes, parser) }, parser); } + if (parser.format === "tntp") { + const network = parts.find((part) => part.name === "network"); + const trips = parts.find((part) => part.name === "trips"); + if (network === undefined || trips === undefined) { + throw new NativeParseError(parser.format, 1, "a TNTP recipe reads the parts network and trips"); + } + + return parseTntp({ network: decode(network.bytes, parser), trips: decode(trips.bytes, parser) }); + } throw new NativeParseError(parser.format, 1, `the ${parser.format} parser reads one file, not named parts`); } diff --git a/src/parsers/steinlib.ts b/src/parsers/steinlib.ts new file mode 100644 index 00000000..013e7d31 --- /dev/null +++ b/src/parsers/steinlib.ts @@ -0,0 +1,232 @@ +import { NativeParseError } from "../errors"; +import type { NativeAttributedEdge, NativeAttributedGraph, NativeRow } from "./native"; +import { lines } from "./native"; +import { numberReader } from "./number-lines"; + +const FORMAT = "steinlib"; + +const read = numberReader(FORMAT); + +/** The magic number that opens every SteinLib file, which the files write in either case. */ +const MAGIC = "33d32945"; +/** A section header, `SECTION `, which the files write in upper or title case. */ +const SECTION = /^section\s+(\S+)$/i; +/** The keys of a Comment section, each one string; the files quote it or not. */ +const COMMENT_KEYS: ReadonlyMap = new Map([ + ["name", "name"], + ["date", "date"], + ["creator", "creator"], + ["remark", "remark"], + ["problem", "problem"], +]); +/** The scalar keys of a Presolve section and the meta field and kind each is read as: the value to add to the optimum of the reduced instance to get the original's, the bounds, the processing time in seconds and the original instance's counts. */ +const PRESOLVE_KEYS: ReadonlyMap = new Map([ + ["fixed", { field: "fixed", integer: false }], + ["lower", { field: "lower", integer: false }], + ["upper", { field: "upper", integer: false }], + ["time", { field: "presolveTime", integer: false }], + ["orgnodes", { field: "orgNodes", integer: true }], + ["orgedges", { field: "orgEdges", integer: true }], + ["orgterminals", { field: "orgTerminals", integer: true }], +]); +/** The Presolve section's lines that describe how the original instance maps onto the reduced one (aggregated, chosen, deleted and upper-bound edges), which are not read. */ +const PRESOLVE_EDGE_LINES: ReadonlySet = new Set(["ea", "ec", "ed", "es"]); + +/** The fields of an edge line, `E U V WEIGHT`. */ +const EDGE_FIELDS = ["E", "U", "V", "WEIGHT"] as const; + +/** Which section a line is in, and for a Terminals section whether it is the instance's own or the original instance's after a Presolve section. */ +type Section = "comment" | "graph" | "terminals" | "original-terminals" | "presolve" | "coordinates"; + +/** + * Parses a Steiner tree problem in graphs in the SteinLib STP format (https://steinlib.zib.de/format.php): the magic number line `33D32945 STP File, STP Format Version 1.0`, then sections from `SECTION ` to `END` (either in upper or title case), then `EOF`; `#` starts a comment on a line holding no quoted string. The Comment sections give `meta` their `Name`, `Date`, `Creator`, `Remark` and `Problem` as text, unquoted. The Graph section's `Nodes` and `Edges` counts and `E U V WEIGHT` lines (the weight with or without an exponent) give the nodes `1` to `n` and one undirected edge per line with its `weight`, in file order. The first Terminals section's `Terminals` count and `T V` lines make those nodes of type `terminal` and the rest of type `node`. A Presolve section, which says the instance is the reduced form of an original one, gives `meta` its `fixed` value (added to the reduced optimum to give the original's), `lower` and `upper` bounds, `presolveTime` and the original's `orgNodes`, `orgEdges` and `orgTerminals`; its EA, EC, ED and ES lines, which map the reduced instance back onto the original, are not read, nor is the Terminals section after it, which lists the original's terminals. Coordinates sections are not read, as graph/v1 carries no positions. + * @param text - The file text. + * @returns The native graph. + * @throws NativeParseError naming the line for a file without the magic number, a line outside a section, an unknown section or key, a directed (`Arcs`), obstacle, root, prize or degree-constrained instance, a line of the wrong shape, a key given twice, a node outside 1 to n, a terminal given twice, a count that disagrees with its lines, a missing Graph or Terminals section, a Presolve section before the instance's own Terminals section (the format lists the instance's terminals first and the original's after Presolve) or text after `EOF`. + */ +export function parseSteinlib(text: string): NativeAttributedGraph { + const all = lines(text); + const meta: Record = {}; + const edges: NativeAttributedEdge[] = []; + const terminals = new Set(); + const counts: { nodes?: number; edges?: number; terminals?: number } = {}; + let section: Section | undefined; + let started = false; + let ended = false; + let presolved = false; + let graphRead = false; + let terminalsRead = false; + let last = 1; + const node = (field: string, line: number): number => { + if (counts.nodes === undefined) { + throw new NativeParseError(FORMAT, line, "a node before the Nodes count"); + } + const value = read.integer(field, line, "a node number"); + if (value < 1 || value > counts.nodes) { + throw new NativeParseError(FORMAT, line, `expected a node from 1 to ${String(counts.nodes)}, found ${field}`); + } + + return value; + }; + for (const [index, raw] of all.entries()) { + const line = index + 1; + const stripped = raw.includes('"') ? raw : raw.replace(/#.*$/, ""); + const content = stripped.trim(); + if (content === "") { + continue; + } + last = line; + if (!started) { + if (!content.toLowerCase().startsWith(MAGIC)) { + throw new NativeParseError(FORMAT, line, `expected the magic number line 33D32945 STP File, found ${JSON.stringify(content)}`); + } + started = true; + continue; + } + if (ended) { + throw new NativeParseError(FORMAT, line, `a line after EOF: ${JSON.stringify(content)}`); + } + const fields = content.split(/\s+/); + const keyword = (fields[0] ?? "").toLowerCase(); + if (section === undefined) { + if (keyword === "eof") { + ended = true; + continue; + } + const header = SECTION.exec(content); + const name = header?.[1]?.toLowerCase(); + if (name === "comment" || name === "coordinates") { + section = name; + } else if (name === "graph") { + if (graphRead) { + throw new NativeParseError(FORMAT, line, "a second Graph section"); + } + section = "graph"; + } else if (name === "terminals") { + if (presolved) { + section = "original-terminals"; + } else if (terminalsRead) { + throw new NativeParseError(FORMAT, line, "a second Terminals section before any Presolve section"); + } else { + section = "terminals"; + } + } else if (name === "presolve") { + if (presolved) { + throw new NativeParseError(FORMAT, line, "a second Presolve section"); + } + if (!terminalsRead) { + throw new NativeParseError(FORMAT, line, "a Presolve section before the instance's Terminals section (the format gives the instance's own Terminals section first and the original's after Presolve)"); + } + section = "presolve"; + } else { + throw new NativeParseError(FORMAT, line, header === null ? `expected SECTION or EOF, found ${JSON.stringify(content)}` : `the section ${JSON.stringify(header[1] ?? "")} is not one the parser reads (Comment, Graph, Terminals, Presolve or Coordinates)`); + } + continue; + } + if (keyword === "end" && fields.length === 1) { + if (section === "graph") { + if (counts.nodes === undefined || counts.edges === undefined) { + throw new NativeParseError(FORMAT, line, "the Graph section has no Nodes or Edges count"); + } + if (edges.length !== counts.edges) { + throw new NativeParseError(FORMAT, line, `expected ${String(counts.edges)} edges, found ${String(edges.length)}`); + } + graphRead = true; + } + if (section === "terminals") { + if (counts.terminals === undefined) { + throw new NativeParseError(FORMAT, line, "the Terminals section has no Terminals count"); + } + if (terminals.size !== counts.terminals) { + throw new NativeParseError(FORMAT, line, `expected ${String(counts.terminals)} terminals, found ${String(terminals.size)}`); + } + terminalsRead = true; + } + if (section === "presolve") { + presolved = true; + } + section = undefined; + continue; + } + if (section === "coordinates" || section === "original-terminals") { + continue; + } + if (section === "comment") { + const field = COMMENT_KEYS.get(keyword); + if (field === undefined) { + throw new NativeParseError(FORMAT, line, `expected a Comment key (Name, Date, Creator, Remark or Problem), found ${JSON.stringify(fields[0] ?? "")}`); + } + if (field in meta) { + throw new NativeParseError(FORMAT, line, `the Comment key ${fields[0] ?? ""} is given twice`); + } + const value = content.slice((fields[0] ?? "").length).trim(); + meta[field] = value.length >= 2 && value.startsWith('"') && value.endsWith('"') ? value.slice(1, -1) : value; + continue; + } + if (section === "presolve") { + if (PRESOLVE_EDGE_LINES.has(keyword)) { + continue; + } + const key = PRESOLVE_KEYS.get(keyword); + if (key === undefined || fields.length !== 2) { + throw new NativeParseError(FORMAT, line, `expected a Presolve line (Fixed, Lower, Upper, Time, OrgNodes, OrgEdges, OrgTerminals, EA, EC, ED or ES), found ${JSON.stringify(content)}`); + } + if (key.field in meta) { + throw new NativeParseError(FORMAT, line, `the Presolve key ${fields[0] ?? ""} is given twice`); + } + const value = fields[1] ?? ""; + meta[key.field] = key.integer ? read.count(value, line, fields[0] ?? "", 0) : read.decimal(value, line, fields[0] ?? ""); + continue; + } + if (section === "graph") { + if (keyword === "nodes" || keyword === "edges") { + if (fields.length !== 2) { + throw new NativeParseError(FORMAT, line, `expected ${fields[0] ?? ""} N, found ${JSON.stringify(content)}`); + } + if (counts[keyword] !== undefined) { + throw new NativeParseError(FORMAT, line, `the ${fields[0] ?? ""} count is given twice`); + } + counts[keyword] = read.count(fields[1] ?? "", line, `the ${keyword} count`, keyword === "nodes" ? 1 : 0); + continue; + } + if (keyword === "e") { + if (fields.length !== EDGE_FIELDS.length) { + throw new NativeParseError(FORMAT, line, `expected an edge E U V WEIGHT, found ${JSON.stringify(content)}`); + } + edges.push({ source: String(node(fields[1] ?? "", line)), target: String(node(fields[2] ?? "", line)), weight: read.scientific(fields[3] ?? "", line, "the edge weight") }); + continue; + } + throw new NativeParseError(FORMAT, line, `expected Nodes, Edges or an edge E U V WEIGHT in the Graph section (directed and obstacle-avoiding instances are not read), found ${JSON.stringify(content)}`); + } + if (keyword === "terminals") { + if (fields.length !== 2 || counts.terminals !== undefined) { + throw new NativeParseError(FORMAT, line, `expected one Terminals N line, found ${JSON.stringify(content)}`); + } + counts.terminals = read.count(fields[1] ?? "", line, "the terminal count", 1); + continue; + } + if (keyword !== "t" || fields.length !== 2) { + throw new NativeParseError(FORMAT, line, `expected Terminals N or a terminal T V in the Terminals section (rooted and prize-collecting instances are not read), found ${JSON.stringify(content)}`); + } + const terminal = node(fields[1] ?? "", line); + if (terminals.has(terminal)) { + throw new NativeParseError(FORMAT, line, `node ${String(terminal)} is a terminal twice`); + } + terminals.add(terminal); + } + if (!graphRead || counts.nodes === undefined) { + throw new NativeParseError(FORMAT, last, "the file has no complete Graph section"); + } + if (!terminalsRead) { + throw new NativeParseError(FORMAT, last, "the file has no complete Terminals section"); + } + if (section !== undefined) { + throw new NativeParseError(FORMAT, last, "the file ends inside a section"); + } + if (!ended) { + throw new NativeParseError(FORMAT, last, "the file has no EOF line"); + } + const nodes: NativeRow[] = Array.from({ length: counts.nodes }, (_, index) => ({ id: String(index + 1), type: terminals.has(index + 1) ? "terminal" : "node" })); + + return { meta, nodes, edges }; +} diff --git a/src/parsers/steinlib.unit.test.ts b/src/parsers/steinlib.unit.test.ts new file mode 100644 index 00000000..d500b894 --- /dev/null +++ b/src/parsers/steinlib.unit.test.ts @@ -0,0 +1,76 @@ +import { describe, expect, it } from "vitest"; +import { parseNative } from "./parse-native"; + +const steinlib = { format: "steinlib", encoding: "utf-8" } as const; + +const bytes = (...lines: readonly string[]): Uint8Array => new TextEncoder().encode(`${lines.join("\n")}\n`); + +/** The odd wheel of the SteinLib format page, shortened to a triangle with a pendant node, with a title case Terminals section as some files write it. */ +const SMALL = ["33D32945 STP File, STP Format Version 1.0", "", "SECTION Comment", 'Name "small"', "Creator Someone", "END", "", "SECTION Graph", "Nodes 4", "Edges 4", "E 1 2 1", "E 2 3 2.5", "E 3 1 1", "E 3 4 7", "END", "", "Section Terminals", "Terminals 2", "T 1", "T 4", "End", "", "EOF"]; + +/** The tail of a presolved instance: a Presolve section with its scalar values and its mapping lines, the original instance's coordinates and terminals after it. */ +const PRESOLVED = ["SECTION Presolve", "Fixed 963412", "Lower 963412", "Upper 1074705235", "Time 13.31", "OrgNodes 9", "OrgEdges 12", "OrgTerminals 3", "EA 1 2 5 1", "EC 3 4 5", "ED 5 6 7", "END", "", "SECTION Coordinates", "DD 1 10 20", "END", "", "SECTION Terminals", "T 7", "END"]; + +describe("the SteinLib parser", () => { + it("reads the nodes with the terminals typed, the weighted edges and the Comment values", () => { + expect(parseNative(bytes(...SMALL), steinlib)).toEqual({ + meta: { name: "small", creator: "Someone" }, + nodes: [ + { id: "1", type: "terminal" }, + { id: "2", type: "node" }, + { id: "3", type: "node" }, + { id: "4", type: "terminal" }, + ], + edges: [ + { source: "1", target: "2", weight: 1 }, + { source: "2", target: "3", weight: 2.5 }, + { source: "3", target: "1", weight: 1 }, + { source: "3", target: "4", weight: 7 }, + ], + }); + }); + + it("reads a Presolve section's values and skips the original instance's mapping, coordinates and terminals", () => { + const parsed = parseNative(bytes(...SMALL.slice(0, -1), ...PRESOLVED, "EOF"), steinlib); + expect(parsed.meta).toEqual({ name: "small", creator: "Someone", fixed: 963412, lower: 963412, upper: 1074705235, presolveTime: 13.31, orgNodes: 9, orgEdges: 12, orgTerminals: 3 }); + expect(parsed.nodes).toEqual(parseNative(bytes(...SMALL), steinlib).nodes); + }); + + it("refuses a file that disagrees with the format", () => { + expect(() => parseNative(bytes(...SMALL.slice(1)), steinlib)).toThrow(/line 2: expected the magic number line/); + expect(() => parseNative(bytes(...SMALL.filter((line) => line !== "E 3 4 7")), steinlib)).toThrow(/line 14: expected 4 edges, found 3/); + expect(() => parseNative(bytes(...SMALL.map((line) => (line === "T 4" ? "T 5" : line))), steinlib)).toThrow(/line 20: expected a node from 1 to 4, found 5/); + expect(() => parseNative(bytes(...SMALL.map((line) => (line === "T 4" ? "T 1" : line))), steinlib)).toThrow(/line 20: node 1 is a terminal twice/); + expect(() => parseNative(bytes(...SMALL.map((line) => (line === "E 3 4 7" ? "A 3 4 7" : line))), steinlib)).toThrow(/line 14: expected Nodes, Edges or an edge E U V WEIGHT/); + expect(() => parseNative(bytes(...SMALL.map((line) => (line === "T 4" ? "Root 4" : line))), steinlib)).toThrow(/line 20: expected Terminals N or a terminal T V/); + expect(() => parseNative(bytes(...SMALL.map((line) => (line === "SECTION Comment" ? "SECTION MaximumDegrees" : line))), steinlib)).toThrow(/line 3: the section "MaximumDegrees" is not one the parser reads/); + expect(() => parseNative(bytes(...SMALL.slice(0, -1)), steinlib)).toThrow(/the file has no EOF line/); + expect(() => parseNative(bytes(...SMALL, "E 1 2 3"), steinlib)).toThrow(/line 24: a line after EOF/); + }); + + it("refuses a file with no Terminals section rather than typing every node as a plain node", () => { + const withoutTerminals = SMALL.slice(0, SMALL.indexOf("Section Terminals")); + expect(() => parseNative(bytes(...withoutTerminals, "EOF"), steinlib)).toThrow(/steinlib.*the file has no complete Terminals section/); + }); + + it("refuses a Presolve section before the instance's own Terminals section instead of skipping that section", () => { + const own = SMALL.indexOf("Section Terminals"); + const reordered = [...SMALL.slice(0, own), ...PRESOLVED.slice(0, PRESOLVED.indexOf("SECTION Coordinates")), ...SMALL.slice(own, -1), "EOF"]; + expect(() => parseNative(bytes(...reordered), steinlib)).toThrow(/a Presolve section before the instance's Terminals section/); + }); + + it("reads CRLF line endings", () => { + const crlf = new TextEncoder().encode(`${SMALL.join("\r\n")}\r\n`); + expect(parseNative(crlf, steinlib)).toEqual(parseNative(bytes(...SMALL), steinlib)); + }); + + it("ignores # comment lines and trailing comments", () => { + const commented = ["# a comment", ...SMALL.map((line) => (line === "E 3 4 7" ? "E 3 4 7 # heavy" : line)), "# after"]; + expect(parseNative(bytes(...commented), steinlib)).toEqual(parseNative(bytes(...SMALL), steinlib)); + }); + + it("reads an edge weight written with an exponent", () => { + const parsed = parseNative(bytes(...SMALL.map((line) => (line === "E 3 4 7" ? "E 3 4 1.5e+3" : line))), steinlib); + expect(parsed.edges[3]).toEqual({ source: "3", target: "4", weight: 1500 }); + }); +}); diff --git a/src/parsers/tntp.ts b/src/parsers/tntp.ts new file mode 100644 index 00000000..cf35a84d --- /dev/null +++ b/src/parsers/tntp.ts @@ -0,0 +1,331 @@ +import { NativeParseError } from "../errors"; +import type { NativeAttributedEdge, NativeAttributedGraph, NativeRow } from "./native"; +import { lines } from "./native"; +import { numberReader } from "./number-lines"; + +/** What every error from the network file names as its format. */ +const NETWORK = "tntp network"; +/** What every error from the trip table names as its format. */ +const TRIPS = "tntp trips"; + +/** A metadata line, ` value`, the value possibly empty. */ +const METADATA = /^<([^<>]+)>\s*(.*)$/; +/** The key that ends a file's metadata. */ +const END_OF_METADATA = "END OF METADATA"; +/** A whole number of at least one, as a node or zone number is written. */ +const POSITIVE_INTEGER = /^\d+$/; +/** A column name of the harmonised header line, which the files write in lower snake case (`free_flow_time`). */ +const COLUMN_NAME = /^[a-z][a-z0-9]*(?:_[a-z0-9]+)*$/; +/** The first two columns of the harmonised header line, the link's ends, which every network file's links begin with. */ +const ENDS = ["init_node", "term_node"] as const; +/** The field names a link's column must not take, since the edge already carries them. */ +const RESERVED = new Set(["source", "target", "type"]); +/** An origin line of a trip table, `Origin N`. */ +const ORIGIN = /^Origin\s+(\S+)$/; +/** One entry of a trip table's destination list, `D : FLOW`. */ +const DESTINATION = /^(\S+)\s*:\s*(\S+)$/; + +/** The network file's metadata that the parser reads as numbers. */ +const NETWORK_COUNTS = { zones: "NUMBER OF ZONES", nodes: "NUMBER OF NODES", firstThruNode: "FIRST THRU NODE", links: "NUMBER OF LINKS" } as const; +/** The network file's metadata key that restates the source's own column header, carried as text. */ +const ORIGINAL_HEADER = "ORIGINAL HEADER"; +/** The trip table's zone count, which must be the network's. */ +const TRIP_ZONES = "NUMBER OF ZONES"; +/** The trip table's total flow, carried as a number. */ +const TOTAL_OD_FLOW = "TOTAL OD FLOW"; + +/** The two files of one TNTP dataset, each decoded to text. */ +export interface TntpFiles { + readonly network: string; + readonly trips: string; +} + +/** + * Turns a TNTP name (a metadata key in upper case words, or a column in lower snake case) into the camel case the catalogue's fields use: `FIRST THRU NODE` is `firstThruNode`, `free_flow_time` is `freeFlowTime`. + * @param name - The name as the file writes it. + * @returns The field name. + */ +function camel(name: string): string { + const [first = "", ...rest] = name.toLowerCase().split(/[\s_]+/); + + return first + rest.map((word) => word.charAt(0).toUpperCase() + word.slice(1)).join(""); +} + +const networkRead = numberReader(NETWORK); +const tripsRead = numberReader(TRIPS); + +/** + * Makes the reader of a node or zone number of one file. + * @param format - The file's format, which every error names. + * @returns A function reading a field as a number from 1 to `count`: it takes the field as written, its 1-based line, what it numbers and the largest number allowed, and throws NativeParseError when the field is not a whole number in that range. + */ +function numberedReader(format: string): (field: string, line: number, what: string, count: number) => number { + return (field, line, what, count) => { + const value = Number(field); + if (!POSITIVE_INTEGER.test(field) || value < 1 || value > count) { + throw new NativeParseError(format, line, `expected a ${what} from 1 to ${String(count)}, found ${JSON.stringify(field)}`); + } + + return value; + }; +} + +const networkNumbered = numberedReader(NETWORK); +const tripsNumbered = numberedReader(TRIPS); + +/** A file's metadata and the 0-based index of the first line after it. */ +interface Metadata { + readonly values: ReadonlyMap; + readonly next: number; +} + +/** + * Reads the metadata lines that open a TNTP file, up to and including ``. + * @param format - The file's format, for errors. + * @param all - The file's lines. + * @returns The values by key and where the body starts. + * @throws NativeParseError for a line before the end of the metadata that is not a metadata line, a key given twice, or a file with no end of metadata. + */ +function readMetadata(format: string, all: readonly string[]): Metadata { + const values = new Map(); + for (const [index, raw] of all.entries()) { + const text = raw.trim(); + if (text === "") { + continue; + } + const match = METADATA.exec(text); + if (match === null) { + throw new NativeParseError(format, index + 1, `expected a metadata line value before <${END_OF_METADATA}>, found ${JSON.stringify(text)}`); + } + const [, key = "", value = ""] = match; + if (key === END_OF_METADATA) { + return { values, next: index + 1 }; + } + if (values.has(key)) { + throw new NativeParseError(format, index + 1, `the metadata key <${key}> is given twice`); + } + values.set(key, { value: value.trim(), line: index + 1 }); + } + throw new NativeParseError(format, Math.max(all.length, 1), `the file has no <${END_OF_METADATA}> line`); +} + +/** + * Reads a metadata value as a whole number of at least `least`. + * @param format - The file's format, for the error. + * @param metadata - The file's metadata. + * @param key - The key. + * @param least - The smallest value it may take. + * @returns The number and the 1-based line it is on. + * @throws NativeParseError when the key is missing or its value is not such a number. + */ +function metadataCount(format: string, metadata: Metadata, key: string, least: number): { readonly value: number; readonly line: number } { + const entry = metadata.values.get(key); + if (entry === undefined) { + throw new NativeParseError(format, metadata.next, `the metadata has no <${key}>`); + } + const value = Number(entry.value); + if (!/^\d+$/.test(entry.value) || value < least) { + throw new NativeParseError(format, entry.line, `expected <${key}> as a whole number of at least ${String(least)}, found ${JSON.stringify(entry.value)}`); + } + + return { value, line: entry.line }; +} + +/** + * Gives the metadata a parser does not read itself as text, each under its key in camel case, so no value the file states is dropped. + * @param format - The file's format, for the error. + * @param metadata - The file's metadata. + * @param read - The keys the parser has read itself. + * @param taken - The field names the instance's meta already has. + * @returns The text fields. + * @throws NativeParseError when a key's field name is one the meta already has. + */ +function textMetadata(format: string, metadata: Metadata, read: ReadonlySet, taken: ReadonlySet): Record { + const fields: Record = {}; + for (const [key, { value, line }] of metadata.values) { + if (read.has(key)) { + continue; + } + const name = camel(key); + if (taken.has(name) || name in fields) { + throw new NativeParseError(format, line, `the metadata key <${key}> gives the field ${name}, which the dataset already has`); + } + fields[name] = value; + } + + return fields; +} + +/** + * Reads the harmonised column header line's names after `~`, checking that each further column can name a link field. + * @param names - The names the line gives, the link's ends first. + * @param line - Its 1-based line. + * @returns The link field each further column is read into, in camel case. + * @throws NativeParseError when a column is not a lower snake case name, takes a name the edge already carries, or is named twice. + */ +function linkFields(names: readonly string[], line: number): string[] { + const columns = names.slice(ENDS.length); + for (const name of columns) { + if (!COLUMN_NAME.test(name) || RESERVED.has(camel(name))) { + throw new NativeParseError(NETWORK, line, `the column header names a column ${JSON.stringify(name)} that is not a lower snake case name a link field can take`); + } + } + const fields = columns.map(camel); + if (new Set(fields).size !== fields.length) { + throw new NativeParseError(NETWORK, line, "the column header names a column twice"); + } + + return fields; +} + +/** The network a TNTP network file describes: its counts and the fields of its links. */ +interface Network { + readonly zones: number; + readonly nodes: number; + readonly columns: readonly string[]; + readonly edges: NativeAttributedEdge[]; + readonly meta: Record; +} + +/** + * Reads a TNTP network file (`_net.tntp`). + * @param text - The file's text. + * @returns The network. + * @throws NativeParseError as `parseTntp` documents. + */ +function readNetwork(text: string): Network { + const all = lines(text); + const metadata = readMetadata(NETWORK, all); + const { value: zones, line: zonesLine } = metadataCount(NETWORK, metadata, NETWORK_COUNTS.zones, 1); + const { value: nodes } = metadataCount(NETWORK, metadata, NETWORK_COUNTS.nodes, 1); + const { value: firstThruNode, line: firstThruNodeLine } = metadataCount(NETWORK, metadata, NETWORK_COUNTS.firstThruNode, 1); + const { value: links } = metadataCount(NETWORK, metadata, NETWORK_COUNTS.links, 0); + if (zones > nodes) { + throw new NativeParseError(NETWORK, zonesLine, `the zone count ${String(zones)} is above the node count ${String(nodes)}`); + } + if (firstThruNode > nodes) { + throw new NativeParseError(NETWORK, firstThruNodeLine, `the first through node ${String(firstThruNode)} is above the node count ${String(nodes)}`); + } + const originalHeader = metadata.values.get(ORIGINAL_HEADER); + const counted: Record = { zones, firstThruNode, ...(originalHeader === undefined ? {} : { originalHeader: originalHeader.value }) }; + const meta: Record = { ...counted, ...textMetadata(NETWORK, metadata, new Set([...Object.values(NETWORK_COUNTS), ORIGINAL_HEADER]), new Set(Object.keys(counted))) }; + + let header: { readonly names: readonly string[]; readonly fields: readonly string[] } | undefined; + const edges: NativeAttributedEdge[] = []; + for (let index = metadata.next; index < all.length; index += 1) { + const line = index + 1; + const raw = (all[index] ?? "").trim(); + if (raw === "") { + continue; + } + if (raw.startsWith("~")) { + const names = raw.slice(1).replace(/;\s*$/, "").trim().split(/\s+/); + if (names[0] === ENDS[0] && names[1] === ENDS[1]) { + if (header !== undefined) { + throw new NativeParseError(NETWORK, line, "the column header line is given twice"); + } + header = { names, fields: linkFields(names, line) }; + } + continue; + } + if (header === undefined) { + throw new NativeParseError(NETWORK, line, `a link line before the column header line ~ ${ENDS.join(" ")} ...`); + } + if (!raw.endsWith(";")) { + throw new NativeParseError(NETWORK, line, "expected a link line ending with ;"); + } + const values = raw.slice(0, -1).trim().split(/\s+/); + if (values.length !== header.names.length) { + throw new NativeParseError(NETWORK, line, `expected a link line of the header's ${String(header.names.length)} columns, found ${String(values.length)} values`); + } + const [init = "", term = "", ...rest] = values; + const source = String(networkNumbered(init, line, "node", nodes)); + const target = String(networkNumbered(term, line, "node", nodes)); + const fields: Record = {}; + for (const [position, field] of header.fields.entries()) { + fields[field] = networkRead.scientific(rest[position] ?? "", line, `the link's ${header.names[position + ENDS.length] ?? field}`); + } + edges.push({ source, target, type: "link", ...fields }); + } + if (header === undefined) { + throw new NativeParseError(NETWORK, all.length, `the file has no column header line ~ ${ENDS.join(" ")} ...`); + } + if (edges.length !== links) { + throw new NativeParseError(NETWORK, all.length, `expected ${String(links)} links, found ${String(edges.length)}`); + } + + return { zones, nodes, columns: header.fields, edges, meta }; +} + +/** + * Parses one dataset of Transportation Networks for Research in the TNTP format (https://github.com/bstabler/TransportationNetworks, "TNTP Data format"): its network file and its trip table, read together into one graph. + * + * The network file opens with ` value` metadata lines up to ``, of which ``, ``, `` and `` are required whole numbers; then `~` comment lines, one of which is the harmonised column header `~ init_node term_node ... ;`; then one link per line, its values separated by whitespace and the line ended by `;`. Nodes are `1` to the node count, those up to the zone count of type `zone` (the zones, which are also nodes of the road network) and the rest of type `node`; each link is a directed edge of type `link` from its init node to its term node, in file order, with one field per further column, named in camel case (`free_flow_time` is `freeFlowTime`) and read as a number. Parallel links are kept. + * + * The trip table opens with metadata whose `` must equal the network's and whose `` is a number; then `~` comment lines and one block per origin, `Origin N` followed by `D : FLOW;` entries on any number of lines. Each entry is a commodity, following the graph/v1 convention for commodities: a node `commodity--` of type `commodity` with its `demand` (the flow as written, zero included), joined by an `origin` edge and a `destination` edge to the zones it runs between. + * + * Every node carries `demand` (null but for a commodity) and every edge every link field (null on the commodity edges), as a crosswalk needs. `meta` holds `zones`, `firstThruNode` and `totalOdFlow`, the network's `` as `originalHeader`, and any other metadata of either file as text under its key in camel case. + * @param files - The two files' text. + * @returns The native graph. + * @throws NativeParseError naming the file and line for a metadata line before the end of the metadata that is not one, a key given twice or missing, a count that is not a whole number, a zone or first through node count above the node count, a second column header line, a link before the column header or of another number of values, a link line without its `;`, a node or zone outside its range, a value that is not a number, a link count other than the metadata's, a trip table whose zone count is not the network's, an entry before any origin or not of the form `D : FLOW`, a negative flow, or an origin and destination pair given twice. + */ +export function parseTntp(files: TntpFiles): NativeAttributedGraph { + const network = readNetwork(files.network); + const all = lines(files.trips); + const metadata = readMetadata(TRIPS, all); + const { value: zones, line: zonesLine } = metadataCount(TRIPS, metadata, TRIP_ZONES, 1); + if (zones !== network.zones) { + throw new NativeParseError(TRIPS, zonesLine, `the trip table has ${String(zones)} zones and the network ${String(network.zones)}`); + } + const total = metadata.values.get(TOTAL_OD_FLOW); + if (total === undefined) { + throw new NativeParseError(TRIPS, metadata.next, `the metadata has no <${TOTAL_OD_FLOW}>`); + } + const counted: Record = { ...network.meta, totalOdFlow: tripsRead.scientific(total.value, total.line, `<${TOTAL_OD_FLOW}>`) }; + const meta: Record = { ...counted, ...textMetadata(TRIPS, metadata, new Set([TRIP_ZONES, TOTAL_OD_FLOW]), new Set(Object.keys(counted))) }; + + const nulls = Object.fromEntries(network.columns.map((field) => [field, null])); + const nodes: NativeRow[] = Array.from({ length: network.nodes }, (_, index) => ({ id: String(index + 1), type: index < network.zones ? "zone" : "node", demand: null })); + const edges: NativeAttributedEdge[] = network.edges; + const pairs = new Set(); + let origin: number | undefined; + for (let index = metadata.next; index < all.length; index += 1) { + const line = index + 1; + const raw = (all[index] ?? "").trim(); + if (raw === "" || raw.startsWith("~")) { + continue; + } + const start = ORIGIN.exec(raw); + if (start !== null) { + origin = tripsNumbered(start[1] ?? "", line, "zone", zones); + continue; + } + if (origin === undefined) { + throw new NativeParseError(TRIPS, line, `expected an origin line Origin N, found ${JSON.stringify(raw)}`); + } + for (const piece of raw.split(";")) { + const entry = piece.trim(); + if (entry === "") { + continue; + } + const match = DESTINATION.exec(entry); + if (match === null) { + throw new NativeParseError(TRIPS, line, `expected a destination entry D : FLOW, found ${JSON.stringify(entry)}`); + } + const destination = tripsNumbered(match[1] ?? "", line, "zone", zones); + const demand = tripsRead.scientific(match[2] ?? "", line, "the flow"); + if (demand < 0) { + throw new NativeParseError(TRIPS, line, `expected a flow of at least 0, found ${match[2] ?? ""}`); + } + const id = `commodity-${String(origin)}-${String(destination)}`; + if (pairs.has(id)) { + throw new NativeParseError(TRIPS, line, `the flow from zone ${String(origin)} to zone ${String(destination)} is given twice`); + } + pairs.add(id); + nodes.push({ id, type: "commodity", demand }); + edges.push({ source: id, target: String(origin), type: "origin", ...nulls }, { source: id, target: String(destination), type: "destination", ...nulls }); + } + } + + return { meta, nodes, edges }; +} diff --git a/src/parsers/tntp.unit.test.ts b/src/parsers/tntp.unit.test.ts new file mode 100644 index 00000000..067344d2 --- /dev/null +++ b/src/parsers/tntp.unit.test.ts @@ -0,0 +1,114 @@ +import { describe, expect, it } from "vitest"; +import { parseNativeParts } from "./parse-native"; + +const tntp = { format: "tntp", encoding: "utf-8" } as const; + +const bytes = (...lines: readonly string[]): Uint8Array => new TextEncoder().encode(`${lines.join("\n")}\n`); + +/** A small network laid out as the repository's network files are: metadata, blank lines, the harmonised column header and tab-separated links ended by `;`. */ +const NETWORK = [ + " 2\t\t", + " 3", + " 3", + " 3\t", + "~ \tInit node \tTerm node \tCapacity \tFree Flow Time \t;", + "\t", + "", + "", + "~\tinit_node\tterm_node\tcapacity\tfree_flow_time\t;", + "\t1\t3\t25900.20064\t6\t;", + "\t3\t2\t4958.180928\t5\t;", + "\t3\t2\t1.5e+003\t7\t;", +]; + +/** A trip table laid out as the repository's are: metadata, comments, and each origin's destination entries over one or more lines. */ +const TRIPS = [" 2", " 1.5e+002", " \"Somewhere\"", "", "~ a comment", "", "Origin \t1 ", " 1 : 0.0; 2 : 100.0; ", "", "Origin \t2 ", " 1 : 50.0; "]; + +const parse = (network: readonly string[], trips: readonly string[]): ReturnType => + parseNativeParts( + [ + { name: "network", bytes: bytes(...network) }, + { name: "trips", bytes: bytes(...trips) }, + ], + tntp, + ); + +describe("the TNTP parser", () => { + it("reads the zones, nodes and links of the network and each origin and destination pair as a commodity joined to its zones", () => { + expect(parse(NETWORK, TRIPS)).toEqual({ + meta: { zones: 2, firstThruNode: 3, originalHeader: "~ \tInit node \tTerm node \tCapacity \tFree Flow Time \t;", totalOdFlow: 150, location: "\"Somewhere\"" }, + nodes: [ + { id: "1", type: "zone", demand: null }, + { id: "2", type: "zone", demand: null }, + { id: "3", type: "node", demand: null }, + { id: "commodity-1-1", type: "commodity", demand: 0 }, + { id: "commodity-1-2", type: "commodity", demand: 100 }, + { id: "commodity-2-1", type: "commodity", demand: 50 }, + ], + edges: [ + { source: "1", target: "3", type: "link", capacity: 25900.20064, freeFlowTime: 6 }, + { source: "3", target: "2", type: "link", capacity: 4958.180928, freeFlowTime: 5 }, + { source: "3", target: "2", type: "link", capacity: 1500, freeFlowTime: 7 }, + { source: "commodity-1-1", target: "1", type: "origin", capacity: null, freeFlowTime: null }, + { source: "commodity-1-1", target: "1", type: "destination", capacity: null, freeFlowTime: null }, + { source: "commodity-1-2", target: "1", type: "origin", capacity: null, freeFlowTime: null }, + { source: "commodity-1-2", target: "2", type: "destination", capacity: null, freeFlowTime: null }, + { source: "commodity-2-1", target: "2", type: "origin", capacity: null, freeFlowTime: null }, + { source: "commodity-2-1", target: "1", type: "destination", capacity: null, freeFlowTime: null }, + ], + }); + }); + + it("reads CRLF line ends alike", () => { + const crlf = (lines: readonly string[]): Uint8Array => new TextEncoder().encode(`${lines.join("\r\n")}\r\n`); + expect( + parseNativeParts( + [ + { name: "network", bytes: crlf(NETWORK) }, + { name: "trips", bytes: crlf(TRIPS) }, + ], + tntp, + ), + ).toEqual(parse(NETWORK, TRIPS)); + }); + + it("refuses a network whose metadata or links disagree with the format", () => { + expect(() => parse(NETWORK.filter((line) => !line.startsWith("")), TRIPS)).toThrow(/tntp network line \d+: the metadata has no /); + expect(() => parse(NETWORK.slice(0, -1), TRIPS)).toThrow(/expected 3 links, found 2/); + expect(() => parse(NETWORK.map((line) => line.replace("\t3\t2\t4958.180928\t5\t;", "\t3\t2\t4958.180928\t5\t1;")), TRIPS)).toThrow(/tntp network line 11: expected a link line of the header's 4 columns, found 5 values/); + expect(() => parse(NETWORK.map((line) => line.replace("\t1\t3\t25900.20064\t6\t;", "\t1\t4\t25900.20064\t6\t;")), TRIPS)).toThrow(/line 10: expected a node from 1 to 3, found "4"/); + expect(() => parse(NETWORK.map((line) => line.replace("\t1\t3\t25900.20064\t6\t;", "\t1\t3\t25900.20064\t6")), TRIPS)).toThrow(/line 10: expected a link line ending with ;/); + expect(() => parse(NETWORK.filter((line) => !line.startsWith("~")), TRIPS)).toThrow(/line 9: a link line before the column header line/); + expect(() => parse(NETWORK.map((line) => line.replace("", "END")), TRIPS)).toThrow(/line 6: expected a metadata line/); + }); + + it("refuses a trip table that disagrees with the network or the format", () => { + expect(() => parse(NETWORK, TRIPS.map((line) => line.replace(" 2", " 3")))).toThrow(/tntp trips line 1: the trip table has 3 zones and the network 2/); + expect(() => parse(NETWORK, TRIPS.map((line) => line.replace(" 1 : 50.0; ", " 3 : 50.0; ")))).toThrow(/tntp trips line 11: expected a zone from 1 to 2, found "3"/); + expect(() => parse(NETWORK, [...TRIPS, " 1 : 5.0; "])).toThrow(/line 12: the flow from zone 2 to zone 1 is given twice/); + expect(() => parse(NETWORK, TRIPS.map((line) => line.replace("2 : 100.0", "2 : -100.0")))).toThrow(/line 8: expected a flow of at least 0/); + expect(() => parse(NETWORK, TRIPS.filter((line) => !line.startsWith("Origin \t1")))).toThrow(/line 7: expected an origin line Origin N/); + expect(() => parse(NETWORK, TRIPS.map((line) => line.replace("2 : 100.0", "2 = 100.0")))).toThrow(/line 8: expected a destination entry D : FLOW/); + }); + + it("reads a link value and the total flow written with an exponent", () => { + const parsed = parse(NETWORK.map((line) => line.replace("\t3\t2\t4958.180928\t5\t;", "\t3\t2\t4.95818e+3\t5E-1\t;")), TRIPS); + expect(parsed.edges[1]).toMatchObject({ capacity: 4958.18, freeFlowTime: 0.5 }); + expect(parsed).toMatchObject({ meta: { totalOdFlow: 150 } }); + }); + + it("refuses a metadata key given twice, in either file", () => { + expect(() => parse([NETWORK[0] ?? "", NETWORK[1] ?? "", " 3", ...NETWORK.slice(2)], TRIPS)).toThrow(/tntp network line 3: the metadata key is given twice/); + expect(() => parse(NETWORK, [TRIPS[0] ?? "", TRIPS[0] ?? "", ...TRIPS.slice(1)])).toThrow(/tntp trips line 2: the metadata key is given twice/); + }); + + it("refuses a zone count or first through node above the node count", () => { + expect(() => parse(NETWORK.map((line) => line.replace(" 2", " 4")), TRIPS)).toThrow(/tntp network line 1: the zone count 4 is above the node count 3/); + expect(() => parse(NETWORK.map((line) => line.replace(" 3", " 4")), TRIPS)).toThrow(/tntp network line 3: the first through node 4 is above the node count 3/); + }); + + it("refuses a second column header line rather than ignoring it", () => { + const header = "~\tinit_node\tterm_node\tcapacity\tfree_flow_time\t;"; + expect(() => parse(NETWORK.flatMap((line) => (line === header ? [line, line] : [line])), TRIPS)).toThrow(/tntp network line 10: the column header line is given twice/); + }); +});