Split comm method#89
Draft
TomMelt wants to merge 9 commits into
Draft
Conversation
- Implement Grid copy constructor (previously deleted) - Add set_global(), set_globalExt(), and recompute_ids() to allow modification of grid geometry after construction - In tripolar mode, partition a subgrid copy with adjusted extents instead of the original grid
Add setter/getter accessors for internal partitioning state (total number of procs, communicator, global coordinates, local extents) on both Partitioner and Grid classes. Moves discover_neighbours() from private to public so it can be called after partition results are populated.
For tripolar mode, create a separate subpartitioner for each sub-grid, then merge the results into the main partitioner before calling discover_neighbours(). This also enables metadata saving in tripolar mode by removing the previous known limitation.
Move the 6 grid-loading statements from ZoltanPartitioner::partition() into a new ZoltanPartitioner::initialize(Grid&) method. Add the corresponding initialize() method to the Partitioner base class as a pure virtual function so all future partitioners must implement it. This improves separation of concerns: initialize() handles setting method variables (_numProcs, _globalExt, _global, _localExt, _px, _py) from the grid, while partition() focuses on the partitioning logic itself.
Expose the process ID vector of the latest 2D domain decomposition via getter and setter methods.
…ar mode - Move the MPI color variable outside the tripolar conditional so it's available for downstream use. - Add partitioner initialization for the tripolar case. - Remove redundant setTotalNumProcs/setGlobalExt calls. - Offset process IDs of the East half (color==1) by numProcs/2 to avoid duplicates when merging results from both sub-communicators. - Restore mask and metadata saving for both tripolar and non-tripolar modes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP Do Not Merge
Two-stage tripolar domain decomposition using split MPI communicators
fixes #85
Summary
Implement two-stage tripolar domain decomposition: when the
--tripolarflag is passed, the domain is split East/West by coordinate viaMPI_Comm_split, each half is partitioned independently with Zoltan, and results are merged back into a single global partitioning. The basepartition()flow is preserved for non-tripolar runs.Changes
main.cpp--tripolar(-t) CLI flag to enable split-communicator modeMPI_COMM_WORLDinto East/West sub-communicators usingMPI_Comm_split(color = 0 West, 1 East, midpoint =globalExt[0] / 2)subgrid) for sub-communicator worksubgrid+subpartitionerwithin each sub-communicator, or fall through to standard single-communicator pathpartitioner: copyGlobalNew,LocalExtNew, and adjustProcIdoffsets for the East halfdiscover_neighbours()after merge to rebuild neighbour info on the unified communicatorsubgridresourcesGrid.cpp/Grid.hppGrid(const Grid& other)copy ctor (needed for subgrid creation)set_comm(),set_global(),set_globalExt()setters to allow reconfiguring a grid copy for sub-communicator userecompute_ids()to regenerate_global_id/_local_idafter the grid's bounding box changes (East/West halves get different global positions)_ignore_maskmember (defaultsfalse) to control whether the land mask is applied during id recomputationPartitioner.cpp/Partitioner.hppdiscover_neighbours()from protected to publicinitialize(Grid&)virtual method to basePartitioner— extracts grid extents/bounding-box/periodicity into partitioner member variables, separating setup from partitioningsetTotalNumProcs,setComm,get/setLocalExtNew,get/setGlobalNew,get/setGlobal,get/setGlobalExt,get/setProcIdZoltanPartitioner.cpp/ZoltanPartitioner.hppinitialize(Grid&)by copying the grid setup logic previously insidepartition()partition(Grid&)callinitialize(grid)first, preserving all original partitioning logicdiscover_neighbours()call from insidepartition()(now called once after merge inmain)