create an "atlas" or "cheat sheet" for accessing workflow-relevant information

[SteVwonder]

We have a CLI for poking at job-specific information stored within the KVS: job info, but @Larofeticus pointed out that having some documentation about where to find job-specific information within the KVS might also be helpful. To start, maybe we can make a table of various workflow/job-relevant information, where it is stored in the KVS and if/where it is stored in the environment, and maybe even where you used to be able to find it in Slurm etc.

Data that comes to mind:

• jobid
• job size (in nodes)
• rank id
• hostnames for all nodes in job
• job endtime
• cpu mask (i.e., allocated cores)
• job account/bank
• job working directory
• job stdout/stderr location (either file or KVS stream)

[wihobbs]

This was discussed today at our Flux weekly developers meeting.

@garlick mentioned that several of these items are documented in Flux RFCs. Specifically, RFCs 16, 18, and 21 seem relevant. Based on my first read I will try to summarize how each one relates to this issue:

RFC 16 - More generally describes the KVS event log and the content produced by it.
RFC 18 - Provides some keywords for the KVS log, such as timestamp, name, and context, and what they do.
RFC 21 - Describes the states that get logged to KVS, with example KVS entries for each logged change in state.

@chu11 suggested that we have some flux-specific commands to get at this information, specifically:

• flux job info (described above in initial comment)
• flux job attach for getting stdout/stderr location.

Just starting a thread here to document as we get more information about KVS out.

[chu11]

Just to add, flux jobs is the more user friendly facing tool for getting tons of job information (see manpage for list of things you can retrieve). flux job list or flux job list-ids <JOBID> can also be used to get all raw job list data in a json dump.