Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,19 @@ The visual health indicator for a Pipeline. Derived from the latest Execution st
| 🟢 Green | Healthy | `Succeeded` |
| 🔴 Red | Broken or blocked | `Failed`, `Stopped` |
| 🟡 Yellow | In progress | `InProgress` |
| ⏸ Awaiting approval | Waiting on a person | `InProgress` at a manual approval action with an open approval token |
| ⚪ Grey | No signal | `Superseded`, no executions yet |

Awaiting approval ranks between Yellow and Red: it needs a person, not more time, but a failure still outranks it. It is raised in `state.FromData` from the action's approval token, because execution statuses alone can't tell an approval gate from a running build.

_Avoid_: badge, indicator, light

**Approval**: A manual approval action waiting on a decision. `GetPipelineState` returns a token for it only while it is open; approving or rejecting (`a` / `x`) sends that token to `PutApprovalResult`. The token comes from the last poll, so it can already be stale if someone decided in the console; that is reported as "already decided", not as a failure. A pipeline awaiting approval never raises a stuck alert.
_Avoid_: gate (in prose), sign-off

## Active-first sorting

Pipelines are sorted by Stoplight priority so the most actionable items appear at the top: 🟡 in-progress → 🔴 failing → 🟢 passing → ⚪ no signal. Order is stable within each tier.
Pipelines are sorted by Stoplight priority so the most actionable items appear at the top: ⏸ awaiting approval → 🟡 in-progress → 🔴 failing → 🟢 passing → ⚪ no signal. Order is stable within each tier.
_Avoid_: bubbling, floating

## Dashboard tree
Expand Down Expand Up @@ -111,4 +117,4 @@ _Avoid_: task fetch, stopped tasks (use Task detail)
## Flagged ambiguities

- "account" vs "profile" — resolved: Account is the logical concept; profile is the credential mechanism
- "manual approval" — open: whether to show approval gates as a special Stoplight state (e.g. ⏸) is not yet decided
- "manual approval" — resolved: approval gates get their own ⏸ Stoplight state (see **Approval**)
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ A terminal dashboard for live AWS resource health across multiple accounts and r

## Features

- **Stoplight-per-project** — 🟢 🔴 🟡 ⚪ derived from worst-case across pipeline, CloudFormation stacks, and ECS services
- **Stoplight-per-project** — 🟢 🔴 🟡 ⏸ ⚪ derived from worst-case across pipeline, CloudFormation stacks, and ECS services
- **Per-resource-type summary** — collapsed row shows `Pipeline 🟡 Stacks 🟢 ECS 🟢` at a glance
- **Stage-level expand** — expand any project to see pipeline stages, stack statuses with elapsed timers, and ECS running/desired task counts
- **Active-first sorting** — in-progress and failing projects surface to the top automatically
Expand All @@ -17,6 +17,7 @@ A terminal dashboard for live AWS resource health across multiple accounts and r
- **CloudFormation monitoring** — maps stack status to stoplight; in-progress stacks show elapsed timer
- **ECS service monitoring** — shows running/desired task counts; flags active deployments. Stopped-task detail is fetched only for services that are actually unhealthy, so a green fleet costs one call per cluster
- **Throttle-aware** — AWS clients use the SDK's adaptive retry mode, self-throttling before AWS has to reject a burst
- **Manual approvals** — pipelines waiting at an approval gate show ⏸ and sort to the top; approve or reject from the dashboard with `a` / `x`
- **Stuck alerts** — POSTs a signed JSON event to your webhooks when a pipeline, stack, or service stays wedged past a threshold, once per incident
- **Multi-account** — per-account AWS profile config with named profiles or environment credentials
- **In-TUI project management** — add, edit, delete, and enable/disable projects without leaving the terminal
Expand Down Expand Up @@ -106,7 +107,7 @@ A dashboard only helps when someone is looking at it. Configure one or more
`[[webhooks]]` and aws-green POSTs a JSON event the moment a resource has been
wedged for longer than `stuck_threshold_minutes` (default `30`):

- **Pipeline** — a stage of the latest execution is `Failed`/`Stopped`, or still `InProgress`
- **Pipeline** — a stage of the latest execution is `Failed`/`Stopped`, or still `InProgress` (a pipeline waiting on a manual approval is not stuck, and never alerts)
- **CloudFormation stack** — any `*_IN_PROGRESS` or `*_FAILED` status
- **ECS service** — running task count does not match desired

Expand Down Expand Up @@ -185,6 +186,9 @@ aws-green uses the standard AWS credential chain via `aws-sdk-go-v2`. Any of the
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, etc.)
- IAM instance/task roles when running on AWS

Approving or rejecting from the dashboard (`a` / `x`) additionally needs
`codepipeline:PutApprovalResult` on the pipeline for that profile.

## Keybindings

### Dashboard
Expand All @@ -196,6 +200,7 @@ aws-green uses the standard AWS credential chain via `aws-sdk-go-v2`. Any of the
| `enter` / `space` | Expand / collapse project row |
| `r` | Force refresh |
| `f` | Smart fix (restart pipeline / force deploy / continue rollback) |
| `a` / `x` | Approve / reject the selected pipeline's pending manual approval (⏸) |
| `o` | Open pipeline in AWS Console |
| `m` | Open project manager |
| `q` | Quit |
Expand Down
9 changes: 8 additions & 1 deletion internal/aggregator/aggregator.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ const (
StoplightGrey Stoplight = iota // no executions, or superseded
StoplightGreen // succeeded
StoplightYellow // in progress
StoplightRed // failed or stopped
// StoplightAwaitingApproval is a pipeline paused at a manual approval. It
// outranks yellow because it needs a person, not more time. Aggregate never
// returns it: execution statuses can't tell a gate from a build, so
// state.FromData raises it from the action's approval token.
StoplightAwaitingApproval
StoplightRed // failed or stopped
)

func (s Stoplight) String() string {
Expand All @@ -18,6 +23,8 @@ func (s Stoplight) String() string {
return "🔴"
case StoplightYellow:
return "🟡"
case StoplightAwaitingApproval:
return "⏸"
default:
return "⚪"
}
Expand Down
14 changes: 14 additions & 0 deletions internal/aggregator/aggregator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,17 @@ func TestStoplight_String(t *testing.T) {
}
}
}

// A pipeline waiting on a person outranks one that is merely running, but a
// failure anywhere still outranks both.
func TestStoplightAwaitingApprovalOrdering(t *testing.T) {
if aggregator.StoplightAwaitingApproval <= aggregator.StoplightYellow {
t.Error("awaiting approval should outrank yellow")
}
if aggregator.StoplightRed <= aggregator.StoplightAwaitingApproval {
t.Error("red should outrank awaiting approval")
}
if got := aggregator.StoplightAwaitingApproval.String(); got != "⏸" {
t.Errorf("String() = %q, want ⏸", got)
}
}
75 changes: 75 additions & 0 deletions internal/aws/approval_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package aws_test

import (
"testing"

"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/codepipeline"
"github.com/aws/aws-sdk-go-v2/service/codepipeline/types"
"github.com/ericdahl-dev/aws-green/internal/aggregator"
awsclient "github.com/ericdahl-dev/aws-green/internal/aws"
)

// GetPipelineState only fills in an action's token while an approval request
// is open. Anything else (a finished approval, a build action) has none.
func TestPipelineDataFromState_capturesOpenApprovalToken(t *testing.T) {
out := &codepipeline.GetPipelineStateOutput{
StageStates: []types.StageState{
{
StageName: aws.String("Source"),
LatestExecution: &types.StageExecution{Status: types.StageExecutionStatusSucceeded},
ActionStates: []types.ActionState{{
ActionName: aws.String("Checkout"),
LatestExecution: &types.ActionExecution{Status: types.ActionExecutionStatusSucceeded},
}},
},
{
StageName: aws.String("Test"),
LatestExecution: &types.StageExecution{Status: types.StageExecutionStatusInProgress},
ActionStates: []types.ActionState{{
ActionName: aws.String("Approve"),
LatestExecution: &types.ActionExecution{
Status: types.ActionExecutionStatusInProgress,
Token: aws.String("tok-123"),
},
}},
},
},
}

d := awsclient.PipelineDataFromState("my-pipeline", "us-east-1", out)

if got := d.Stages[0].Actions[0].ApprovalToken; got != "" {
t.Errorf("non-approval action token = %q, want empty", got)
}
a := d.Stages[1].Actions[0]
if a.ApprovalToken != "tok-123" {
t.Errorf("approval token = %q, want tok-123", a.ApprovalToken)
}
if a.Status != aggregator.StatusInProgress {
t.Errorf("approval status = %q, want InProgress", a.Status)
}
}

// A token on an action that is no longer in progress is a decided approval;
// acting on it would only fail.
func TestPipelineDataFromState_ignoresTokenOnFinishedAction(t *testing.T) {
out := &codepipeline.GetPipelineStateOutput{
StageStates: []types.StageState{{
StageName: aws.String("Test"),
LatestExecution: &types.StageExecution{Status: types.StageExecutionStatusSucceeded},
ActionStates: []types.ActionState{{
ActionName: aws.String("Approve"),
LatestExecution: &types.ActionExecution{
Status: types.ActionExecutionStatusSucceeded,
Token: aws.String("old-tok"),
},
}},
}},
}

d := awsclient.PipelineDataFromState("my-pipeline", "us-east-1", out)
if got := d.Stages[0].Actions[0].ApprovalToken; got != "" {
t.Errorf("finished approval token = %q, want empty", got)
}
}
16 changes: 14 additions & 2 deletions internal/aws/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ import (
type ActionData struct {
Name string
Status aggregator.ExecutionStatus
// ApprovalToken is set only while this is a manual approval waiting on a
// decision. PutApprovalResult needs it, and it goes stale once anyone
// approves, rejects, or the request times out.
ApprovalToken string
}

// StageState holds the current status of a single Pipeline stage.
Expand Down Expand Up @@ -71,9 +75,14 @@ func (c *Client) FetchPipeline(ctx context.Context, name string) (PipelineData,
return PipelineData{}, fmt.Errorf("GetPipelineState(%q): %w", name, err)
}

return PipelineDataFromState(name, c.region, out), nil
}

// PipelineDataFromState converts a GetPipelineState response into PipelineData.
func PipelineDataFromState(name, region string, out *codepipeline.GetPipelineStateOutput) PipelineData {
data := PipelineData{
Name: name,
ConsoleURL: consoleURL(c.region, name),
ConsoleURL: consoleURL(region, name),
}

for _, stage := range out.StageStates {
Expand All @@ -86,6 +95,9 @@ func (c *Client) FetchPipeline(ctx context.Context, name string) (PipelineData,
ad := ActionData{Name: aws.ToString(action.ActionName)}
if action.LatestExecution != nil {
ad.Status = mapActionStatus(action.LatestExecution.Status)
if ad.Status == aggregator.StatusInProgress {
ad.ApprovalToken = aws.ToString(action.LatestExecution.Token)
}
if action.LatestExecution.LastStatusChange != nil {
t := *action.LatestExecution.LastStatusChange
if ss.StartedAt == nil || t.Before(*ss.StartedAt) {
Expand All @@ -103,7 +115,7 @@ func (c *Client) FetchPipeline(ctx context.Context, name string) (PipelineData,
data.Stages = append(data.Stages, ss)
}

return data, nil
return data
}

func mapActionStatus(s types.ActionExecutionStatus) aggregator.ExecutionStatus {
Expand Down
40 changes: 40 additions & 0 deletions internal/fix/actioner_aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,15 @@ package fix

import (
"context"
"errors"
"fmt"
"strings"

"github.com/aws/aws-sdk-go-v2/aws"
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/cloudformation"
"github.com/aws/aws-sdk-go-v2/service/codepipeline"
cptypes "github.com/aws/aws-sdk-go-v2/service/codepipeline/types"
awsecs "github.com/aws/aws-sdk-go-v2/service/ecs"
)

Expand Down Expand Up @@ -66,3 +69,40 @@ func (a *AWSActioner) ForceDeployECS(ctx context.Context, cluster, service strin
})
return err
}

func (a *AWSActioner) PutApprovalResult(ctx context.Context, pipeline, stage, action, token string, approved bool, summary string) error {
status := cptypes.ApprovalStatusApproved
if !approved {
status = cptypes.ApprovalStatusRejected
}
_, err := a.pipeline.PutApprovalResult(ctx, &codepipeline.PutApprovalResultInput{
PipelineName: aws.String(pipeline),
StageName: aws.String(stage),
ActionName: aws.String(action),
Token: aws.String(token),
Result: &cptypes.ApprovalResult{
Status: status,
Summary: aws.String(summary),
},
})
return approvalError(err)
}

// approvalError folds AWS's two "someone already decided" errors into
// ErrApprovalAlreadyDecided so the UI can tell a stale token from a failure,
// and access denied into ErrApprovalNotPermitted so it can name the profile.
func approvalError(err error) error {
if err == nil {
return nil
}
var invalid *cptypes.InvalidApprovalTokenException
var completed *cptypes.ApprovalAlreadyCompletedException
if errors.As(err, &invalid) || errors.As(err, &completed) {
return fmt.Errorf("%w: %v", ErrApprovalAlreadyDecided, err)
}
var apiErr interface{ ErrorCode() string }
if errors.As(err, &apiErr) && strings.HasPrefix(apiErr.ErrorCode(), "AccessDenied") {
return fmt.Errorf("%w: %v", ErrApprovalNotPermitted, err)
}
return err
}
43 changes: 43 additions & 0 deletions internal/fix/actioner_aws_internal_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package fix

import (
"errors"
"testing"

"github.com/aws/aws-sdk-go-v2/service/codepipeline/types"
)

// The token is from the last poll, so someone may have decided in the console
// since. AWS reports that two ways; both mean "already decided".
func TestApprovalError_mapsStaleTokenErrors(t *testing.T) {
for _, err := range []error{
&types.InvalidApprovalTokenException{},
&types.ApprovalAlreadyCompletedException{},
} {
if got := approvalError(err); !errors.Is(got, ErrApprovalAlreadyDecided) {
t.Errorf("approvalError(%T) = %v, want ErrApprovalAlreadyDecided", err, got)
}
}
other := errors.New("AccessDenied")
if got := approvalError(other); !errors.Is(got, other) || errors.Is(got, ErrApprovalAlreadyDecided) {
t.Errorf("approvalError(other) = %v, want it passed through", got)
}
if approvalError(nil) != nil {
t.Error("approvalError(nil) should be nil")
}
}

type codeErr string

func (c codeErr) Error() string { return string(c) }
func (c codeErr) ErrorCode() string { return string(c) }

// A read-only profile can see the approval but not decide it. That is a
// configuration fact the user can act on, not a mystery AWS error.
func TestApprovalError_mapsAccessDenied(t *testing.T) {
for _, code := range []string{"AccessDeniedException", "AccessDenied"} {
if got := approvalError(codeErr(code)); !errors.Is(got, ErrApprovalNotPermitted) {
t.Errorf("approvalError(%s) = %v, want ErrApprovalNotPermitted", code, got)
}
}
}
Loading
Loading