Skip to content

development.md

Latest commit

 

History

History
702 lines (543 loc) · 14.5 KB

development.md

File metadata and controls

702 lines (543 loc) · 14.5 KB
Raw

Development Guide

Guide for building, testing, and contributing to Registry.

Prerequisites

  • Go version declared in go.mod
  • Node.js 18 or later
  • Python 3.10 or later
  • Git
  • Make (optional, for convenience)

Getting Started

Clone Repository

git clone https://github.com/burgrp/reg.git
cd reg

Build

# Build binary
go build -o reg

# Build with version information
VERSION=1.0.0
go build -ldflags="-X 'github.com/burgrp/reg/cmd.Version=$VERSION'" -o reg

# Verify build
./reg version

Run Tests

# Run all tests
go test ./...

# Run tests with verbose output
go test -v ./...

# Run tests without cache
go test -count=1 ./...

# Run specific test
go test -v ./pkg/client/rest -run TestClient_Consume

# Run with race detector
go test -race ./...

# Run with coverage
go test -cover ./...
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

Run the client library suites:

npm --prefix clients/js test
npm --prefix clients/node-red test

python -m pip install -e './clients/python[dev]'
python -m unittest discover -s clients/python/test -v
ruff check clients/python
ruff format --check clients/python
mypy clients/python/src

Run Server Locally

# Start on default port (8080)
./reg serve

# Start on custom port
./reg serve --rest :9000

# With custom logging
LOG_LEVEL=debug ./reg serve

Úroveň logování

Od v1.0.17 řídí LOG_LEVEL diagnostické logování všech příkazů, včetně serveru registry, jeho REST rozhraní, metrik a MCP serveru. Logy se zapisují na standardní chybový výstup (stderr). Výsledky příkazů jako get, list nebo version ani JSON-RPC komunikace MCP na stdout se nefiltrují. Chybová hlášení CLI zůstávají viditelná bez ohledu na úroveň logování.

Výchozí úroveň při nenastavené nebo prázdné proměnné je INFO. Přijímají se debug, info, warn a error bez ohledu na velikost písmen, také s číselným posunem podle Go log/slog, například INFO+2. Neplatná hodnota ukončí příkaz chybou ještě před jeho spuštěním; nápověda zůstává dostupná.

LOG_LEVEL=info ./reg serve --metrics :9090
LOG_LEVEL=debug ./reg serve
LOG_LEVEL=error REGISTRY=http://localhost:8080 ./reg list

Nastavení popisuje hlavní nápověda (reg, reg --help) i reg serve --help. Samostatný přepínač pro úroveň logování není zavedený. Starší verze LOG_LEVEL nečetly a serve měl pevně nastavenou úroveň DEBUG.

Metriky Prometheus

Export metrik se zapíná samostatným HTTP rozhraním:

./reg serve --rest :8080 --metrics :9090

Exporter poskytuje na / i /metrics číselné registry jako metriky typu gauge; booleovské hodnoty převádí na 0 a 1. Ostatní typy neexportuje. Nemění názvy ani hodnoty registrů v REST rozhraní.

V názvu metriky nahrazuje tečky dvojtečkami. Platné názvy zachovává, před počáteční číslici přidává _ a znaky mimo ASCII množinu [a-zA-Z0-9_:] nahrazuje _. Prázdný název převádí na _. Například registr nepojmenovaného Zigbee zařízení 0x00158d000483b274.temperature exportuje jako _0x00158d000483b274:temperature. Takové zařízení proto nezneplatní celý scrape kvůli názvu začínajícímu číslicí.

Štítky n1, n2 atd. obsahují původní části názvu před touto normalizací. Registry jako room-1.temperature a room_1.temperature tak mají stejný název metriky, ale odlišné štítky a samostatné hodnoty. Deklarace TYPE se pro společný název zapisuje jen jednou. Existující platné názvy metrik a jejich štítky zůstávají beze změny.

Regresní testy exportu:

go test -race ./internal/metrics

Project Structure

reg/
├── cmd/                    # CLI commands
│   ├── root.go            # Root command
│   ├── serve.go           # Server command
│   ├── provide.go         # Provider CLI command
│   ├── get.go             # Consumer read command
│   ├── list.go            # List registers command
│   ├── browse.go          # Interactive TUI
│   └── version.go         # Version command
├── internal/              # Private application code
│   ├── registry/          # Core business logic
│   │   ├── core.go        # Registry implementation
│   │   └── listener.go    # Notification system
│   └── rest/              # REST protocol layer
│       ├── server.go      # HTTP server
│       ├── consumer.go    # Consumer endpoints
│       ├── provider.go    # Provider endpoints
│       └── duration.go    # Duration JSON handling
├── pkg/                   # Public library code
│   ├── client/            # Client interface
│   │   ├── client.go      # Protocol-agnostic interface
│   │   ├── factory/       # Client factory
│   │   └── rest/          # REST client implementation
│   │       ├── client.go  # Client struct
│   │       ├── consumer.go # Consumer implementation
│   │       └── provider.go # Provider implementation
│   ├── wire/              # Wire protocol clients
│   │   └── rest/          # Low-level REST clients
│   └── browser/           # TUI browser
├── docs/                  # Documentation
├── clients/               # JavaScript, Node-RED, and Python clients
├── main.go                # Entry point
├── go.mod                 # Go modules
└── build.sh               # Release build script

Development Workflow

Making Changes

  1. Create a feature branch:
git checkout -b feature/my-feature

  1. Make your changes

  2. Run tests:

go test ./...
  1. Format code:
go fmt ./...
  1. Run linters:
go vet ./...
golangci-lint run  # If installed
  1. Commit changes:
git add .
git commit -m "Add feature: description"

Testing Changes

Manual Testing

Terminal 1 (Server):

./reg serve

Terminal 2 (Provider):

export REGISTRY=http://localhost:8080
./reg provide temperature 22.5 '{"unit":"celsius"}' --ttl 10s

Terminal 3 (Consumer):

export REGISTRY=http://localhost:8080
./reg get temperature

Integration Testing

# Start server in background
./reg serve &
SERVER_PID=$!

# Run tests against server
export REGISTRY=http://localhost:8080

# Test provider
./reg provide test 123 &
PROVIDER_PID=$!

sleep 1

# Test consumer
RESULT=$(./reg get test)
echo "$RESULT"

# Cleanup
kill $PROVIDER_PID $SERVER_PID

Writing Tests

Unit Tests

package rest

import (
    "testing"
)

func TestMyFunction(t *testing.T) {
    // Arrange
    input := "test"

    // Act
    result := myFunction(input)

    // Assert
    if result != expected {
        t.Errorf("got %v, want %v", result, expected)
    }
}

Integration Tests

func TestServerIntegration(t *testing.T) {
    // Create test server
    server := httptest.NewServer(handler)
    defer server.Close()

    // Test client
    client := rest.NewClient(server.URL)

    // Run test
    ctx := context.Background()
    values, _, err := client.Consume(ctx, "test")
    if err != nil {
        t.Fatal(err)
    }

    // Verify
    select {
    case v := <-values:
        if v.Value != expected {
            t.Errorf("got %v, want %v", v.Value, expected)
        }
    case <-time.After(1 * time.Second):
        t.Fatal("timeout")
    }
}

Table-Driven Tests

func TestDuration_UnmarshalJSON(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    time.Duration
        wantErr bool
    }{
        {"seconds", `"5s"`, 5 * time.Second, false},
        {"minutes", `"10m"`, 10 * time.Minute, false},
        {"invalid", `"invalid"`, 0, true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            var d Duration
            err := d.UnmarshalJSON([]byte(tt.input))

            if (err != nil) != tt.wantErr {
                t.Errorf("UnmarshalJSON() error = %v, wantErr %v", err, tt.wantErr)
                return
            }

            if time.Duration(d) != tt.want {
                t.Errorf("UnmarshalJSON() = %v, want %v", d, tt.want)
            }
        })
    }
}

Debugging

Enable Debug Logging

import "log/slog"

// In main.go or test
logger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
    Level: slog.LevelDebug,
}))
slog.SetDefault(logger)

Debugging with Delve

# Install delve
go install github.com/go-delve/delve/cmd/dlv@latest

# Debug test
dlv test ./pkg/client/rest -- -test.run TestClient_Consume

# Debug application
dlv debug . -- serve --rest :8080

# Common commands
(dlv) break main.main
(dlv) continue
(dlv) print variable
(dlv) next
(dlv) step
(dlv) quit

Tracing HTTP Requests

client := rest.NewClientWithHTTPClient("http://localhost:8080", &http.Client{
    Transport: &http.Transport{
        // Enable tracing
        DisableKeepAlives: true,
    },
})

Or use environment variable:

export GODEBUG=http2debug=1
./reg serve

Performance Testing

Benchmarking

func BenchmarkRegistry_SetRegister(b *testing.B) {
    r := registry.New()
    ctx := context.Background()

    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        r.SetRegister(ctx, "test", i, nil, 10*time.Second)
    }
}

Run benchmarks:

# Run all benchmarks
go test -bench=. ./...

# Run specific benchmark
go test -bench=BenchmarkRegistry_SetRegister ./internal/registry

# With memory profiling
go test -bench=. -benchmem ./...

# Generate CPU profile
go test -bench=. -cpuprofile=cpu.prof ./...
go tool pprof cpu.prof

Load Testing

# Install hey
go install github.com/rakyll/hey@latest

# Load test provider endpoint
hey -n 10000 -c 100 -m PUT \
  -H "Content-Type: application/json" \
  -d '{"registers":{"temp":{"value":22.5,"ttl":"10s"}}}' \
  http://localhost:8080/provider

# Load test consumer endpoint
hey -n 10000 -c 100 \
  http://localhost:8080/consumer?name=temp

Continuous Integration

GitHub Actions

Tests run automatically on push via .github/workflows/test.yml:

name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v4
        with:
          go-version: '1.21'
      - run: go test ./...

Pre-commit Hooks

Create .git/hooks/pre-commit:

#!/bin/bash

echo "Running tests..."
go test ./...
if [ $? -ne 0 ]; then
    echo "Tests failed"
    exit 1
fi

echo "Running go fmt..."
go fmt ./...

echo "Running go vet..."
go vet ./...
if [ $? -ne 0 ]; then
    echo "go vet failed"
    exit 1
fi

exit 0

Make it executable:

chmod +x .git/hooks/pre-commit

Release Process

Version Tagging

# Tag release
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

Building Release Binaries

Build script creates binaries for multiple platforms:

# Build for all platforms
GITHUB_REF=refs/tags/v1.0.0 ./build.sh

# Binaries created in bin/
ls bin/
# reg-linux-amd64
# reg-linux-arm64
# reg-darwin-amd64
# reg-darwin-arm64
# reg-windows-amd64.exe

Manual Release Build

# Linux AMD64
GOOS=linux GOARCH=amd64 go build \
  -ldflags="-X 'github.com/burgrp/reg/cmd.Version=1.0.0'" \
  -o reg-linux-amd64

# macOS ARM64
GOOS=darwin GOARCH=arm64 go build \
  -ldflags="-X 'github.com/burgrp/reg/cmd.Version=1.0.0'" \
  -o reg-darwin-arm64

# Windows AMD64
GOOS=windows GOARCH=amd64 go build \
  -ldflags="-X 'github.com/burgrp/reg/cmd.Version=1.0.0'" \
  -o reg-windows-amd64.exe

Common Tasks

Add New CLI Command

  1. Create command file in cmd/:
// cmd/mycommand.go
package cmd

import "github.com/spf13/cobra"

func newMyCommandCmd() *cobra.Command {
    return &cobra.Command{
        Use:   "mycommand",
        Short: "Description",
        RunE: func(cmd *cobra.Command, args []string) error {
            // Implementation
            return nil
        },
    }
}
  1. Register in root command:
// cmd/root.go
func newRootCmd() *cobra.Command {
    root := &cobra.Command{...}
    root.AddCommand(
        newServeCmd(),
        newMyCommandCmd(), // Add here
    )
    return root
}

Add New REST Endpoint

  1. Add handler to internal/rest/:
// internal/rest/myendpoint.go
func handleMyEndpoint(reg *registry.Registry) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // Implementation
    }
}
  1. Register in server:
// internal/rest/server.go
func RunServer(ctx context.Context, addr string, reg *registry.Registry) error {
    mux := http.NewServeMux()
    mux.HandleFunc("/myendpoint", handleMyEndpoint(reg))
    // ...
}

Add Client Method

  1. Add to interface:
// pkg/client/client.go
type Client interface {
    Consume(...) (...)
    Provide(...) (...)
    MyNewMethod(...) (...) // Add here
}
  1. Implement in REST client:
// pkg/client/rest/myclient.go
func (c *Client) MyNewMethod(...) (...) {
    // Implementation
}

Troubleshooting

Tests Failing

# Clear test cache
go clean -testcache

# Run with verbose output
go test -v ./...

# Run specific failing test
go test -v ./pkg -run TestName

Import Cycle

Reorganize code:

  • Move shared types to separate package
  • Use interfaces to break dependencies
  • Ensure internal/ doesn't import pkg/

Race Condition

# Run with race detector
go test -race ./...

# Fix by:
# - Adding mutexes
# - Using channels
# - Avoiding shared state

Code Style

Formatting

# Format all code
go fmt ./...

# Check formatting
gofmt -l .

Naming Conventions

  • Packages: lowercase, single word
  • Exported: PascalCase
  • Unexported: camelCase
  • Interfaces: noun or -er suffix
  • Constants: PascalCase or SCREAMING_SNAKE_CASE

Comments

// Package rest implements the REST protocol layer.
package rest

// Client implements the client.Client interface using REST.
type Client struct {
    // consumerClient handles consumer operations.
    consumerClient *wire.ConsumerClient
}

// NewClient creates a new REST client with the given base URL.
func NewClient(baseURL string) *Client {
    // Implementation
}

Resources