Effect-Services

A collection of various Effectful services to communicate with 3rd party services, as well as utilities.

The modules included are used for my specific use-cases and so may be opionated, but I have attempted to make them as generic as possible nevertheless.

Installation

bun add effect-services effect
# or
npm install effect-services effect
# or
yarn add effect-services effect

Note: effect is a peer dependency and must be installed separately. Check the Peer Dependencies in package.json to see which effect version is supported.

Modules

Azure Cosmos NoSQL

Effectful wrapper for Azure Cosmos DB NoSQL operations. Provides a type-safe interface for database and container operations with streaming support.

import * as Cosmos from "effect-services/cosmos";

Features:

Effectful container SDK wrapper for CRUD operations
Custom client implementation including rate limiting handling
Query to Effect Stream support

Companies House

Integration with the UK Companies House API for business information retrieval.

import * as CompaniesHouse from "effect-services/companieshouse";

Features:

Response types provided from underlying Companies House SDK.
Rate limit handling in CompaniesHouse

FreshService

Integration with Freshservice ticketing and IT service management platform.

import * as FreshService from "effect-services/freshservice";

Features:

Authentication handling
Rate limit handling
Automatic API pagination via Effect Stream

iManage

Integration with iManage document and content management system.

import * as iManage from "effect-services/imanage";

Features:

Automatic OAuth2 token refreshing
File upload helper function

Azure Key Vault

Secure secrets management using Azure Key Vault.

import * as KeyVault from "effect-services/keyvault";

Features:

Secrets retrieval with optional caching
No bootstrap secrets required
Can be used as an Effect ConfigProvider

Legl

Integration with Legl automation platform.

import * as Legl from "effect-services/legl";

Features:

Automatic API pagination via Effect Stream
Comprehensive Legl API Schemas

Microsoft Graph

Comprehensive wrapper for Microsoft Graph API with streaming capabilities.

import * as Graph from "effect-services/msgraph";

Features:

Effectful wrapper for query construction
Support for Graph Query to Effect Stream

MSSQL

Effectful MSSQL database client with query pooling and streaming.

import * as MSSQL from "effect-services/mssql";

Features:

Effectful connection pooling
SQL query to Effect Stream support
Effectful wrapper for the mssql SDK

Utils

Utility functions and helpers for Effect-based applications.

import * as Utils from "effect-services/utils";

Features:

Task scheduling for development environments via Crons and CLI
IP Address and CIDR parsing
Dynamic bloom filters
Math functions

Azure File Share

Integration with Azure Storage File Shares.

import * as AzureFS from "effect-services/azurefs";

Features:

File and directory operations
Directory items to Effect Stream helper

Import Guide

Important! There is no single entrypoint for this repo. When importing, use namespace imports:

import * as Legl from "effect-services/legl";
import * as Graph from "effect-services/msgraph";
import * as MSSQL from "effect-services/mssql";

Not:

import Legl from "effect-services"; // ❌ This won't work

Examples

All public APIs include JsDoc examples (WIP). Refer to the relevant function documentation for detailed usage patterns.

Microsoft Graph Example

Interact with Microsoft 365 services including users, mail, and OneDrive:

import * as Graph from "effect-services/msgraph";
import { Effect, Stream } from "effect";

const ExampleListUsers = Effect.gen(function* () {
    const graph = yield* Graph.MsGraph;

    const request = yield* graph.use(
        (c) => c.api("/users")
            .select(["id", "displayName", "mail"])
            .top(10)
    );

    const stream = yield* Graph.MakeStream(request);

    yield* Stream.runForEach(stream, (user) => 
        Effect.log(`User: ${user.displayName} (${user.mail})`)
    );
});

const LayerLive = Graph.layer({
    tenantID: "your-tenant-id",
    clientID: "your-client-id",
    clientSecret: "your-client-secret",
    scopes: ["https://graph.microsoft.com/.default"]
});

ExampleListUsers.pipe(
    Effect.provide(LayerLive),
    Effect.runPromise
);

MSSQL Example

Execute queries and handle streaming results with transaction support:

import * as MSSQL from "effect-services/mssql";
import { Effect, Stream } from "effect";

const QueryExample = Effect.gen(function* () {
    const sql = yield* MSSQL.MsSqlClient;
    
    // Access the connection pool
    const pool = yield* sql.MakePool;

    const queryString = "SELECT * FROM Users WHERE Active = 1";

    // Simple query
    const result = yield* Effect.tryPromise(() => pool.query(queryString));

    yield* Effect.log("result:", result.output);

    // Streaming for large datasets
    const stream = yield* sql.MakeStream(queryString);
    yield* Stream.runForEach(stream, (r) => Effect.log("record:", r));
});

const LayerLive = MSSQL.MsSqlClient.layer({
    server: "your-server.database.windows.net",
    database: "your-database",
    user: "username",
    password: "password",
    // ...
});

QueryExample.pipe(
    Effect.provide(LayerLive),
    Effect.runPromise
);

Azure Cosmos DB Example

Work with document collections using the Cosmos SDK:

import * as Cosmos from "effect-services/cosmos";
import { Effect, Stream } from "effect";

const CosmosExample = Effect.gen(function* () {
    const cosmos = yield* Cosmos.CosmosClient;

    const container = yield* cosmos.container("Users");

    // Container operations
    const updateUser = yield* container.upsert({
        id: "123",
        status: "inactive",
    });

    yield* Effect.log("new status:", updateUser.resource?.status);

    {
        // Stream all items in container
        const stream = yield* container.allItems;
        yield* Stream.runForEach(stream, (r) => Effect.log("record ID:", r.id));
    }

    {
        // Stream items from query
        const stream = yield* container.queryToStream({
            query: "SELECT * FROM c WHERE c.status = @status",
            parameters: [{ name: "@status", value: "active" }]
        });

        yield* Stream.runForEach(stream, (u) => Effect.log("user:", u));
    }

    // Access underlying clients
    const url = yield* Effect.tryPromise(() => cosmos.client.getReadEndpoint());
    const response = yield* Effect.tryPromise(() => cosmos.database.read());
});

const LayerLive = Cosmos.CosmosClient.layer({
    connectionString: "https://your-account.documents.azure.com:443/",
    databaseID: "your-database-name"
});

CosmosExample.pipe(
    Effect.provide(LayerLive),
    Effect.runPromise
);

Azure Key Vault Example

Securely retrieve and manage secrets:

import * as KeyVault from "effect-services/keyvault";
import { Effect, Layer } from "effect";
import { DefaultAzureCredential } from "@azure/identity";

const SecretExample = Effect.gen(function* () {
    const vault = yield* KeyVault.KeyVault;

    // Retrieve a secret
    const dbPassword = yield* vault.getSecret("database-password");

    // Retrieve with cache
    const cachedVault = yield* KeyVault.KeyVaultAsCache;
    const apiKey = yield* cachedVault.lookup("api-key");

    // Use in application
    const connectionString = `Server=myserver;Password=${dbPassword};`;

    yield* Effect.log(`Connected with key version: ${apiKey.version}`);
});

const KeyVaultLayer = KeyVault.KeyVault.layer({
    vaultURL: "https://your-vault.vault.azure.us/",
    credential: new DefaultAzureCredential()
});

const CacheLayer = KeyVault.KeyVaultAsCache.layer({
    capacity: 10,
    timeToLive: "1 hour",
});

const LiveLayer = Layer.merge(KeyVaultLayer, CacheLayer);

SecretExample.pipe(
    Effect.provide(LiveLayer),
    Effect.runPromise
);

Companies House Example

Retrieve UK business information:

import * as CompaniesHouse from "effect-services/companieshouse";
import { Effect } from "effect";

const CompanyLookup = Effect.gen(function* () {
    const client = yield* CompaniesHouse.CompaniesHouse;

    const profile = yield* client.use((c) => c.companyProfile.getCompanyProfile("07424016")).pipe(
        Effect.map((r) => r.resource)
    );

    yield* Effect.log(profile?.companyName);
    yield* Effect.log(profile?.accounts.nextAccounts);
});

const LayerLive = CompaniesHouse.CompaniesHouse.layer({
    apiKey: "your-api-key"
});

CompanyLookup.pipe(
    Effect.provide(LayerLive),
    Effect.runPromise
);

Requirements

Bun, Node.js or equivelent runtimes.
For versions ^2.0.0: Effect ^3.0.0 (installed separately as peer dependency)
For versions ^3.0.0: Effect ^4.0.0 (installed separately as peer dependency)
All other module requirements are bundled as dependencies (e.g. @azure/identity)

Best Practices

Error Handling

All modules which use custom implementations of services will have Effectful, tagged errors:

import { Effect } from "effect";
import * as MsGraph from "effect-services/msgraph";

const Main = Effect.gen(function* () {
    const graph = yield* MsGraph.MsGraph;

    // Effect<GraphRequest, MsGraph.MsGraphError>
    const query = graph.use((c) => c.api("/invalid/path"));
});

Streaming Large Datasets

Use streaming for memory-efficient processing of large result sets:

import * as Graph from "effect-services/msgraph";
import { Stream, Effect } from "effect";

// Good: Memory-efficient streaming
const StreamUsers = Effect.gen(function* () {
    const graph = yield* Graph.MsGraph;
    const request = yield* graph.use((c) => c.api("/users"));
    const stream = yield* Graph.MakeStream(request);
    
    yield* Stream.runForEach(stream, (user) => Effect.log(user));
});

Development

Setup

Bun is required for development:

# Clone and install dependencies
git clone https://github.com/Shiv-SB/effect-services
cd effect-services
bun i

Building

# Build TypeScript and generate type definitions
bun run build

# This will:
# - Transpile .ts to .js
# - Generate .d.ts type files
# - Validate exports in package.json

Workspace Structure

src/                 # Source TypeScript files
├── cosmos/          # Azure Cosmos DB module
├── msgraph/         # Microsoft Graph module
├── mssql/           # MSSQL database module
├── keyvault/        # Azure Key Vault module
└── ...              # Other service modules

build/               # Compiled JavaScript and type definitions
lib/                 # Build utilities and helpers
tests/               # Unit and integration tests

Development Tips

Try to keep each module independent. Any shared functions should be stored in internals.
The package exports are validated on build to ensure correctness

Common Patterns

Dependency Injection

All modules use Effect's dependency injection for configuration:

import * as MSSQL from "effect-services/mssql";
import { Effect } from "effect";

const MyApp = Effect.gen(function* () {
    const client = yield* MSSQL.MsSqlClient;
    // Use client
});

const layer = MSSQL.MsSqlClient.layer({ /* config */ });

MyApp.pipe(Effect.provide(layer), Effect.runPromise);

Combining Multiple Services

This example showcase using three different services concurrently, mssql, msgraoh and keyvault. msgraph and mssql are used to generate Streams, and keyvault is used as the ConfigProvider.

import { DefaultAzureCredential } from "@azure/identity";
import { Config, ConfigProvider, Effect, Layer, Stream } from "effect";
import { MakeKeyVaultProvider } from "effect-services/keyvault";
import * as MsGraph from "effect-services/msgraph";
import * as SQL from "effect-services/mssql";

const GetGraphUsers = Effect.gen(function* () {
    const graph = yield* MsGraph.MsGraph;

    // .use method to access the api builder
    const query = yield* graph.use(
        (c) => c.api("/users").select(["id", "displayName", "officeLocation"])
    );

    // Pass the query to the Stream factory
    const stream = yield* MsGraph.MakeStream(query);
    return stream;
});

const GetSqlUsers = Effect.gen(function* () {
    const sql = yield* SQL.MsSqlClient;

    const stream = yield* sql.MakeStream("SELECT * FROM Users");
    return stream;
});

const ExampleApp = Effect.all([GetGraphUsers, GetSqlUsers]).pipe(
    Effect.map(([s1, s2]) => Stream.merge(s1, s2)),
    Effect.tap(Effect.log("Starting...")),
    Effect.andThen(Stream.runForEach((user) => Effect.log("user:", user))),
    Effect.tap(Effect.log("Finished!")),
)

// Create a ConfigProvider backed by Azure Key Vault
// No secrets needed! Authentication happens via the credential,
// so your codebase can be completely free of secrets and keys.
const SecretsProvider = MakeKeyVaultProvider({
    vaultURL: "https://my-vault.vault.azure.us/",
    credential: new DefaultAzureCredential()
});

// Provide a fallback to process.env
const MergedProvider = SercretsProvider.pipe(
    ConfigProvider.orElse(ConfigProvider.fromEnv)
);

// Convert the provider into a layer.
const providerLayer = ConfigProvider.layer(MergedProvider);

// A simple provider function which constructs and then provides our layers.
// We use an Effectful function for this because we need to yield* Config values.
const ProvideLayers = <A, E, R>(
    runnable: Effect.Effect<A, E, R>
) => Effect.gen(function* () {
    // Construct each layer...

    const graphLayer = MsGraph.MsGraph.layer({
        clientID: yield* Config.string("example-secret-clientID"),
        clientSecret: yield* Config.string("example-secret-clientSecret"),
        tenantID: yield* Config.string("example-secret-tenantID"),
        scopes: ["https://graph.microsoft.com/.default"]
    });

    const sqlLayer = SQL.MsSqlClient.layer({
        server: yield* Config.string("example-secret-server"),
        password: yield* Config.string("example-secret-password"),
        port: yield* Config.port("server-port").pipe(Config.withDefault(1433))
    });

    // Combine all your layers!
    const allLayers = Layer.mergeAll(graphLayer, sqlLayer);

    return yield* runnable.pipe(
        Effect.provide(allLayers)
    );
}).pipe(
    Effect.provide(providerLayer)
);

ExampleApp.pipe(
    ProvideLayers,
    Effect.runPromise,
);

Contributing

Contributions are welcome! Please:

Fork the repository and create a feature branch
Follow the existing patterns - Each module should follow the same structure and conventions
Write tests for new functionality
Build and validate with bun run build & bun audit before submitting
Keep Effect v4 compatibility unless there's a major version bump

Module Development Guidelines

Use Effect for all async operations.
Do not create any side-effects.
Provide JSDoc comments with usage examples
Include type definitions (even when verbose to do so)
Create corresponding tests
Update the README with module description

License

See LICENSE file for details.

Support

For issues, questions, or feature requests, please open an issue on GitHub at Shiv-SB/effect-services

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
.vscode		.vscode
lib		lib
src		src
tests		tests
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
bun.lock		bun.lock
bunfig.toml		bunfig.toml
package.json		package.json
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation

Effect-Services

Installation

Modules

Azure Cosmos NoSQL

Companies House

FreshService

iManage

Azure Key Vault

Legl

Microsoft Graph

MSSQL

Utils

Azure File Share

Import Guide

Examples

Microsoft Graph Example

MSSQL Example

Azure Cosmos DB Example

Azure Key Vault Example

Companies House Example

Requirements

Best Practices

Error Handling

Streaming Large Datasets

Development

Setup

Building

Workspace Structure

Development Tips

Common Patterns

Dependency Injection

Combining Multiple Services

Contributing

Module Development Guidelines

License

Support

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages