diff --git a/README.md b/README.md index 622553c..639f00c 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,8 @@ Create a `.env` file in the root directory or export the variables in your shell | `SPRING_DATA_REDIS_PORT` | ❌ | `6379` | Redis port | | `HIKARI_MAX_POOL` | ❌ | `10` | Max JDBC connection pool size | | `SMS_PRICE_PER_SEGMENT` | ❌ | `20.0` | Price per SMS segment (in TZS) | +| `FLEXTUMA_SMS_BEEM_DELIVERY_POLL_INTERVAL_MS` | ❌ | `60000` | Beem delivery-report polling interval in milliseconds | +| `FLEXTUMA_SMS_BEEM_DELIVERY_MINIMUM_DELAY_MINUTES` | ❌ | `5` | Minimum delay before the first Beem delivery lookup | ### 3. Build the application @@ -329,12 +331,14 @@ POST /api/webhooks/{provider} Content-Type: application/json { - "messageId": "provider-message-id", + "request_id": "provider-request-id", "status": "DELIVERED|FAILED|PENDING", "timestamp": "2024-01-15T10:30:00Z" } ``` +For Beem, delivery reports are normally retrieved by the scheduled polling worker rather than a callback. The worker starts at least five minutes after submission, uses the Beem `request_id` stored as `providerMessageId`, and polls every 60 seconds by default. Configure the interval with `flextuma.sms.beem.delivery-poll-interval-ms`. + **Supported Providers:** - `beem` - Beem SMS provider - `next` - NextSMS provider @@ -717,7 +721,7 @@ Two concrete `SmsSender` implementations: | Provider | Class | Auth Method | Status | |---|---|---|---| -| **Beem** | `BeemSender` | API key + secret (Basic Auth header) | ✅ Production ready | +| **Beem** | `BeemSender` + `BeemDeliveryReportWorker` | API key + secret (HTTP Basic); delivery-report polling | ✅ Production ready | | **NextSMS** | `NextSmsSender` | API key + secret (Basic Auth header) | ✅ Production ready | Adding a new provider: implement `SmsSender`, annotate with `@Service`, and set the matching `provider` string on the `SmsConnector` record. @@ -771,7 +775,7 @@ Request with memberId CSRF protection uses `CookieCsrfTokenRepository` (token sent as `XSRF-TOKEN` cookie, readable by SPA). Exemptions: - `/api/login` — no session exists yet at this point -- `/api/webhooks/**` — reserved for PAT-authenticated provider callbacks +- `POST /api/webhooks/{provider}` — public provider callback endpoint; no custom secret header is required ### Tenant-Aware Resource Filtering diff --git a/build.gradle b/build.gradle index ec8667a..cef93bd 100644 --- a/build.gradle +++ b/build.gradle @@ -8,7 +8,7 @@ plugins { } group = 'com.flexcodelabs' -version = '0.0.37' +version = '0.0.38' description = 'Flextuma App' java { diff --git a/docs/deployment.md b/docs/deployment.md index 7caa789..d8baabc 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -27,7 +27,8 @@ Provide secrets through the platform secret manager, never in the image, reposit | `HIKARI_MAX_POOL`, `HIKARI_MIN_IDLE` | Recommended | Size across all replicas below PostgreSQL’s connection limit. | | `SESSION_TIMEOUT` | Recommended | Session lifetime, e.g. `30m`. | | `SMS_PRICE_PER_SEGMENT` | Yes | Decimal cost used for wallet accounting; confirm the business unit and currency. | -| `FLEXTUMA_WEBHOOKS_SMS_SHARED_SECRET` | Yes, for DLRs | Shared callback secret supplied by providers in `X-Flextuma-Webhook-Secret`; use a high-entropy secret from the secret manager. | +| `FLEXTUMA_SMS_BEEM_DELIVERY_POLL_INTERVAL_MS` | Optional | Beem delivery-report polling interval in milliseconds; defaults to `60000`. Beem polling starts five minutes after send. | +| `FLEXTUMA_SMS_BEEM_DELIVERY_MINIMUM_DELAY_MINUTES` | Optional | Minimum wait before the first Beem delivery lookup; defaults to `5`, as recommended by Beem. | | `APP_FRONTEND_DIRECTORY` | If serving UI | Read-only directory containing `index.html` and assets. | | `APP_UPLOAD_DIRECTORY` | If app uploads are enabled | Durable, access-controlled storage; `/tmp` loses uploads on restart. | | `LOG_MIN_LEVEL`, `LOG_RETENTION_DAYS` | Recommended | Tune for operating requirements; database log retention has storage impact. | diff --git a/docs/frontend-design-spec.md b/docs/frontend-design-spec.md index 494a3b6..76184dd 100644 --- a/docs/frontend-design-spec.md +++ b/docs/frontend-design-spec.md @@ -430,8 +430,8 @@ Both follow the same pattern: | URL | Text | Provider API base URL | | API Key | Password input | Write-only; shows masked on read | | Secret | Password input | Write-only; shows masked on read | -| Sender ID | Text | Optional — display name on SMS | -| Extra Settings | JSON editor or key-value pairs | Optional | +| Sender ID | Text | Required for BEEM and NEXT; must be an active provider sender ID | +| Extra Settings | JSON editor or key-value pairs | For BEEM, optional `encoding` and `schedule_time` fields, e.g. `{"encoding":"0","schedule_time":""}` | | Default | Toggle | One connector should be default | > **Important:** `key` and `secret` are write-only. On GET, the API returns masked values (e.g. `****abcd`). The form should leave these fields blank on edit and only send them if the user explicitly types new values. @@ -986,9 +986,10 @@ When creating or updating entities via `POST` / `PUT` requests, submit a JSON bo "key": "api-key-here", "secret": "api-secret-here", "isDefault": true, - "extraSettings": "{\"timeout\": 5000}" + "extraSettings": "{\"encoding\":\"0\",\"schedule_time\":\"\"}" } ``` +* **BEEM:** Use `https://apisms.beem.africa/v1/send`; `key`, `secret`, and an active `senderId` are required. `extraSettings.schedule_time` is optional GMT+0 in `yyyy-mm-dd hh:mm` format. * **key, secret:** Write-only fields. Appears masked (`****abcd`) in reads. Do not send on `PUT` unless changing. ### 13.9 ConnectorConfig diff --git a/docs/third-party-integration.md b/docs/third-party-integration.md index bdfe581..2a55583 100644 --- a/docs/third-party-integration.md +++ b/docs/third-party-integration.md @@ -6,7 +6,7 @@ Flextuma currently integrates with: | Party | Direction | Mechanism | | --- | --- | --- | -| BEEM SMS | Outbound SMS; intended inbound delivery reports | JSON over HTTPS using provider credentials. | +| BEEM SMS | Outbound SMS and delivery-report polling | JSON over HTTPS using HTTP Basic credentials. | | NextSMS | Outbound SMS; intended inbound delivery reports | JSON over HTTPS using Basic authentication. | | Tenant/customer data API | Outbound recipient lookup and member hydration | Configurable GET requests plus JSONPath field mapping. | | Client/automation | Inbound API requests | Session cookie or personal access token (PAT) in `X-API-KEY`. | @@ -23,7 +23,7 @@ X-API-KEY: ft_ PATs act as the owning user and inherit that user’s privileges. Use one token per integration, give it an expiry, rotate it, and disable/delete it when no longer needed. Do not put PATs in browser code, query strings, logs, or support tickets. -All API paths other than login, registration, and frontend assets require authentication under the active security configuration. In particular, there is no separate, functioning API-key scheme for anonymous webhooks despite the `flextuma.auth.api-key-endpoints` configuration property. +All API paths other than login, registration, frontend assets, and the one-segment DLR callback path (`POST /api/webhooks/{provider}`) require authentication under the active security configuration. There is no separate, functioning API-key scheme for anonymous webhooks despite the `flextuma.auth.api-key-endpoints` configuration property. ## SMS provider setup @@ -31,7 +31,19 @@ Create an SMS connector using `POST /api/connectors` with the provider string, p ### BEEM -The BEEM adapter sends a JSON request with `source_addr`, `message`, `schedule_time`, `encoding`, and a one-item `recipients` list. It supplies credentials both as `api_key`/`secret_key` headers and HTTP Basic authentication. Configure the provider endpoint and sender ID supplied by BEEM. Confirm the expected response contains an actual provider message ID before enabling delivery reports. +Configure the connector with `provider: "BEEM"`, URL `https://apisms.beem.africa/v1/send`, the BEEM API key in `key`, the BEEM secret key in `secret`, and an active BEEM sender ID in `senderId`. The adapter sends HTTP Basic authentication (`key:secret`) and this JSON body: + +```json +{ + "source_addr": "ACME", + "schedule_time": "", + "encoding": "0", + "message": "Hello world", + "recipients": [{ "recipient_id": "1", "dest_addr": "255700000001" }] +} +``` + +Set optional Beem request fields through `extraSettings`, for example `{"encoding":"0","schedule_time":"2026-08-15 10:30"}`. `schedule_time` is GMT+0 in `yyyy-mm-dd hh:mm` format. Beem returns `request_id`; Flextuma stores it as `providerMessageId` for delivery tracking. ### NextSMS @@ -39,6 +51,19 @@ The NextSMS adapter sends `{ "from", "to", "text" }` as JSON and uses HTTP Basic ### Delivery reports (DLRs) +#### BEEM polling (primary) + +Beem’s documented delivery mechanism is polling, not callback registration. Starting five minutes after a successful send, Flextuma polls the following endpoint for BEEM messages in `SENT` status: + +```http +GET https://dlrapi.beem.africa/public/v1/delivery-reports?dest_addr={recipient}&request_id={providerMessageId} +Authorization: Basic base64(key:secret) +``` + +The polling interval defaults to 60 seconds and is configurable with `flextuma.sms.beem.delivery-poll-interval-ms`. A Beem `DELIVERED` status becomes `DELIVERED`, `UNDELIVERED` becomes `FAILED`, and `PENDING` remains `SENT` until a terminal status is returned. + +#### Provider callbacks (optional) + The implemented route is: ```http @@ -47,9 +72,9 @@ POST https:///api/webhooks/NEXT Content-Type: application/json ``` -BEEM parser expects `messageID` and `status`; Next parser expects `message_id` (or `messageId`) and `status`. The documented source comment incorrectly describes `/api/webhooks/sms/{provider}/dlr`; that route does not exist. +The callback path is public and accepts no custom authentication header. Beem callbacks are correlated with `request_id` (with legacy `messageID` accepted) plus `status`; Next callbacks use `message_id` (or `messageId`) plus `status`. The callback route is optional for Beem because polling is the supported integration. -DLR routes are public only for the one-segment callback path and require the `X-Flextuma-Webhook-Secret` header. Set `FLEXTUMA_WEBHOOKS_SMS_SHARED_SECRET` from the deployment secret manager and configure the same high-entropy value at the provider. Requests with a missing/incorrect secret are rejected. The callback now correlates using `providerMessageId`; verify the provider’s response/callback message-ID shape in staging before enabling it. +DLR routes are public only for the one-segment callback path. The callback correlates using `providerMessageId`; validate the provider callback payload in staging before relying on it. ## Tenant/customer data API @@ -77,8 +102,8 @@ These are code-observed findings as of this repository revision, ordered by impa | Priority | Finding | Impact and recommended action | | --- | --- | --- | -| Resolved | DLR endpoint authentication and lookup were incompatible with provider callbacks. | The callback path is now narrowly public, requires a constant-time compared shared secret, and correlates with `providerMessageId`. Upgrade next to provider-specific HMAC signatures, timestamp/replay controls, and IP allowlists where supported. | -| Resolved | BEEM did not retain its response message ID. | The adapter now reads `message_id` and saves it as `providerMessageId`; validate the exact live provider schema during staging. | +| Resolved | BEEM delivery tracking used a callback-only shape that did not match the documented API. | The adapter stores Beem `request_id` and polls the documented delivery endpoint after five minutes. | +| Resolved | Generic callback IDs did not match Beem delivery identifiers. | The public callback parser accepts Beem `request_id` (and legacy `messageID`) and correlates through `providerMessageId`. | | Resolved | Generic single-record read, update, and delete skipped the tenant specification. | These operations now use the tenant-scoped specification. Maintain cross-tenant authorization tests as new endpoints are added. | | Resolved | Raw dispatch used `content` while the queue required `message`. | The trigger now maps its request content to the required queue field. | | Resolved | PAT authentication ignored a token’s `active` flag. | Inactive tokens are now rejected. | @@ -94,4 +119,4 @@ These are code-observed findings as of this repository revision, ordered by impa ## Minimum acceptance tests -Before enabling any external party in production, automate these tests: valid and invalid PAT authentication; provider credential rejection; one successful send and one provider failure; idempotent retry behavior; a signed DLR that changes the correct SMS log; malformed/duplicate/out-of-order DLRs; tenant API timeout/5xx/oversize response; recipient pagination; wallet debit/refund reconciliation; and authorization isolation between organisations. +Before enabling any external party in production, automate these tests: valid and invalid PAT authentication; provider credential rejection; one successful send and one provider failure; Beem `request_id` persistence; Beem delivery polling for `PENDING`, `DELIVERED`, and `UNDELIVERED`; malformed/duplicate/out-of-order callbacks; tenant API timeout/5xx/oversize response; recipient pagination; wallet debit/refund reconciliation; and authorization isolation between organisations. diff --git a/src/main/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnector.java b/src/main/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnector.java index 3d73555..32b0816 100644 --- a/src/main/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnector.java +++ b/src/main/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnector.java @@ -59,4 +59,4 @@ public String getMaskedKey() { public String getMaskedSecret() { return MaskingUtil.mask(this.secret); } -} \ No newline at end of file +} diff --git a/src/main/java/com/flexcodelabs/flextuma/core/repositories/SmsLogRepository.java b/src/main/java/com/flexcodelabs/flextuma/core/repositories/SmsLogRepository.java index fe16f1e..da60395 100644 --- a/src/main/java/com/flexcodelabs/flextuma/core/repositories/SmsLogRepository.java +++ b/src/main/java/com/flexcodelabs/flextuma/core/repositories/SmsLogRepository.java @@ -22,6 +22,8 @@ public interface SmsLogRepository extends BaseRepository, List findTop50ByStatusOrderByCreatedAsc(SmsLogStatus status); + List findTop50ByStatusAndProviderMessageIdIsNotNullOrderByCreatedAsc(SmsLogStatus status); + @org.springframework.data.jpa.repository.Query("SELECT s FROM SmsLog s WHERE s.status = :status AND (s.scheduledAt IS NULL OR s.scheduledAt <= :now) ORDER BY s.created ASC") List findDueMessages( @org.springframework.data.repository.query.Param("status") SmsLogStatus status, diff --git a/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemDeliveryReportClient.java b/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemDeliveryReportClient.java new file mode 100644 index 0000000..f45c5c2 --- /dev/null +++ b/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemDeliveryReportClient.java @@ -0,0 +1,66 @@ +package com.flexcodelabs.flextuma.core.senders; + +import java.nio.charset.StandardCharsets; +import java.util.Base64; +import java.util.List; + +import org.springframework.core.ParameterizedTypeReference; +import org.springframework.http.HttpEntity; +import org.springframework.http.HttpHeaders; +import org.springframework.http.HttpMethod; +import org.springframework.http.MediaType; +import org.springframework.http.ResponseEntity; +import org.springframework.stereotype.Service; +import org.springframework.web.client.RestTemplate; +import org.springframework.web.util.UriComponentsBuilder; + +import com.fasterxml.jackson.annotation.JsonProperty; +import com.flexcodelabs.flextuma.core.entities.sms.SmsConnector; + +import lombok.Data; + +/** Client for Beem's documented delivery-report lookup API. */ +@Service +public class BeemDeliveryReportClient { + + static final String DELIVERY_REPORTS_URL = "https://dlrapi.beem.africa/public/v1/delivery-reports"; + + private final RestTemplate restTemplate; + + public BeemDeliveryReportClient(RestTemplate restTemplate) { + this.restTemplate = restTemplate; + } + + public BeemDeliveryStatus lookup(SmsConnector connector, String recipient, String requestId) { + String url = UriComponentsBuilder.fromUriString(DELIVERY_REPORTS_URL) + .queryParam("dest_addr", recipient) + .queryParam("request_id", requestId) + .toUriString(); + + ResponseEntity> response = restTemplate.exchange(url, HttpMethod.GET, + new HttpEntity(headers(connector)), new ParameterizedTypeReference<>() { + }); + List reports = response.getBody(); + return reports == null || reports.isEmpty() ? null : reports.get(0); + } + + private HttpHeaders headers(SmsConnector connector) { + HttpHeaders headers = new HttpHeaders(); + headers.setContentType(MediaType.APPLICATION_JSON); + String credentials = connector.getKey() + ":" + connector.getSecret(); + headers.set(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder() + .encodeToString(credentials.getBytes(StandardCharsets.UTF_8))); + return headers; + } + + @Data + public static class BeemDeliveryStatus { + @JsonProperty("dest_addr") + private String destinationAddress; + + @JsonProperty("request_id") + private String requestId; + + private String status; + } +} diff --git a/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemSender.java b/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemSender.java index fbbfde7..a29afd5 100644 --- a/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemSender.java +++ b/src/main/java/com/flexcodelabs/flextuma/core/senders/BeemSender.java @@ -65,9 +65,6 @@ private ResponseEntity makeApiCall(SmsConnector config, String private HttpHeaders createHeaders(SmsConnector config) { HttpHeaders headers = new HttpHeaders(); headers.setContentType(MediaType.APPLICATION_JSON); - headers.set("api_key", config.getKey()); - headers.set("secret_key", config.getSecret()); - String auth = config.getKey() + ":" + config.getSecret(); String encodedAuth = Base64.getEncoder().encodeToString(auth.getBytes()); headers.set("Authorization", "Basic " + encodedAuth); @@ -76,11 +73,12 @@ private HttpHeaders createHeaders(SmsConnector config) { } private BeemSmsRequest createRequestBody(SmsConnector config, String to, String message) { + BeemRequestSettings settings = getRequestSettings(config); BeemSmsRequest requestBody = new BeemSmsRequest(); requestBody.setSourceAddr(config.getSenderId()); requestBody.setMessage(message); - requestBody.setScheduleTime(""); - requestBody.setEncoding("0"); + requestBody.setScheduleTime(settings.scheduleTime()); + requestBody.setEncoding(settings.encoding()); Recipient recipient = new Recipient(); recipient.setDestAddr(to); @@ -90,22 +88,47 @@ private BeemSmsRequest createRequestBody(SmsConnector config, String to, String return requestBody; } + /** + * Beem's optional request settings can be supplied in a connector's + * {@code extraSettings}, for example {@code {"encoding":"0","schedule_time":"2026-08-15 10:30"}}. + */ + private BeemRequestSettings getRequestSettings(SmsConnector config) { + if (config.getExtraSettings() == null || config.getExtraSettings().isBlank()) { + return new BeemRequestSettings("0", ""); + } + + try { + Map settings = objectMapper.readValue(config.getExtraSettings(), + new TypeReference>() { + }); + return new BeemRequestSettings( + String.valueOf(settings.getOrDefault("encoding", "0")), + String.valueOf(settings.getOrDefault("schedule_time", ""))); + } catch (Exception e) { + throw new IllegalArgumentException("BEEM extraSettings must be valid JSON", e); + } + } + private SmsSendResult processResponse(ResponseEntity response, String to) { BeemSmsResponse responseBody = response.getBody(); Map responseMap = objectMapper.convertValue(responseBody, new TypeReference>() { }); - if (response.getStatusCode().is2xxSuccessful()) { + if (response.getStatusCode().is2xxSuccessful() + && (responseBody == null || !Boolean.FALSE.equals(responseBody.getSuccessful()))) { log.info("BEEM: SMS sent successfully to {}", to); return SmsSendResult.success( - "SMS sent successfully", - responseBody != null ? responseBody.getMessageId() : null, + responseBody != null && responseBody.getMessage() != null ? responseBody.getMessage() + : "SMS sent successfully", + responseBody != null ? responseBody.getRequestId() : null, responseMap); } else { return SmsSendResult.failure( - "Beem API Error", - String.valueOf(response.getStatusCode().value()), + responseBody != null && responseBody.getMessage() != null ? responseBody.getMessage() + : "Beem API Error", + responseBody != null && responseBody.getCode() != null ? String.valueOf(responseBody.getCode()) + : String.valueOf(response.getStatusCode().value()), responseMap); } } @@ -175,17 +198,20 @@ static class Recipient { @NoArgsConstructor @AllArgsConstructor static class BeemSmsResponse { - private boolean valid; + private Boolean successful; private String message; - private int code; + private Integer code; - @JsonProperty("message_id") - private String messageId; + @JsonProperty("request_id") + private String requestId; - BeemSmsResponse(boolean valid, String message, int code) { - this.valid = valid; + BeemSmsResponse(boolean successful, String message, int code) { + this.successful = successful; this.message = message; this.code = code; } } + + private record BeemRequestSettings(String encoding, String scheduleTime) { + } } diff --git a/src/main/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParser.java b/src/main/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParser.java index f3a37bf..6598809 100644 --- a/src/main/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParser.java +++ b/src/main/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParser.java @@ -21,12 +21,12 @@ public String getProvider() { @Override public DlrResult parse(Map payload) { - String messageId = String.valueOf(payload.getOrDefault("messageID", "")); + String messageId = String.valueOf(payload.getOrDefault("request_id", payload.getOrDefault("messageID", ""))); String rawStatus = String.valueOf(payload.getOrDefault("status", "")).toLowerCase(); SmsLogStatus status = null; if (DELIVERED_STATUSES.contains(rawStatus)) { - status = SmsLogStatus.SENT; + status = SmsLogStatus.DELIVERED; } else if (FAILED_STATUSES.contains(rawStatus)) { status = SmsLogStatus.FAILED; } diff --git a/src/main/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorker.java b/src/main/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorker.java new file mode 100644 index 0000000..564d640 --- /dev/null +++ b/src/main/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorker.java @@ -0,0 +1,68 @@ +package com.flexcodelabs.flextuma.modules.notification.services; + +import java.time.LocalDateTime; +import java.util.List; + +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; +import org.springframework.beans.factory.annotation.Value; + +import com.flexcodelabs.flextuma.core.entities.sms.SmsLog; +import com.flexcodelabs.flextuma.core.enums.SmsLogStatus; +import com.flexcodelabs.flextuma.core.repositories.SmsLogRepository; +import com.flexcodelabs.flextuma.core.senders.BeemDeliveryReportClient; + +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; + +/** Polls Beem after its recommended five-minute delivery-report delay. */ +@Slf4j +@Service +@RequiredArgsConstructor +public class BeemDeliveryReportWorker { + + private final SmsLogRepository logRepository; + private final BeemDeliveryReportClient deliveryReportClient; + + @Value("${flextuma.sms.beem.delivery-minimum-delay-minutes}") + private int minimumReportDelayMinutes = 5; + + @Scheduled(fixedDelayString = "${flextuma.sms.beem.delivery-poll-interval-ms}") + @Transactional + public void pollDeliveryReports() { + List sentMessages = logRepository + .findTop50ByStatusAndProviderMessageIdIsNotNullOrderByCreatedAsc(SmsLogStatus.SENT); + LocalDateTime eligibleBefore = LocalDateTime.now().minusMinutes(minimumReportDelayMinutes); + + sentMessages.stream() + .filter(this::isBeemMessage) + .filter(log -> log.getCreated() != null && !log.getCreated().isAfter(eligibleBefore)) + .forEach(this::updateDeliveryStatus); + } + + private boolean isBeemMessage(SmsLog log) { + return log.getConnector() != null && "BEEM".equalsIgnoreCase(log.getConnector().getProvider()); + } + + private void updateDeliveryStatus(SmsLog smsLog) { + try { + BeemDeliveryReportClient.BeemDeliveryStatus report = deliveryReportClient.lookup(smsLog.getConnector(), + smsLog.getRecipient(), smsLog.getProviderMessageId()); + if (report == null || report.getStatus() == null) { + return; + } + + switch (report.getStatus().trim().toUpperCase()) { + case "DELIVERED" -> smsLog.setStatus(SmsLogStatus.DELIVERED); + case "UNDELIVERED" -> smsLog.setStatus(SmsLogStatus.FAILED); + default -> { + return; + } + } + logRepository.save(smsLog); + } catch (Exception e) { + log.warn("Unable to retrieve Beem delivery report for SmsLog [{}]: {}", smsLog.getId(), e.getMessage()); + } + } +} diff --git a/src/main/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookController.java b/src/main/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookController.java index 064e91b..bd41ba8 100644 --- a/src/main/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookController.java +++ b/src/main/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookController.java @@ -9,7 +9,6 @@ import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestHeader; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.server.ResponseStatusException; import org.springframework.http.HttpStatus; @@ -25,15 +24,12 @@ import com.flexcodelabs.flextuma.core.webhooks.DlrResult; import lombok.extern.slf4j.Slf4j; -import org.springframework.beans.factory.annotation.Value; -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; /** * Receives Delivery Report (DLR) callbacks from SMS providers. * *

- * Endpoint: {@code POST /api/webhooks/sms/{provider}/dlr} + * Endpoint: {@code POST /api/webhooks/{provider}} * *

* The {@code provider} path variable must match the @@ -55,9 +51,6 @@ public class SmsWebhookController { private final DataHydratorService hydratorService; private final NotificationService notificationService; - @Value("${flextuma.webhooks.sms.shared-secret:}") - private String webhookSharedSecret; - public SmsWebhookController(SmsLogRepository logRepository, List dlrParsers, ConnectorConfigService configService, DataHydratorService hydratorService, NotificationService notificationService) { @@ -71,16 +64,7 @@ public SmsWebhookController(SmsLogRepository logRepository, List dlrP @PostMapping("/{provider}") public ResponseEntity deliveryReport( @PathVariable String provider, - @RequestHeader(value = "X-Flextuma-Webhook-Secret", required = false) String providedSecret, @RequestBody Map payload) { - - if (webhookSharedSecret.isBlank() || providedSecret == null - || !MessageDigest.isEqual(webhookSharedSecret.getBytes(StandardCharsets.UTF_8), - providedSecret.getBytes(StandardCharsets.UTF_8))) { - log.warn("Rejected DLR from provider [{}] due to invalid webhook credential", provider); - return ResponseEntity.status(HttpStatus.UNAUTHORIZED).build(); - } - log.debug("DLR received from provider [{}]: {}", provider, payload); DlrParser parser = dlrParsers.stream() @@ -114,8 +98,8 @@ public ResponseEntity deliveryReport( SmsLog smsLog = logOpt.get(); - if (SmsLogStatus.SENT.equals(smsLog.getStatus()) && SmsLogStatus.FAILED.equals(result.status())) { - log.warn("DLR: ignoring FAILED update for already-SENT log [{}]", smsLog.getId()); + if (SmsLogStatus.DELIVERED.equals(smsLog.getStatus()) && SmsLogStatus.FAILED.equals(result.status())) { + log.warn("DLR: ignoring FAILED update for already-DELIVERED log [{}]", smsLog.getId()); return ResponseEntity.ok().build(); } diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties index d5225e1..c29e33b 100644 --- a/src/main/resources/application.properties +++ b/src/main/resources/application.properties @@ -1,21 +1,21 @@ -spring.application.name=flextuma +spring.application.name=${SPRING_APPLICATION_NAME:flextuma} spring.datasource.hikari.maximum-pool-size=${HIKARI_MAX_POOL:10} spring.datasource.hikari.minimum-idle=${HIKARI_MIN_IDLE:5} spring.datasource.hikari.idle-timeout=${HIKARI_IDLE_TIMEOUT:300000} spring.datasource.hikari.connection-timeout=${HIKARI_CONN_TIMEOUT:20000} -logging.config=classpath:logback-spring.xml -spring.output.ansi.enabled=ALWAYS -spring.devtools.restart.enabled=true +logging.config=${LOGGING_CONFIG:classpath:logback-spring.xml} +spring.output.ansi.enabled=${SPRING_OUTPUT_ANSI_ENABLED:ALWAYS} +spring.devtools.restart.enabled=${SPRING_DEVTOOLS_RESTART_ENABLED:true} # Enable INFO logging to see startup and seeding -logging.level.com.flexcodelabs.flextuma=INFO -logging.level.org.springframework.boot.autoconfigure=INFO -logging.level.com.flexcodelabs.flextuma.core.config.RequestLoggingFilter=DEBUG +logging.level.com.flexcodelabs.flextuma=${LOG_LEVEL_FLEXTUMA:INFO} +logging.level.org.springframework.boot.autoconfigure=${LOG_LEVEL_SPRING_BOOT_AUTOCONFIGURE:INFO} +logging.level.com.flexcodelabs.flextuma.core.config.RequestLoggingFilter=${LOG_LEVEL_REQUEST_LOGGING_FILTER:DEBUG} spring.data.redis.host=${REDIS_HOST:redis} spring.data.redis.port=${REDIS_PORT:6379} -spring.data.redis.repositories.enabled=false +spring.data.redis.repositories.enabled=${SPRING_DATA_REDIS_REPOSITORIES_ENABLED:false} server.servlet.session.timeout=${SESSION_TIMEOUT:30m} @@ -32,36 +32,40 @@ logging.level.org.springframework.transaction=${TRANSACTION_LEVEL:WARN} logging.level.org.hibernate.engine.jdbc.spi.SqlExceptionHelper=${SQL_EXCEPTION_HELPER_LEVEL:WARN} # Disable verbose Spring exception resolver logs -logging.level.org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver=WARN +logging.level.org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver=${LOG_LEVEL_SPRING_EXCEPTION_HANDLER_RESOLVER:WARN} # Disable Hibernate constraint violation warnings logging.level.org.hibernate.orm.jdbc.warn=${HIBERNATE_JDBC_WARN_LEVEL:ERROR} # Clean up Spring Boot logging format -logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n +logging.pattern.console=${LOG_CONSOLE_PATTERN:%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n} # Disable specific verbose loggers -logging.level.org.apache.catalina.core=WARN -logging.level.org.apache.catalina=WARN -logging.level.org.springframework.boot=WARN -logging.level.org.springframework.web=WARN +logging.level.org.apache.catalina.core=${LOG_LEVEL_CATALINA_CORE:WARN} +logging.level.org.apache.catalina=${LOG_LEVEL_CATALINA:WARN} +logging.level.org.springframework.boot=${LOG_LEVEL_SPRING_BOOT:WARN} +logging.level.org.springframework.web=${LOG_LEVEL_SPRING_WEB:WARN} spring.web.error.include-message=${ERROR_INCLUDE_MESSAGE:always} # SMS Pricing -flextuma.sms.price-per-segment=${SMS_PRICE_PER_SEGMENT:} +flextuma.sms.price-per-segment=${SMS_PRICE_PER_SEGMENT:20.0} + +# Beem delivery reports +flextuma.sms.beem.delivery-poll-interval-ms=${FLEXTUMA_SMS_BEEM_DELIVERY_POLL_INTERVAL_MS:60000} +flextuma.sms.beem.delivery-minimum-delay-minutes=${FLEXTUMA_SMS_BEEM_DELIVERY_MINIMUM_DELAY_MINUTES:5} # App Upload Directories flextuma.app.upload.directory=${APP_UPLOAD_DIRECTORY:/tmp/apps} flextuma.app.frontend.directory=${APP_FRONTEND_DIRECTORY:/tmp/frontend} # Frontend Static Files - Completely disabled to use custom FrontendController -spring.web.resources.add-mappings=false -spring.web.resources.static-locations= -spring.mvc.static-path-pattern=/** +spring.web.resources.add-mappings=${SPRING_WEB_RESOURCES_ADD_MAPPINGS:false} +spring.web.resources.static-locations=${SPRING_WEB_RESOURCES_STATIC_LOCATIONS:} +spring.mvc.static-path-pattern=${SPRING_MVC_STATIC_PATH_PATTERN:/**} # System Logging flextuma.logging.min-level=${LOG_MIN_LEVEL:WARN} flextuma.logging.retention-days=${LOG_RETENTION_DAYS:30} flextuma.logging.batch-size=${LOG_BATCH_SIZE:50} -flextuma.logging.flush-interval-ms=${LOG_FLUSH_INTERVAL:2000} \ No newline at end of file +flextuma.logging.flush-interval-ms=${LOG_FLUSH_INTERVAL:2000} diff --git a/src/test/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnectorTest.java b/src/test/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnectorTest.java index 34baa05..44d1d83 100644 --- a/src/test/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnectorTest.java +++ b/src/test/java/com/flexcodelabs/flextuma/core/entities/sms/SmsConnectorTest.java @@ -72,4 +72,5 @@ void isDefault_shouldBeTrueByDefault() { assertTrue(connector.getIsDefault()); } + } diff --git a/src/test/java/com/flexcodelabs/flextuma/core/senders/BeamSenderTest.java b/src/test/java/com/flexcodelabs/flextuma/core/senders/BeamSenderTest.java index 62ae006..9ed0002 100644 --- a/src/test/java/com/flexcodelabs/flextuma/core/senders/BeamSenderTest.java +++ b/src/test/java/com/flexcodelabs/flextuma/core/senders/BeamSenderTest.java @@ -13,9 +13,13 @@ import org.springframework.http.ResponseEntity; import org.springframework.web.client.RestTemplate; -import static org.junit.jupiter.api.Assertions.*; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) @@ -87,4 +91,26 @@ void sendSms_shouldReturnFailure_whenConnectionFails() { assertEquals("SEND_ERROR", result.getErrorCode()); assertNotNull(result.getProviderResponse()); } + + @Test + void sendSms_shouldUseDocumentedOptionalSettingsAndRequestId() { + config.setExtraSettings("{\"encoding\":\"8\",\"schedule_time\":\"2026-08-15 10:30\"}"); + BeemSender.BeemSmsResponse responseBody = new BeemSender.BeemSmsResponse(true, + "Message Submitted Successfully", 100); + responseBody.setRequestId("67"); + ResponseEntity response = new ResponseEntity<>(responseBody, HttpStatus.OK); + when(restTemplate.postForEntity(eq(config.getUrl()), any(HttpEntity.class), eq(BeemSender.BeemSmsResponse.class))) + .thenReturn(response); + + SmsSendResult result = beemSender.sendSms(config, "255712345678", "Hello World"); + + org.mockito.ArgumentCaptor captor = org.mockito.ArgumentCaptor.forClass(HttpEntity.class); + verify(restTemplate).postForEntity(eq(config.getUrl()), captor.capture(), eq(BeemSender.BeemSmsResponse.class)); + BeemSender.BeemSmsRequest request = (BeemSender.BeemSmsRequest) captor.getValue().getBody(); + assertEquals("8", request.getEncoding()); + assertEquals("2026-08-15 10:30", request.getScheduleTime()); + assertEquals("Basic dGVzdC1rZXk6dGVzdC1zZWNyZXQ=", captor.getValue().getHeaders().getFirst("Authorization")); + assertEquals("67", result.getProviderMessageId()); + assertEquals("Message Submitted Successfully", result.getMessage()); + } } diff --git a/src/test/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParserTest.java b/src/test/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParserTest.java index a556fd4..5997b42 100644 --- a/src/test/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParserTest.java +++ b/src/test/java/com/flexcodelabs/flextuma/core/webhooks/BeemDlrParserTest.java @@ -17,15 +17,15 @@ void getProvider_shouldReturnBeem() { } @Test - void parse_withDeliveredStatus_shouldReturnSent() { + void parse_withDeliveredStatus_shouldReturnDelivered() { Map payload = Map.of( - "messageID", "12345", + "request_id", "12345", "status", "Delivered"); DlrResult result = parser.parse(payload); assertEquals("12345", result.messageId()); - assertEquals(SmsLogStatus.SENT, result.status()); + assertEquals(SmsLogStatus.DELIVERED, result.status()); assertEquals("delivered", result.rawStatus()); } diff --git a/src/test/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorkerTest.java b/src/test/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorkerTest.java new file mode 100644 index 0000000..a7d42d9 --- /dev/null +++ b/src/test/java/com/flexcodelabs/flextuma/modules/notification/services/BeemDeliveryReportWorkerTest.java @@ -0,0 +1,58 @@ +package com.flexcodelabs.flextuma.modules.notification.services; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.time.LocalDateTime; +import java.util.List; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; + +import com.flexcodelabs.flextuma.core.entities.sms.SmsConnector; +import com.flexcodelabs.flextuma.core.entities.sms.SmsLog; +import com.flexcodelabs.flextuma.core.enums.SmsLogStatus; +import com.flexcodelabs.flextuma.core.repositories.SmsLogRepository; +import com.flexcodelabs.flextuma.core.senders.BeemDeliveryReportClient; + +@ExtendWith(MockitoExtension.class) +class BeemDeliveryReportWorkerTest { + + @Mock + private SmsLogRepository logRepository; + + @Mock + private BeemDeliveryReportClient deliveryReportClient; + + @InjectMocks + private BeemDeliveryReportWorker worker; + + @Test + void pollDeliveryReports_shouldMarkDeliveredBeemMessage() { + SmsConnector connector = new SmsConnector(); + connector.setProvider("BEEM"); + SmsLog smsLog = new SmsLog(); + smsLog.setConnector(connector); + smsLog.setRecipient("255700000001"); + smsLog.setProviderMessageId("31951"); + smsLog.setStatus(SmsLogStatus.SENT); + smsLog.setCreated(LocalDateTime.now().minusMinutes(6)); + BeemDeliveryReportClient.BeemDeliveryStatus report = new BeemDeliveryReportClient.BeemDeliveryStatus(); + report.setStatus("DELIVERED"); + + when(logRepository.findTop50ByStatusAndProviderMessageIdIsNotNullOrderByCreatedAsc(SmsLogStatus.SENT)) + .thenReturn(List.of(smsLog)); + when(deliveryReportClient.lookup(connector, "255700000001", "31951")).thenReturn(report); + + worker.pollDeliveryReports(); + + assertEquals(SmsLogStatus.DELIVERED, smsLog.getStatus()); + verify(logRepository).save(smsLog); + } +} diff --git a/src/test/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookControllerTest.java b/src/test/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookControllerTest.java index dd97f7a..e0bfad0 100644 --- a/src/test/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookControllerTest.java +++ b/src/test/java/com/flexcodelabs/flextuma/modules/webhook/controllers/SmsWebhookControllerTest.java @@ -31,7 +31,6 @@ import java.util.UUID; import org.springframework.http.ResponseEntity; import static org.mockito.Mockito.times; -import org.springframework.test.util.ReflectionTestUtils; @ExtendWith(MockitoExtension.class) class SmsWebhookControllerTest { @@ -52,10 +51,8 @@ class SmsWebhookControllerTest { private NotificationService notificationService; private SmsWebhookController buildController() { - SmsWebhookController controller = new SmsWebhookController(logRepository, List.of(dlrParser), configService, hydratorService, + return new SmsWebhookController(logRepository, List.of(dlrParser), configService, hydratorService, notificationService); - ReflectionTestUtils.setField(controller, "webhookSharedSecret", "test-secret"); - return controller; } private Map payload(String msgId, String status) { @@ -75,7 +72,7 @@ void deliveryReport_shouldUpdateToSent_whenDelivered() { when(logRepository.findByProviderMessageId("msg-123")).thenReturn(Optional.of(log)); when(logRepository.save(any())).thenReturn(log); - buildController().deliveryReport("beem", "test-secret", payload("msg-123", "delivered")); + buildController().deliveryReport("beem", payload("msg-123", "delivered")); verify(logRepository).findByProviderMessageId("msg-123"); verify(logRepository).save(any()); @@ -92,22 +89,22 @@ void deliveryReport_shouldUpdateToFailed_whenFailed() { when(logRepository.findByProviderMessageId("msg-789")).thenReturn(Optional.of(log)); when(logRepository.save(any())).thenReturn(log); - buildController().deliveryReport("beem", "test-secret", payload("msg-789", "failed")); + buildController().deliveryReport("beem", payload("msg-789", "failed")); verify(logRepository).save(any()); assertEquals(SmsLogStatus.FAILED, log.getStatus()); } @Test - void deliveryReport_shouldNotSave_whenAlreadySentAndFailedDlrArrives() { + void deliveryReport_shouldNotSave_whenAlreadyDeliveredAndFailedDlrArrives() { when(dlrParser.getProvider()).thenReturn("BEEM"); when(dlrParser.parse(any())).thenReturn(new DlrResult("msg-456", SmsLogStatus.FAILED, "failed")); SmsLog log = new SmsLog(); - log.setStatus(SmsLogStatus.SENT); + log.setStatus(SmsLogStatus.DELIVERED); lenient().when(logRepository.findByProviderMessageId("msg-456")).thenReturn(Optional.of(log)); - buildController().deliveryReport("beem", "test-secret", payload("msg-456", "failed")); + buildController().deliveryReport("beem", payload("msg-456", "failed")); verify(logRepository, never()).save(any()); } @@ -116,7 +113,7 @@ void deliveryReport_shouldNotSave_whenAlreadySentAndFailedDlrArrives() { void deliveryReport_shouldNotSave_whenUnknownProvider() { when(dlrParser.getProvider()).thenReturn("BEEM"); - buildController().deliveryReport("unknown_provider", "test-secret", payload("msg-000", "delivered")); + buildController().deliveryReport("unknown_provider", payload("msg-000", "delivered")); verify(logRepository, never()).save(any()); } @@ -126,7 +123,7 @@ void deliveryReport_shouldNotSave_whenNoLogFound() { when(dlrParser.getProvider()).thenReturn("BEEM"); when(dlrParser.parse(any())).thenReturn(new DlrResult("msg-999", SmsLogStatus.SENT, "delivered")); - buildController().deliveryReport("beem", "test-secret", payload("msg-999", "delivered")); + buildController().deliveryReport("beem", payload("msg-999", "delivered")); verify(logRepository, never()).save(any()); } @@ -136,7 +133,7 @@ void deliveryReport_shouldNotSave_whenIntermediateStatus() { when(dlrParser.getProvider()).thenReturn("BEEM"); when(dlrParser.parse(any())).thenReturn(new DlrResult("msg-001", null, "submitted")); - buildController().deliveryReport("beem", "test-secret", payload("msg-001", "submitted")); + buildController().deliveryReport("beem", payload("msg-001", "submitted")); verify(logRepository, never()).save(any()); }