Feature/qiniu prefetch - #16
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an optional “prefetch after upload” workflow to the @lancercomet/qiniu-uploader CLI so that successfully uploaded assets can be cache-warmed via Qiniu Fusion CDN.
Changes:
- Add
--prefetch,--prefetch-domain, and--prefetch-productCLI options and implement post-upload prefetch batching/retry logic. - Update README with prefetch usage and behavior notes.
- Bump package version metadata.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| QiniuUploader/package.json | Updates package version. |
| QiniuUploader/package-lock.json | Updates lockfile package version fields accordingly. |
| QiniuUploader/index.mjs | Implements upload key collection + Qiniu Fusion CDN prefetch request flow. |
| QiniuUploader/README.md | Documents prefetch options and behavior. |
Files not reviewed (1)
- QiniuUploader/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "name": "@lancercomet/qiniu-uploader", | ||
| "version": "0.1.2", | ||
| "version": "0.2.1-test", |
There was a problem hiding this comment.
package-lock.json also records the package version as "0.2.1-test". If the prerelease suffix is not meant to be published, revert both version fields here to the release version to avoid lockfile churn and unexpected published metadata.
| let data = '' | ||
| res.on('data', (chunk) => { data += chunk }) | ||
| res.on('end', () => { | ||
| try { | ||
| resolve(JSON.parse(data)) | ||
| } catch { | ||
| reject(new Error(`Failed to parse prefetch response: ${data}`)) |
There was a problem hiding this comment.
Prefetch response handling can crash the whole CLI: if Qiniu returns a non-JSON body (e.g. HTML error page, empty body, proxy error), JSON.parse will throw and reject, and callers don't catch it. Consider checking res.statusCode, buffering only on expected content-types, and returning a structured error (or catching the rejection in prefetchAllUrls and treating it as retryable/failed batch) so uploads don't fail due to a transient prefetch error.
| let data = '' | |
| res.on('data', (chunk) => { data += chunk }) | |
| res.on('end', () => { | |
| try { | |
| resolve(JSON.parse(data)) | |
| } catch { | |
| reject(new Error(`Failed to parse prefetch response: ${data}`)) | |
| const statusCode = res.statusCode || 0 | |
| const contentType = String(res.headers['content-type'] || '').toLowerCase() | |
| const isJsonResponse = contentType.includes('application/json') | |
| let data = '' | |
| res.on('data', (chunk) => { data += chunk }) | |
| res.on('end', () => { | |
| if (!data) { | |
| resolve({ | |
| code: statusCode, | |
| error: `Prefetch API returned an empty response${statusCode ? ` (HTTP ${statusCode})` : ''}` | |
| }) | |
| return | |
| } | |
| if (!isJsonResponse) { | |
| resolve({ | |
| code: statusCode, | |
| error: `Prefetch API returned a non-JSON response${statusCode ? ` (HTTP ${statusCode})` : ''}: ${data}` | |
| }) | |
| return | |
| } | |
| try { | |
| resolve(JSON.parse(data)) | |
| } catch { | |
| resolve({ | |
| code: statusCode, | |
| error: `Failed to parse prefetch response${statusCode ? ` (HTTP ${statusCode})` : ''}: ${data}` | |
| }) |
|
|
||
| while (retries <= maxRetries && !success) { | ||
| if (retries > 0) { | ||
| const delay = retries * 2000 |
There was a problem hiding this comment.
The comment says "exponential backoff", but the retry delay is currently linear (retries * 2000). Either update the delay calculation to be exponential (e.g. baseDelay * 2^(retries-1)) or adjust the comment to match the actual behavior.
| while (retries <= maxRetries && !success) { | |
| if (retries > 0) { | |
| const delay = retries * 2000 | |
| const baseDelay = 2000 | |
| while (retries <= maxRetries && !success) { | |
| if (retries > 0) { | |
| const delay = baseDelay * Math.pow(2, retries - 1) |
|
|
||
| - URLs are submitted in batches of 20 (Qiniu Fusion CDN API limit). | ||
| - A 1-second interval is applied between batches to respect the rate limit. | ||
| - On QPS throttling (403024) or server errors (500000), the batch will be retried up to 3 times with exponential backoff. |
There was a problem hiding this comment.
The README says retries use "exponential backoff", but the implementation uses a linear delay (retries * 2000). Please update either the documentation or the retry logic so they match.
| - On QPS throttling (403024) or server errors (500000), the batch will be retried up to 3 times with exponential backoff. | |
| - On QPS throttling (403024) or server errors (500000), the batch will be retried up to 3 times with a linearly increasing delay (2s, 4s, 6s). |
| { | ||
| "name": "@lancercomet/qiniu-uploader", | ||
| "version": "0.1.2", | ||
| "version": "0.2.1-test", |
There was a problem hiding this comment.
The package version was changed to a prerelease string ("0.2.1-test"). If this is intended only for local testing, consider keeping the real semver version here (e.g. 0.2.1) and publish the prerelease via npm dist-tags or a separate prerelease versioning strategy, otherwise consumers may accidentally depend on the test build.
| "version": "0.2.1-test", | |
| "version": "0.2.1", |
No description provided.