Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use node-fetch instead of request for default transport #40

Merged
merged 69 commits into from
Sep 22, 2023
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
b76a152
chore: use node-fetch wip
Simon2828 Aug 15, 2023
eddb7a9
get request working with node fetch
jamierbower Aug 15, 2023
19e6f71
chore: remove customrequest
Simon2828 Aug 16, 2023
31701fc
add https agent
jamierbower Aug 16, 2023
1bd71cc
rename _request prop to _fetch
jamierbower Aug 21, 2023
3765d7d
get cert opts
jamierbower Aug 21, 2023
d99dff6
chore: wip replicating json body behaviour from request in node-fetch
vaughan-rich Aug 22, 2023
e8dffe5
feat: convert response body to json/text accordingly
vaughan-rich Aug 22, 2023
9b79c5c
feat: add AbortController signal opt
vaughan-rich Aug 22, 2023
644041e
WIP handle errors correctly
jamierbower Aug 23, 2023
e0976c7
feat: custom error message for timed out requests
vaughan-rich Aug 23, 2023
20ac4b3
feat: remove elapsedTime from response
vaughan-rich Aug 23, 2023
4c286cf
reads agent options from agentOpts
jamierbower Aug 23, 2023
3ed1120
WIP use http proxy agent
jamierbower Aug 23, 2023
af4c98c
remove proxy agent stuff
jamierbower Aug 24, 2023
b904a2c
uninstall proxy agent
jamierbower Aug 24, 2023
082b4f3
remove unnecessary override of dns lookup
jamierbower Aug 24, 2023
cf4b233
get test working
jamierbower Aug 29, 2023
2b6b2e5
get first retry test working
jamierbower Aug 29, 2023
cfb0707
fix more timeout tests
jamierbower Aug 29, 2023
657f2df
chore: wip tests
onlyonehas Aug 29, 2023
7ffcb88
WIP fix query string test
jamierbower Aug 29, 2023
1f34bf5
client test file working
jamierbower Aug 31, 2023
c86bb8d
chore: fixing linting + tests
onlyonehas Aug 31, 2023
42b7262
adds elapsedTime to response
jamierbower Sep 5, 2023
39eb8e2
change status to statusCode on response
jamierbower Sep 5, 2023
1798742
fix linting errors in request
jamierbower Sep 5, 2023
4b661b5
whitespace
jamierbower Sep 5, 2023
1547124
add elapsedTime back to type def file
jamierbower Sep 5, 2023
616f1bd
whitespace
jamierbower Sep 5, 2023
b76ee49
tidy up response
jamierbower Sep 5, 2023
ecc7acb
further tidy up
jamierbower Sep 5, 2023
b363f49
fix: correcting typos
simongregory Sep 5, 2023
1ff8ac9
fix: correcting typos
simongregory Sep 5, 2023
6d25604
removes all mention of the callbackifying stuff from code
jamierbower Sep 5, 2023
8260c0b
removes statusText
jamierbower Sep 5, 2023
4dc1e4f
remove bluebird dependency
jamierbower Sep 6, 2023
50118cd
remove some uses of lodash
jamierbower Sep 6, 2023
1fda64b
renames RequestTransport to FetchTransport
jamierbower Sep 7, 2023
c010147
rename class
jamierbower Sep 8, 2023
b674b92
enables setting timeout by default
jamierbower Sep 8, 2023
06d9638
updates type index
jamierbower Sep 8, 2023
57fad88
passes through compress from default config, makes agentOpts optional…
jamierbower Sep 11, 2023
689c8dd
docs: update js docs with one section to todo
vaughan-rich Sep 11, 2023
18e5716
chore: remove request dependencies
vaughan-rich Sep 11, 2023
84767dd
3.5.7-0
vaughan-rich Sep 11, 2023
7b10791
chore: remove request types
vaughan-rich Sep 11, 2023
f2075a7
docs: fix js docs
vaughan-rich Sep 12, 2023
5ca8f60
add section to docs about setting default behaviour of node-fetch
jamierbower Sep 12, 2023
459028c
docs: update readme
vaughan-rich Sep 12, 2023
d316b08
feat: adds http support
vaughan-rich Sep 14, 2023
411d93d
test: wip httpsAgent unit test
vaughan-rich Sep 14, 2023
5f8b5f4
convert module to ESM
jamierbower Sep 18, 2023
2a7bc68
allows disbling timing via setContextProperty
jamierbower Sep 18, 2023
212da7d
tests: check agent is using correct protocol for http/https
onlyonehas Sep 18, 2023
af933d3
test: test correct agent protocol is added
vaughan-rich Sep 18, 2023
612c126
remove unecessary import|
jamierbower Sep 18, 2023
16785cc
3.5.7-1
jamierbower Sep 18, 2023
2c474dc
revert ESM conversion
jamierbower Sep 18, 2023
df7a86e
3.5.7-2
jamierbower Sep 18, 2023
5d682f5
converts headers from iterable to serializable object, fixes leak in …
jamierbower Sep 19, 2023
7cb5ac0
3.5.7-3
jamierbower Sep 19, 2023
27c5bf3
default to from.text() if content type is not json
jamierbower Sep 20, 2023
a411bcf
4.0.0-0
jamierbower Sep 20, 2023
2f68d8e
parse body as json if json default is set
jamierbower Sep 21, 2023
9b3c6e4
4.0.0-1
jamierbower Sep 21, 2023
a72a892
catches error if JSON parsing fails, and defaults response to text
jamierbower Sep 21, 2023
df2c4c5
4.0.0-2
jamierbower Sep 21, 2023
7272c4f
update docs
jamierbower Sep 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
18
12 changes: 6 additions & 6 deletions docs.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# HttpTranport
# HttpTransport

> A flexible, modular REST client built for ease-of-use and resilience

Expand Down Expand Up @@ -32,7 +32,7 @@ PATCH, DELETE, HEAD are also supported.

#### Query strings

Make a HTTP GET request specifiying query strings using `.query`
Make a HTTP GET request specifying query strings using `.query`

Single query string
```js
Expand Down Expand Up @@ -62,7 +62,7 @@ Multiple query strings:

#### Headers

Make a HTTP GET request specifiying request headers using `.headers`
Make a HTTP GET request specifying request headers using `.headers`

Add a single header:
```js
Expand Down Expand Up @@ -176,7 +176,7 @@ Middleware are functions that can be executed with before and after a request. M
* Terminate a request early e.g for caching purposes
* Modify the response object e.g format the response body

Middlware can be executed **per request** using the `.use` method:
Middleware can be executed **per request** using the `.use` method:
```js
const exampleMiddleware = require('exampleMiddleware');

Expand All @@ -193,7 +193,7 @@ Middlware can be executed **per request** using the `.use` method:
}
```

Middlware can also be executed **for every request** using the `.use` of the client builder. The client builder is created using the `createBuilder` method:
Middleware can also be executed **for every request** using the `.use` of the client builder. The client builder is created using the `createBuilder` method:

```js
const exampleMiddleware = require('exampleMiddleware');
Expand All @@ -214,7 +214,7 @@ Middlware can also be executed **for every request** using the `.use` of the cli

For writing middleware, see the [offical guide](/~https://github.com/koajs/koa/blob/master/docs/guide.md)

#### Offical HttpTransport middleware
#### Official HttpTransport middleware
See [Caching](/~https://github.com/bbc/http-transport-cache)

See [Collapsing](/~https://github.com/bbc/http-transport-request-collapse)
Expand Down
17 changes: 10 additions & 7 deletions index.d.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import * as request from "request";
import * as fetch from "node-fetch";
import FetchTransport from "./lib/transport/node-fetch";

export declare function createBuilder(
transport?: Transport
Expand Down Expand Up @@ -85,7 +86,7 @@ export declare class Response {
url: string;
statusCode: number;
body: Body;
httpResponse?: request.Response;
httpResponse?: fetch.Response;
readonly length: number;
static create(opts?: JsonResponse): Response;
addHeader(key: string, value: string): Request;
Expand All @@ -102,7 +103,6 @@ export declare class HttpTransportBuilder<
use<ContextExtra = {}>(
fn: Plugin<ContextExtra, ContextCurrent>
): HttpTransportBuilder<ContextExtra & ContextCurrent>;
asCallback(): HttpTransportBuilder<ContextCurrent>;
createClient(): HttpTransportClient<ContextCurrent>;
}

Expand Down Expand Up @@ -146,7 +146,7 @@ declare class Context {
addPlugin(plugin: Plugin): Context;
}

export declare class defaultTransport extends RequestTransport {
export declare class defaultTransport extends FetchTransport {
constructor(
params: request.RequestAPI<
request.Request,
Expand All @@ -155,8 +155,11 @@ export declare class defaultTransport extends RequestTransport {
>
);
}
export declare class RequestTransport extends Transport {
export declare class FetchTransport extends Transport {
constructor(
agentOpts: {

}
params: request.RequestAPI<
request.Request,
request.CoreOptions,
Expand All @@ -168,10 +171,10 @@ export declare class RequestTransport extends Transport {
export declare class Transport {
toError(err: ErrorObject, ctx: Context): Error;
createError(err: ErrorObject, ctx: Context): Error;
execute(ctx: Context): Promise<RequestTransport>;
execute(ctx: Context): Promise<FetchTransport>;
onError(ctx: Context): Function;
toOptions(ctx: Context): RequestOptions;
toResponse(ctx: Context, from: request.Response): Response;
toResponse(ctx: Context, from: fetch.Response): Response;
makeRequest(ctx: Context, opts: RequestOptions): Promise<Response>;
}

Expand Down
4 changes: 2 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
'use strict';

const HttpTransportBuilder = require('./lib/builder');
const DefaultTransport = require('./lib/transport/request');
const DefaultTransport = require('./lib/transport/node-fetch');

module.exports.defaultTransport = DefaultTransport;
module.exports.RequestTransport = DefaultTransport;
module.exports.FetchTransport = DefaultTransport;
module.exports.builder = HttpTransportBuilder;
module.exports.transport = require('./lib/transport/transport');
module.exports.context = require('./lib/context');
Expand Down
23 changes: 2 additions & 21 deletions lib/builder.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

const _ = require('lodash');

const CallbackDecorator = require('./callbacks/decorator');
const HttpTransportClient = require('./client');

function validatePlugin(plugin) {
Expand All @@ -16,7 +15,6 @@ class HttpTransportBuilder {
* @param {Transport} transport - Transport instance.
*/
constructor(transport) {
this._callback = false;
this._transport = transport;
this._defaults = {
plugins: []
Expand All @@ -26,7 +24,7 @@ class HttpTransportBuilder {
/**
* Sets a default user agent
*
* @param {string} agent - user agant
* @param {string} agent - user agent
* @return a HttpTransportBuilder instance
* @example
* const httpTransport = require('@bbc/http-transport');
Expand Down Expand Up @@ -89,19 +87,6 @@ class HttpTransportBuilder {
return this;
}

/**
* Callbackify the client
*
* @return a HttpTransport instance supporting callbacks
* @example
*
* const client = httpTransport.asCallback().createClient();
*/
asCallback() {
this._callback = true;
return this;
}

/**
* Instantiates a HttpTransport
*
Expand All @@ -112,11 +97,7 @@ class HttpTransportBuilder {
* const client = httpTransport.createClient();
*/
createClient() {
const transportClient = new HttpTransportClient(this._transport, this._defaults);
if (this._callback) {
return new CallbackDecorator(transportClient);
}
return transportClient;
return new HttpTransportClient(this._transport, this._defaults);
}
}

Expand Down
81 changes: 0 additions & 81 deletions lib/callbacks/decorator.js

This file was deleted.

5 changes: 2 additions & 3 deletions lib/client.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
'use strict';

const _ = require('lodash');
const compose = require('koa-compose');
const context = require('./context');
const rejectedPromise = require('./rejectedPromise');
Expand All @@ -11,11 +10,11 @@ class HttpTransportClient {
/**
* Create a HttpTransport.
* @param {Transport} transport - Transport instance.
* @param {object} defaults - default configutation
* @param {object} defaults - default configuration
*/
constructor(transport, defaults) {
this._transport = transport;
this._instancePlugins = _.get(defaults, 'plugins', []);
this._instancePlugins = defaults.plugins || [];
this._defaults = defaults;
this._initContext();
bind(this);
Expand Down
11 changes: 4 additions & 7 deletions lib/context.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
'use strict';

const _ = require('lodash');

const Request = require('./request');
const Response = require('./response');
const packageInfo = require('../package');
Expand All @@ -21,8 +19,7 @@ class Context {
}

get retryAttempts() {
if (_.isUndefined(this._retryAttempts)) return [];
return this._retryAttempts;
return this._retryAttempts || [];
}

set retryAttempts(retryAttempts) {
Expand All @@ -38,9 +35,9 @@ class Context {
}

_applyDefaults(defaults) {
this.userAgent = _.get(defaults, 'ctx.userAgent', USER_AGENT);
this.retries = _.get(defaults, 'ctx.retries', RETRIES);
this.retryDelay = _.get(defaults, 'ctx.retryDelay', RETRY_DELAY);
this.userAgent = defaults.ctx?.userAgent || USER_AGENT;
this.retries = defaults.ctx?.retries || RETRIES;
this.retryDelay = defaults.ctx?.retryDelay || RETRY_DELAY;
}

static create(defaults) {
Expand Down
11 changes: 3 additions & 8 deletions lib/middleware/logger.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
'use strict';

const _ = require('lodash');

function isRetry(ctx) {
const attempts = ctx.retryAttempts || [];
return attempts.length > 0;
Expand All @@ -12,7 +10,7 @@ function isCriticalError(ctx) {
}

function hasElapsedTime(ctx) {
return !_.isUndefined(_.get(ctx, 'res.elapsedTime'));
return !!ctx.res?.elapsedTime;
}

function getBaseMessage(ctx) {
Expand All @@ -29,11 +27,8 @@ function createRequestMessage(ctx) {
}

function createRetryMessage(ctx) {
const res = ctx.res;
const attempts = ctx.retryAttempts;

const message = `Attempt ${attempts.length} ${getBaseMessage(ctx)}`;
if (hasElapsedTime(ctx)) return `${message} ${res.elapsedTime} ms`;
const message = `Attempt ${ctx.retryAttempts.length} ${getBaseMessage(ctx)}`;
if (hasElapsedTime(ctx)) return `${message} ${ctx.res.elapsedTime} ms`;

return message;
}
Expand Down
2 changes: 0 additions & 2 deletions lib/response.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
'use strict';

const _ = require('lodash');

const REQUIRED_PROPERTIES = [
'url',
'body',
Expand Down
Loading