v2.0.0 - The next iteration

Today I'm excited to announce the 2.0.0 release of the Travetto framework. This release represents nearly a years worth of enhancements, refactors, and changes. This a major release with some breaking changes. The general flow, and usage of the framework is unchanged, but some substatial under the hood changes have been implemented.

One of the biggest drivers for this release was removal of overlapping functionality. This means areas where functionality was similar or in certain cases identical, that functionality was moved to the appropriate location to be able to be leveraged by each consumer. The main areas of simplification were:

• Schema and input validation
• Model and data store accessing
• Logging behavior between the base module and the logging module
• Compiler behavior between the compiler module and the bootstrap transpiler support.
• Entrypoints for direct invocation of framework code
• Source code scanning, and module loading

Major and Breaking Changes

Schema Overhaul

Schema has taken a role as the gatekeeper of all inbound data into the application. Now application, config and rest utilize the schema transformations and validations for entrypoints. This enables consistent use of schema type information, validators, in all of these modules. This also means the error messaging is consistent and behaves the same way across all of these modules.

Model Overhaul

The model service has now become a building block for other services (e.g. Asset, Cache, Auth-Model), and has transitioned to contract based definitions. This means that model implementations no longer have a requirement to be identical in functionality, but can opt into whatever contracts the model service is able to provide.

• Asset now relies on Models with Streaming support
• Cache now relies on Models with Expiry support
• Auth-Model relies on Models alone
• S3, Firebase, Redis, Dynamo were all added as standard model providers
• All model implementations now have extension testing for the services that they are compatible with
• asset-mongo, and asset-s3 are gone, and relies on a model provider that has streaming
• cache/src/extension/{redis,dynamdb} are gone and are also now model provided
• model is gone and has been replaced by model-core and model-query.
• model-core is a series of interfaces/contracts, and some minor utility functions. All ownership has been pushed to the various providers.
• Method names have been standardized as verbNoun e.g. getStream or deleteExpired

Auth Overhaul

Authentication has also been overhauled, to reduce complexity and number of contracts/interfaces that must be understood to implement a custom authentication flow. Additionally, with the changes in the DI module to prefer interfaces to classes, implementing authenticators, and authorizers can now be as simple as providing a simple function.

• Greatly simplified number of interfaces/classes to understand
• Identity has been folded into principal, and is now the standard bearer for a known user
• Request object has been reduced in complexity, and AuthContext is gone.
• Session has been reworked to be the counterpart to the JWT for encoding a principal to the user

Rest Internals Overhaul

Rest has been migrated to leverage the schema module for declaring and validating endpoint inputs. Additionally a lot of work has been put into inferring sensible assumptions for undeclared endpoint parameters. This allows for the simplification of endpoint creation.

• Testing support greatly increased, and provides clearer behavior for testing as a server, and as a lambda.
• Streamlined internals, and separated lambda from general usage

Support for Dynamic Module references, specifically, third party

The module loading system has been rewitten to allow for easier testing of extensions. One of the key side-effects of this change, is the ability to map paths to custom modules. This allows for end-users to create their own libraries, and allow them to operate on the same level as framework modules.

• No longer symlinking for local dev
• Allows for better testing of extensions
• Moved the source code indexing into boot to be used by the CLI
  • All support/* files are converted to .ts
  • Indexing only looks for typescript files
  • Will speedup tools that rely upon full file system scanning

Logging overhaul (Base no longer duplicates functionality of log)

The logging functionality used to share quite a bit of overlap with the base module, as there is a delay between the bootstrapping process and when the logging module is initialized. The difference was split, and the base module still provides the general hooks for controlling logging requests, but the logging module is now responsible for formatting, filtering, and everything else logging specific.

• All filtering and formatting now belong to the log module
• All log statements are encouraged to following the pattern of message, { payload }
• Startup logs may still need some support if the goal is suppression

Typescript Upgrade

The framework attempts to stay abreast of every major release of the Typescript library, and moving to the last ECMAScript features as they reach stability.

• Shifted codebase away from use of any to unknown where applicable (over 750 instances migrated)
• Migrated all private var usages to #var, and aligning with class initialization changes.
• Moving to typescript 4.3
• Converted all available files to .ts, only build scripts remain in .js

Dependency Injection Enhancements

Dependency injection has been pushed to honor interfaces over classes, where possible. There is some "magic" that has to happen behind the scenes, but for the majority of the developer experience, interfaces are just as usable, and even shape types can be leveraged in certain scenarios.

• Using more interfaces where possible (and less reliance on abstract clasess)
  • This allows for better control at the cost of potentially duplicated functionality
• Support for injecting/registering by interfaces
• Better default behavior on multiple providers, with local code breaking ties.

Module reorg

One of the other simplification goals, was to remove the sprawl of modules, where the functionality was very thin. The new design is to push as much to "extensions" where possible and it makes sense. If the extra functionality is too divergent, it will continue to be a module. A great example of this would be auth vs auth-rest.

• auth-passport is gone, and is now an extension of auth-rest.
• auth-model is gone, and is now an extension of auth.
• asset-* for implementations, are now model modules
• cache's built in extensions have been removed. model with expiry support is all that is needed now.
• Extensions have been moved to the module which owns the complexity (e.,g. schema rest support dealt more with the internals of rest than schema, and has been moved).

Compiler Ownership

The compiler module and the bootstrap transpiler had a lot of overlap, and made it slightly confusing to trace the flow of execution depending on the file name/type. This has been simplified, and now the bootstrap transpilation code is required for initiating all compilations. The compiler module hooks into this flow, and registers an alternate transpiler (with the appropriate transformers).

• Reworked @travetto/boot compiler to create clear responsiblity for managing the compiler/transpiler relationsip with node runtime.
• Modified @travetto/compiler to no longer register extension directly, but extend functionality defined in boot

Entrypoint Standardization

Entrypoints have always been a little complicated, as the targetted entrypoint has to bootstrap the framework, and then delegate to the appropriate method. This resulted in a lot of extra code that was highly repetitive. To that end, the framework now has established a pattern for invoking a file relying on the presence of a main method. Boot provides it's own launcher, which does not initialize the framework, but the just the transpilation support. The base module also provides a launcher, which does initialize the framework.

• Allow for use of main functions to allow for direct invocation of any file, primaryily used for plugins and cli activities
• Removed all plugin-*.js files as in lieu of exposing a main function in the target files.
• Removed almost all *.js files in the test folders, in lieu of exposing main functions. Tests no longer auto execute on import.

Removed sync versions of ResourceManager methods

All resource lookups are considered to be async, as runtime support for sync was an anti-pattern

Generator Simplification

Moved away from using yeoman due to dependency bloat, and went with a simple @travetto/cli based solution. Can be invoked with npx @travetto/scaffold. The main difference here is reducing the amount of dependencies needed to be downloaded before the generator begins to run.

Internal Development Changes

• Separated out configuration of which folders to scan during execution (and allowing for soft optional)
• This has the affect of removing a bunch of custom logic around tests
• Alt folders have been removed, and can be emulated by specifying TRV_SRC_LOCAL values as needed.
• Extension Overhaul (and testing thereof overhaul)
• Extensions are now tested in isolation allowing for various combinations of extensions to be tested
• Relies on use of Dynamic Modules to allow for creating a custom cache related to the modules being loaded
• Local Dev Overhual
• Now relies on environment variables (direnv makes it easier) for augmenting what would have been embedded in the framework.
• Using tsconfig paths in lieu of symlinks, general development performance, and refactoring are substantially improved.
• Removed dependency on symlinks
• Lessened dependency on bash scripts, and moved build processes over to @arcsine/nodesh
• Local development should now support windows, but there may still be a few edge cases
• Docs have been converted back to typescript, and the doc folders have been simplified
• Proper typechecking on all docs
• Renamed file from DOCS.js to doc.ts
• Moved main README.md to related/overview/doc.ts
• Lerna Removal
• Removed usage of lerna within framework, handling mono repo tasks manually
• Reduced hoisted node_module size by 40%