7.0.0 - Dynamic Reimagination

 

What's Changed

This is major release that internally has reworked the "Dynamic" portion of the framework, along with a refactor of the "Registry" structure.  Nearly all of this is internal, and usually non-user facing, but there are some caveats.

Re-imagining Dynamic 

Since the beginnging of the framework, the goal has always been towards being dynamic. This included concepts like hot module reloading, route registration, and  updating database schemas on demand.  This has generally been working for nearly a decade, but there were a lot of edge cases that wouldn't work just right.  These stemmed around things like base classes, shared external types (type unions), etc.  Essentially indirection and complex structures, especially externalized, left us open for gaps.

With this release, hot module reloading is gone (along with heaps of supporting code).  The framework now supports running in a mode that will automatically restart the application on the appropriate file changes.  This has verifiably worked well with debuggers, and should provide a nearly identical experiance as compared to before.  

Additionally, when in development, theh framework will reconcile the database schema on startup. This means inspecting the existing database structure, and comparing to what we would have created if writing from scratch, and adjusting the difference.  This covers a litany of edge cases, the least not being switching branches, or making any other changes when the app isn't running.

Registry Unification

As we march closer to Typescript 7 (now with more Go), one of the biggest points of conflict will be the transformer logic the framework employs.  Before this release we had about 5 major transformer: cli, di, schema, model, test and web.  Each of these modules' transformer was doing nearly the same thing.  Check for decorators, add metadata, verify types.

With this release, all that remains are schema and test.  For schema, all the logic/metadata those other modules were collecting are either subsumed into the schema structure, or stored explicitly via the decorator and not the transformer.

This greatly reduces the surface needed to handle when looking towards Typescript 7.0. The goal will most likely be to port the typescript to Go and provide a native version of the transformers that can operate within a custom compiled version.

ES Module Migration

With the removal of the dyanmic module reloading, the last remaining piece keeping the framework supporting CommonJS has been removed.  This allows the framework to completely drop all "dual" support for CommonJS/ESM, and focus solely on ESM.  The codebase already supported  ESM, but this will allow for a more streamlined experience, as well as reduced burden for testing/verification.  Given ESM is the future, this is a natural optimization given the other changes in this release.

General Improvements

General house keeping (depenency bumps), down stream alignments (e.g. VS Code plugin), code hygenie are all on the list of general improvements. 

Whats Next?

The ecosystem is still waiting (5+ years now) on ES Decorators being fully ratified (including parameter decorators), including support for standalone functions.  Once all the necessary features are available within Typescript, the framework will overhaul decorator behavior, and allow for isolated functions where it make sense (e.g. DI factory functions, standlone web routes, caching, etc).

Also, with the notice that Typescript will be moving to Go for version 7+, we will continue to keep an eye on what the story is for custom code transformers.  There is a high chance the transformation logic, that is currently in Typescript will need to be ported to Go. 

6.0.0 - There and Back Again

What's Changed

This is major release with a focus on long-term realignment of web (formerly rest) and auth modules.

Web (Rest) Overhual

The Rest module is one of the core pillars of the Travetto framework, and was built upon the idea of wrapping existing Node frameworks (express, koa, fastify).  While this worked, initially, gaps started to show up across the different implementations.  The implementations relied upon the framework specific middleware for most of the HTTP related operations (cookies, body parse, compression, decompression, etc), and they all did not work the same way.  

With the this release, Travetto has dropped implicit support for external frameworks.  This means that the Web module is now a standalone framework, and is a peer to the frameworks its replacing.  All the standard middleware that was previously external framework dependent, has been rewritten as core Travetto intercpetors. This new web framework departs from express/connect structure, and aligns with a request/response model that is similar to AWS's lambda event model.

Speaking of lambda, another fallout of the rest module deprecation, is that the various lambda implementations have been removed, making way for a single lambda interface.  

Additionally, the RPC module has  been promoted to general availability and has replaced the Rest-Client module.  The Web framework has also been reorganized to ensure that the endpoint structure is aligned with method invocations.  This means that all request contextual data (the request object itself, principal, or session information), is no longer able to be declared on the endpoint, but will be defined via a new decorator @ContextParam.

Auth Overhaul

The auth family of modules have also been changed.  The primary change was more of a musical chairs than it was a rewrite.  The biggest goal was to shift the core functionality from Auth-Web, and Auth-Web-Session into Auth, and Auth-Session.  The core of auth functionality is now accessible outside of web contexts, and is ideal for building auth-aware applications, e.g. CLI commands that want to establish the auth context.  

General Improvements

The compiler has continued to receive improvements, and many fixes were introduced for file deletions/additions within the watching process, along with the related functionality for the testing framework.

Beyond the overhauls of the Rest and Auth module-families, continued improvements have been made to the compiler, the testing infrastructure, the VS Code plugin, documentation, etc.  So many core changes have been made, and the foundation for the future of the framework is solid.  

Whats Next?

The ecosystem is still waiting (4+ years now) on ES Decorators being fully ratified (including parameter decorators), including support for standalone functions.  Once all the necessary features are available within Typescript, the framework will overhaul decorator behavior, and allow for isolated functions where it make sense (e.g. DI factory functions, standlone web routes, caching, etc).

Also, with the notice that Typescript will be moving to Go for version 7+, we will continue to keep an eye on what the story is for custom code transformers.  There is a high chance the transformation logic, that is currently in Typescript will need to be ported to Go.  The window for the change is still years out, but is worth keeping in mind.

5.0.0 - Runtime is Funtime

What's Changed

This release is the continutation of the 4.x goal of aligning closer to NodeJS.  Under the cover much of the changes are nominal, but there have been a few substantial sets of features and improvments to the core platform. Beyond NodeJS alignment, there is also a push to defer to ecosystem standard libraries where possible.

Path

The path implementation in the manifest module ensures POSIX-like behavior for all join/resolve/etc. operations.  The challenge was to remember to pull in the proper module, and some of the slight differences beteween the module and what a full path implementation looked like.  This release sees the shift to asking the user to import path from the node standard library, and the framework will swap it out behind the scenes.  This ensures the code looks and feels like it should, while ensuring consistency at the platform level.

Runtime

Previously known as Base, this represented the foundation for which all modules build apon.  Base unfortunately wasn't a clear name for what it should contain, and ended up being a general catch-all for various functionalities.  Going forward, Runtime will act as the layer between the operating system and the various modules.  The goal is to have this module become as small as possible, specifically as NodeJS evolves.  In addition to general renames, quite a few functions were moved closer to where they belong, and a handful of functionality was removed as it was no longer needed. 

Model + Blob - Asset

Moving towards standard behavior, the Asset service acted as a thin wrapper around the Model + Stream services.  Essentialy it was just NodeJS streams + metadata.  That recipe is basically what blobs are.  By moving the Model contract from streams to blobs, the Asset service was no longer necessary.  This work also touched upload, as upload is no longer tied to assets, but just a standard blob/file upload interceptor that is usable by the Rest framework.  Blobs are the new stanard for streams + metadata, and have reduced the coupling between various modules.

Test

The test framework was given an internal overhaul, allowing for some neat new features, and also shoring up quite a few bugs in the process.  The primary new feature is the ability to tag tests via decorators, and to target or exclude certain tests during invocation.  This is helpful for tests that shouldn't be run in CI/CD for example.  Outstanding bugs were also fixed in the unresolved promise detection.  

As an extra feature, suites are now randomized as well when executing.  This helps to normalize resource usage (and speed up processes), as well as uncover any bugs that may have been hidden due to specific ordering of test runs.

Tooling (Compiler + VSCode)

Additional love and attention has been provided to the compiler, resolving a few outstanding bugs, specifically where a kill -1 was able to be invoked (this kills all userland processes).  Additional love was given to how many open connections/processes VSCode has at once.  Other fixes included ensuring a workspace reload properly kills the compiler (we were ending up with multiple).

Pack

The final enhancement was to allow for marking certain dependencies as external to the bundle process which allows for them to be installed via npm post bundling.  The docker support already picks this up and will install them.  This helps to minimize the amount of boilerplate needed for the most trivial of configurations.

Whats Next?

The ecosystem is still waiting on ES Decorators being fully ratified (including parameter decorators), including support for standalone functions. Once all the necessary features are available within Typescript, the framework will overhaul decorator behavior, and allow for isolated functions where it make sense (e.g. DI factory functions, standalone rest routes, caching, etc).

4.0.0 - Spring Cleaning

What's Changed

This release is aimed at re-evaluating some of the core assumptions for the base of the framework.

The primary modules impacted in this endeavor are, Manifest, Compiler, Base, Pack, and Terminal. In this new world, Base and Terminal have had their scope's reduced, and some core functionality either removed, or migrated to a location that is closer to its primary use case.

NodeJS Alignment

In addition to shuffling code, there was a big push to lean into native NodeJS functionality when applicable. Some of the native functionality was 1-1 (or nearly so), while other areas required rethinking and reworking. As NodeJS has matured, there is more and more of an opportunity to shift ownership of some common patterns into core NodeJS libraries.

Terminal

In theory the role terminal had been playing was aimed at increasing the usability of the command line interface. The implementation relied on some of complexity of terminals that had issues in a cross-platform environment in addition to being really easy to get to an inconsistent state with SIGKILL occurring in the middle of an execution.

To that end terminal has been removed as a core module, but moved to a more specialized module, that has had a dramatic reduction in scope. This reduction comes in the form of only a few methods being available for rendering (and now constrained and better verified), along with outsourcing all color support to chalkjs. The code is smaller, more focused, more reliable, and now ultimately providing a superior user experience.

Manifest

The manifest structure has been overhauled to provide a more logical grouping of common values, and providing increased clarity of which fields belong together. This clarity, also coupled with some stronger assumptions about how the compiler should be working, allowed for a vast reduction in the number of times we need to produce a novel manifest context, which is the core unit of truth for every compilation/execution.

Beyond the context cleanup, internal work was done on module visiting/painting for determining runtime roles, and prod designations for all modules. This once nuanced and delicate piece of code is now far simplified and far more robust. The goal here is simplification of edge cases, and erring on the side of clarity over flexibility.

Compiler

The compiler has received an incredible amount of tlc in this release. Compiler client/server logic has been completely rewritten to centralize ownership to the compiler and to minimize the amount of information needed to integrate at runtime. Additionally types are now being shared appropriately which helps to mitigate the gap between client/server entirely.

The server has also been updated to control the logic around stop/restart/terminate/kill signals to have a repeatable and reliable method for handling shutdown cleanly and being able to recover cleanly when a process is killed immediately. The net result here is a more reliable process that is more resilient against unexpected behaviors.

File Watching

File watching, once again has been updated. Many odd bugs (some were logical, and some were tied to file write behavior of VS Code and how that translates into @parcel/watcher's event stream) have been fixed. Additionally in lieu of watching at each folder to mitigate the number of watchers (but also increased the watch complexity), all watching has been reduced to a single watcher at the repo root. This may have some impact for linux with the inotify limitations, a new field has been added to the `travetto.build` field of the package.json to allow for setting exclusions for watching.

The net-result here is a far more reliable watch experience, and the errant hangs on buffered file writes or git checkouts should be a thing of the past.

VSCode / Tooling Overhaul

In the course of stabilizing and ensuring consistency/repeatability/reliability of all the command line tools and the VS Code functionality, some of the patterns were entirely re-written. There was a non-trivial amount of logic geared towards restarting, checking for healthiness of the underlying tools, capturing odd error states, and the like.

This has all been rewritten to the point of pushing it all of the complexity into the command line tools. The net result here is that every tool invocation within VSCode is far simpler, and no longer has the possibility of runaway process spawning in edge cases. This also provides the benefit that other uses of the tools will now have access to a more robust functional interface allowing all callers to be simpler.

Base

This was a labor of love as base had grown into a hodgepodge of shared utilities that was a little too convenient. This is to say that more focused was needed here along with expectation of logic living where it is being used.

Process execution, environment variable interaction, time utilities, data utilities, stream interactions, shutdown behavior, and startup behavior were all on the chopping block. As stated in the first section, the primary goal here was alignment with standard NodeJS behavior along with migrating code to where it needed to be. The result was a significant change to the core functionality that represented simplicity and clarity, but is definitely not backwards compatible.

Out of all the changes, the Base module rewrite was the primary motivator for the major version release as this impacts far more than just the framework itself.

Pack

Last but not least, Pack was revisited with a unification between the CommonJS and ESM output formats, to allow for a consistent compilation experience, and help to resolve some unexpected bugs that rose during compilation in ESM. Everything is now consistent but this did introduce a behavior change in CommonJS around optional dependencies.

To that end, a new field in the package.json is exposed for defining ignorable modules during pack. This will help to minimize the need of creating a custom rollup entrypoint.

Whats Next?

The ecosystem is still waiting on ES Decorators being fully ratified (including parameter decorators), including support for standalone functions. Once all the necessary features are available within Typescript, the framework will overhaul decorator behavior, and allow for isolated functions where it make sense (e.g. DI factory functions, standalone rest routes, caching, etc).

3.4.0 - Under the Hood

What's Changed

This release is aimed at bringing quality of life improvements to general tooling, with a strong emphasis on compiler orchestration.

Compiler Overhaul

The compiler orchestration has been completely rewritten.  The logic for managing concurrent build requests, watching files, communicating with running servers, etc. has been centralized into a compiler server.  This server owns the lifecycle for compliation, restarting on workspace invalidations, and the like.  The logic that was previously shared between the compiler and the VS Code extension now lives solely within the compiler.  

This overhaul also precipated an audit of all file-based operations, ensuring atomicity of write behavior and also that concurrent requests would not interfere (e.g. a request to clean occurring in the middle of a build cycle).

File Watching 

Previously, the manifest module provided primitives for file watching, and the base module would build upon that for simple use cases.  The main need for watching within the framework, was to watch for changed source files. These file watches were extremely problematic across multiple operating systems, and ensuring everything worked correctly was a challenge.  Even with the introduction of @parcel/watcher, challenges still arose. 

To ensure the primary use case was honored, the compiler server now provides a stream of change events that fire after the file is guaranteed to be compiled, and the manifest has been updated.  This means that filesystem watching is no longer a concern for the framework, but listening to compiler changes is.  The main benefit here is that the watch logic can be replaced/changed/etc, and it only affects the compiler.

As a note, there is still need a for watching files in the email-compiler module, and so a simple integration with parcel's watcher makes quick work of it.  

Config

To the outside, there is nothing that has changed here.  The biggest difference is now a clearer contract for prioritization of various config sources, as well as the ability to support multiple files. This has a direct benefit for the mono-repo setup. but provides a clear contract of what happens when the same file shows up in multiple folders.  This follows the resource provider pattern of, mono-repo >> module >> resource paths.  

General Simplifications

As with every release, one of the underlying goals is to align the codebase with standards within the NodeJS ecosystem, as well as incoporating changes to the built-in APIs (e.g. the switch to fetch).  This release sees more and more internal code removed in favor of standards or simpler solutions.  The net result is a smaller, simpler code base that follows closer and closer with best practices.  

The beauty of this process is that the public API shape is unchanged, which allows for such an invasive change to register as a minor release.  

Whats Next?

The ecosystem is still waiting on ES Decorators being fully ratified (including parameter decorators), including support for standalone functions.  Once all the necessary features are available within Typescript, the framework will overhaul decorator behavior, and allow for isolated functions where it make sense (e.g. DI factory functions, standlone rest routes, caching, etc).

3.2.0 - You're an All-Star

What's Changed

The hits keep coming and they don't stop coming.  Hot on the heels of the 3.1.0 release, the 3.2.0 release has landed, with a another round of substantive improvements.

Email Templating Overhaul

The email templating system has been completely overhauled. What was previously the Email Template module has now been broken into two sub modules:

• Inky Module - This module now hosts the JSX components (no more html files) used for generating the Inky email content.  This module has zero dependencies on the compilation process, and so can safely be included as a non-dev dependency without fear of affecting the final output size. 
  
  The biggest change here is that templates are now compilable (and checkable) for errors, etc.  This was the last major piece of the framework that wasn't based in Typescript, and now that has gone away.
  
  
• Compiler Module - The compilation process is now completely decoupled from the underlying templating framework.  This allows for all the compilation dependencies to be safely isolated as dev dependencies, but also has the added benefit of cleanly allowing future email templating frameworks to be used as well.
  
  In addition to the structural change all the CLI tooling has been updated for emails, including the abillity to trigger email sending from the command line for testing purposes.  Previously all the commands relied on the resource key of the email template, which was confusing to say the least.  Now that all emails are source code, the file path is all that is needed for execution.

Rest Client Support (separate from Openapi Generator)

Until recently, the only way to generate client code was using a custom Openapi code generator and leveraging the openapi.yaml generation provided by the Openapi module.  While useful, and providing a huge array of output formats, left a lot to be desired in the performance arena.  This release sees the introduction of a new module, Rest Client, that enables fast, clean, and simplified client generations specifically for Typescript ecosystems. 

• Tighter Integration with Code Change Detection - The main goal here is to piggyback on the class/file change detection that already exists, and leverage it to produce a minimal client.  
• Better Usage of Native Typescript Primitives -  This provides the ability to have full control over the output versus depending on whatever design decisions the Openapi Generator codebase has.  This means tighter integation on Typescript features, and less bulky generation processes.  
• Optimized Output - The output now produces far fewer files, adds zero additional file watches, and responds much faster to changes in the rest/model schemas.
• Multiple Outputs with a Single Project - Additionally, the code generation runs in process with the containing code, and is able to produce multiple outputs with a single process.  This opens the door for generating node-based clients, as well as web-based clients simultaneously.
• Initial Supported Formats -
• Fetch + Node - A node-based fetch-backed implementation
• Fetch + Web - A web-based fetch-backed implementation
• Angular 15+ - An angular-backed implementation

Native Fetch

Throughout the framework, there has been a perpetual need for making HTTP requests, but generally in the realm of testing and verification.  With the advent of the new RestClient module, there is now a tighter integration with crafting and invoking HTTP requests.  As of Node v18, the Fetch client is baked into the runtime, and is globally accessible (providing parity with Web-based environments).  While still an experimental feature, it is well on its way to achieving stability.  To that end, all references to node-fetch have been removed throughout the framework, and the framework's minimum node version is now bumpted to v18.  

Model + Query Enhancements:

• Inheritance Standardization - Previously Model  and Schema approached the concept of base classes and differentiation slightly differently, even though Model depended on Schema.  This latest round of changes has standardized these patterns providing consistent understanding of how that is modeled within the framework.
• Atomic Update Support - Within the Model query module, there is now an atomic update operation, that relies on a query to match before the update is applied.  This is very useful for optimistic locking based on a field value (e.g. a version key).   
• Existance Checks and Empty Arrays - The Model query module now supports the ability to filter elements based on whether or not an array is empty or not.  Some model providers do not store empty arrays where others do.  This helps to standardize the behavior, treating empty arrays as not existing.
• General Fixups - File, Memory and S3 all received various bug fixes related to stability and standardization of behavior.

Auth Cleanup

• Auth JWT Session-Like Support - The Auth JWT module now allows for auto regenerating a new token depending on how close the token is to expiration.  When combined with the cookie encoder, this provides a mode in which the user will continue to renew the token, providing a semblance of the session timeout/renewal.
• Removal of Auth Rest Context - This functionality is now basked into Auth-Rest

Miscellaneous Enhancements

• Upgraded all dependencies to latest versions
• Custom Rollup Support - Allow for complete control of the rollup behavior, providing similar functionality to that of specifying/controlling the entire docker file.
• Standardization of all Query-based complex types (e.g. schema-backed objects mapped to query params) - We now have consistency between OpenAPI, Rest, and Rest-Client on how these types will be handled, and how prefixing of values will apply.

3.1.0 - Momentum

Hot on the heels of the 3.0.0 major rewrite come some major changes, along with a whole host of quality of life improvements.

What has changed?

Major/Breaking Changes

• Upgrade Typescript to 5.0.  This is part of the ongoing progress and preparation for ES Decorators rolling out for every touch point (waiting on Parameters)
• CLI / App Overhaul.  This is a breaking changes that most people will not notice.  The TL;DR is that @travetto/app is no more, and it's functionality has been merged into @travetto/cli.  This means that instead of making custom @App entry points, you would create a new CLI target using the @CliCommand decorator.  This also means that custom CLI commands creation has been simplified as the module now handles binding cli command flags to the class with the full support of the @travetto/schema functionality.  This also applies to the main command args.  The end result is that the sum of both modules is now greater than their individual parts.  
• CLI / Repo Tooling Certain commands built into the @travetto/cli module were monorepo specific and the naming did not reflect that.  These commands (`exec`, `version`, `list`) have all been moved to the @travetto/repo module and are now prefixed with `repo:`, (`repo:exec`, `repo:version`, `repo:list`).   Additionally, the Cli is now invokable from sub folders where a package.json is not available. 
• Compiler Cleanup. The new compiler introduced in 3.0.0 is fantastic, but ran into some scenarios in which some processes were continuing to run, even after the parent compiler process exited. In conjunction with stability changes, the compiler was also modified to expose progress via an IPC channel if desired.  This allows external tools to better integrate with the compiler as well.  Additionally, the terminal package received certain enhancements that better deal with narrow terminals, as well as scenarios that CTRL+C was not working properly.
• VSCode Upgrades.  The plugin has been upgraded in line with all the above changes (App/Cli and Compiler upgrades).  This should result in a better experience, along with more control and feedback on asynch processes.
• Manifest/Pack isolation. Pack now produces less files, and the manifest has been enhanced to contain additional context needed.  Package.json files will no longer be read at runtime (only during manifesting).  The result here is a more portable output, and the removal of node_modules in the output format.
• DI Integration with 3rd Party Code. The di module now supports the ability to declare and inject third-party code as dependencies.  There are some limitations with code that is not framework-first, but in general this makes it much easier to integrate libraries and declare injectable components.

Whats Next?

The code will run through a few minor releases for stability, but hopefully this release can sit for a while as this is has been a pretty substantial investment.  That being said, the short list of what's next are still in line with the previous release's future looking goals:

Heap Snapshotting

With the ESM support on the horizon (and the impact it has on unloading modules at runtime), there is real impact on the testing framework performance. With the ongoing work for heap snapshot support in V8/NodeJS, this opens the door for minimizing the performance issues, while dramatically simplifying the testing/loading code.  

ES Decorators

As stated above, the expectation is that the full ES Decorator proposal will land by end of this year.   This should be a pretty minor change within the framework, but will produce much cleaner output with much less dependency on tslib. With the full ES Decorator propsal implemented, the framework will also be able to leverage functions as first class citizens.  This should allow for less code in situtations where the wrapping class was unnecessary.