5.10 Release Notes
Cinchy version 5.10 was released on March 28th, 2024.
Upgrade guides
After upgrading to version 5.10 all users must log out and back in to your Cinchy environment in order for the changes to properly take effect.
Personal Access Tokens
[1] There was an issue affecting Personal Access Tokens (PATs) generated in Cinchy wherein tokens created from v5.7 onwards were incompatible with subsequent versions of the platform. This issue has been resolved, however please note that:
- Any tokens created on versions 5.7.x, 5.8.x, and 5.9.x will need to be regenerated.
- "401 Unauthorized" errors may indicate the need to regenerate the token.
- PATs created before 5.7.x and from 5.10 onwards are unaffected.
General
- We have updated our third-party libraries. [2]
- Various security fixes.
New Capabilities
- We have added the Compress JSON parameter to the Query Builder UI and [Saved Queries] table. JSON compression can:
- Help to reduce the amount of time it takes to query and process data
- Reduce the amount of bandwidth needed to transfer data. This can be especially beneficial for applications that require frequent data updates, such as web applications.
- Reduce the amount of memory needed to store data.
Enhancements
- We have made various enhancements to the Saved Queries table for use cases when your queries are being used as API endpoints. Better management of these queries is possible by way of HTTP methods (GET, POST, PUT, PATCH, DELETE) for distinguishing between types of query operations, Versions for endpoint versioning, and UUIDs for grouping queries. Please review the Queries and Saved Query API pages, as well as the change log, for further details. [3]
- Error messages that pop up in the Connections Experience will provide additional information that will be more useful for troubleshooting.
- To enhance the troubleshooting of Cinchy's Angular SDK, errors will now display additional context. Failed call exceptions will contain more useful errors in the
data.details
property. - The minimum length of a text column created by a CSV import is now 500 characters.
- We have added two new scopes to the [Cinchy].[Integrated Clients] table: read:all and write:all, which can be used to fine-tune your permission sets. These scopes are found in the “Permitted Scopes” column of the table:
-- read:all = clients can read all data.
-- write:all = clients can read and write all data.- Both scopes are automatically assigned to existing integrated clients upon upgrading to v5.10
- Note: All new Cinchy Web API endpoints that use Bearer Tokens or Cookie Authentication Schemes must have at least one of the new scopes assigned. These endpoints are still currently accessible with the old js_scope, however this will be deprecated in a future release. You can update the scopes of existing endpoints in the “Permitted Scopes” column of the [Cinchy].[Integrated Clients] table.
- Note: A user with write entitlements will not be able to write when using a client that only has the read:all scope assigned.
- Note: Clients that receive a 403: Forbidden status code error in the logs should make note of this change as a possible cause, and update the permissions accordingly.
- We have expanded the authentication options available when building a TSQL database connection; including "Active Directory Interactive" in the platform SQL connection string (i.e. the database that hosts the cinchy web/idp application) will now utilize Active Directory Device Code Flow.[4]
- To gain more control over your query creation, we have added a Cancel button to the query builder. The Cancel/Stop button will appear for the duration of running your query; clicking it will abort the active query and return a "Query execution cancelled" message.
Connections
-
Cinchy v5.10 is compatible with the MySql v8.3.0 driver.
-
SQL database read performance logging in Connections now reports a single entry per batch, making the results easier to interpret than the previous fixed-sized intervals (which may not have corresponded directly with batch activity).
-
To improve the user experience of building and running data syncs, we have added a "description" field to the Info section in the Connections experience. This field has a 500 character limit.
-
When selecting columns in a Cinchy Event Broker or Cinchy Table source, four additional system columns have been added: Replaced, Rejection Comments, Changeset, and Change Summary.
-
When configuring or viewing a data sync created by a user with a different permission set as you, you may run into a case where the sync involves tables/queries you do not have access to. Previously, the table/query dropdowns in these cases would appear blank, however they will now populate with the names of those objects. Note that:
- You won’t be able to change the associated schema of a table/query you cannot access. Some fields may appear as disabled (ex: data mappings).
- You can still modify and save other aspects of the sync.
- Event-based syncs can be enabled and run as usual. Batch syncs must be run as a user with the correct permissions. Remember that you can run a job as another user if you have the credentials for that user.
-
Performance and error analysis is easier to accomplish with the addition of logging for Job execution parameters in data syncs. After starting a Batch job, you can navigate to Connections.WebApi logs in OpenSearch on Cinchy v5.0+, or the admin panel on Cinchy IIS, and search for an “Executing Job with parameters” message to see which parameters the job is executing with. (Note that the log location will depend on where you set up your log storage upon deployment.)
Example log:
{"@t":"2024-01-09T21:33:05.5223771Z","@mt":"Executing Job with parameters: {Reconcile Data}; {Degree Of Parallelism}; {Batch Size}; {Retrieval Batch Size}","Reconcile Data":true,"Degree Of Parallelism":2,"Batch Size":4000,"Retrieval Batch Size":3000,"SourceContext":"Cinchy.Connections.WebApi.Services.BatchDataSyncExecutor","ExecutionId":336,"DataSyncContextLogSink":336,"QueuedJobsProcessorId":"4"}
-
Spend less time searching and more time building: You are now able to use the "Models" dropdown field in the Connections UI to quickly select tables scoped within the respective model.
-
When configuring Kafka as a sync destination you can now use use a '@COLUMN' custom formula to enable a dynamic Topic. For further information on this, please review the Kafka Topic documentation.
Bug Fixes
- We have fixed a bug that was causing some data syncs to Cinchy Tables to unnecessarily update multi-select values in the destination. This fix reduces monitoring noise and prevents collaboration log and database size bloat. [5]
- We fixed a bug where using 0 parameters in the ExecuteCQL API would incorrectly modify the API Models.
- We fixed a bug where the Publish Data Change Notifications setting was not being respected during table model loads.
- "Unique constraint" errors will no longer be thrown during parallel batch syncs that contained calculated columns.
- Fixed a bug that was creating duplicate target records when both inserting data in parallel and throwing transition errors that would trigger retries.
- We have addressed the possible out-of-memory errors that could arise during a data sync when "caching linked data values". [6]
- Unnecessary collaboration log bloat will no longer occur due to the presence of parameters (variables) in a Data Sync XML. [7]
- The Connections experience will no longer incorrectly start up if a Redis connection string is not provided. The following error will be thrown to provide troubleshooting assistance: "The Redis connection string is missing. Check your application configuration to ensure this value is provided."
- We fixed a bug where LEFT and RIGHT CQL functions were not working as expected.
- We fixed a bug that was preventing queries with User Defined Functions from executing due to a misalignment between the parser and the application initialization.
- Erased data will be filtered properly again on Left & Right Joined Table References. [8]
- We fixed the following form bugs:
- A bug that prevented new records from being added to multiple child forms in the same view before the parent form was saved.
- A bug that duplicated newly-added records in a child form table if they were edited before the parent form was saved.
- Logging into the Forms application from a direct link in a fresh session resulted in a blank screen. [9]
Change Log
General
- [1] An upgrade to Identity Server 6 introduced additional security regarding PAT tokens. By default, PAT tokens were no longer compatible between Cinchy versions.
- [2] Updated the following third-party libraries:
- Microsoft.Data.SqlClient 5.1.4
- System.IdentityModel.Tokens.Jwt 6.35.0
- Microsoft.AspNetCore.Authentication.JwtBearer 6.0.26
- Snowflake.Data 2.1.5
- Microsoft.AspNetCore.DataProtection.StackExchangeRedis & Microsoft.AspNetCore.Authentication.OpenIdConnect 6.0.26
- AWSSDK.DynamoDBv2 3.7.203.9 -> 3.7.301
- AWSSDK.S3 3.7.205.17 -> 3.7.305.15
- AWSSDK.SecretsManager 3.7.301.6 -> 3.7.302.14
- AWSSDK.SecurityToken 3.7.202.12 -> 3.7.300.40
- AWSSDK.SQS 3.7.200.62 -> 3.7.300.39
- Azure.Storage.Blobs 12.18.0 -> 12.19.1
- Azure.Storage.Blobs.Batch 12.15.0 -> 12.16.1
- Microsoft.VisualStudio.Azure.Containers.Tools.Targets 1.19.5 -> 1.19.6
- MongoDB.Driver 2.22.0 -> 2.23.1
- Oracle.ManagedDataAccess.Core 3.21.120 -> 3.21.130
- Duende.IdentityServer 6.3.5 -> 6.3.8
- MessagePack 2.5.129 -> 2.5.140
- MessagePack.Annotations 2.5.129 -> 2.5.140
- HtmlAgilityPack 1.11.54 -> 1.11.57
- MSTest.TestFramework 3.1.1 -> 3.2.0
- Moq 4.20.69 -> 4.20.70
- WireMock.Net 1.5.39 -> 1.5.46
- coverlet.collector 3.1.2 -> 3.2.0
- Microsoft.AspNetCore.Mvc.NewtonsoftJson 6.0.22 -> 6.0.26
- Microsoft.AspNetCore.SpaServices.Extensions 6.0.22 -> 6.0.26
- Microsoft.AspNetCore.DataProtection.StackExchangeRedis 6.0.22 -> 6.0.26
- Microsoft.Extensions.Hosting.WindowsServices 7.0.0 -> 7.0.1
- System.ServiceProcess.ServiceController 7.0.0 -> 7.0.1 (WindowsServices depends on this package)
- Microsoft.Extensions.Http.Polly 6.0.22 -> 6.0.26
- Microsoft.Extensions.ObjectPool 6.0.22 -> 6.0.26
- Microsoft.Identity.Client 4.56.0 -> 4.59.0
- Microsoft.ML 2.0.1 -> removed
- Microsoft.NET.Test.Sdk 17.7.2 -> 17.8.0
- Serilog 3.0.1 -> 3.1.1
- System.Text.Json 6.0.8 -> 6.0.9
- System.Data.SqlClient 4.8.5 -> 4.8.6
- Refactor of Recursive mechanism to append linked table definitions for reference in column references.
- Complete removal of GraphQL libraries.
- The following assembly names have changed to avoid collision with other processes:
- Agent -> Cinchy.EventListener.Agent
- EventListener.Web -> Cinchy.EventListener.Web
- Cinchy.Ms.Core -> Cinchy.EventListener.Core
Enhancements
- [3] The following columns have been added:
Column Name | Description | Value upon upgrade for existing queries |
---|---|---|
Query Type | The two values possible for this column are Virtual Table and API, where Virtual Table denotes a query best suited for a read-only tabular view of data, useful for sharing reports and queries, and API is one best suited for programmatic access. This parameter is also configurable in the Saved Query UI. | API |
HTTP Method | The API method associated with the query. This parameter is also configurable in the Saved Query UI. The possible values are - GET - POST - PUT - PATCH - DELETE. Note that: - There is no correlation between, or enforcement of, HTTP methods and the query operations. e.g. a query that has an insert statement and a GET HTTP method will run successfully. - If an Http Method is specified, then the Http method used for the request to the API endpoint needs to match. e.g. If a DELETE request is made to an API endpoint where the Http Method for the corresponding query is set to GET, an error will be returned. | Blank |
API Version | This column can be used to version groups of endpoints. | 1.0 |
UUID | A Universally Unique IDentifier. This can be used to link versions of endpoints together. | Blank |
Use Legacy URL Structure | The URL structure of an API endpoint in Cinchy v5.10+ has changed. If this column is set to true then your query API endpoint will continue to use the old structure. - Set to true: /API/[Domain]/[Query Name] . - Set to false: /API/v[API Version]/[Domain]/[Query Name] | True |
CompressJSON | JSON compression can: - Help to reduce the amount of time it takes to query and process data. - Reduce the amount of bandwidth needed to transfer data. This can be especially beneficial for applications that require frequent data updates, such as web applications. - Reduce the amount of memory needed to store data. - Set to false : returns JSON API results in a standard format with an array of objects. - Set to true : returns JSON API results with separate arrays for schema and data. This parameter is also configurable in the Saved Query UI. This value can be overridden at the URL level when making requests by adding ?CompressJSON=true or ?CompressJSON=false to the REST API URL . | True |
- The constraints on REST API URLs have also changed.
- Old constraint: you must have a unique
REST API URL
field. - New constraint: you must have a unique
HTTP Method
+REST API URL
field.
- Old constraint: you must have a unique
- [4] Utilize ActiveDirectoryDeviceCodeFlow when "Active Directory Interactive" is in the SQL connection strings for TSQL database connection in platform.
Bug Fixes
- [5] When comparing multi-select values, data syncs are no longer order-sensitive, or whitespace-sensitive. This avoids the unnecessary updating of such values during data syncs.
- [6] Total memory requirements are now limited via splitting the work into manageable files and consequently merging them.
- [7] Data Sync Parameters (variables) are now compared by name, not by value.
- [8] Shift Erasure filter clause to ON clause for QualifiedJoin Expressions, for left and right joins, on tables where data has been erasured.
- [9] There is a known issue that applications built on Angular using the @cinchy-co/angular-sdk package (as of version 5.1.2) may be unable to access the query parameters passed to the
app.component.ts
file immediately after a login redirect. You can work around this issue by either wrapping this logic in a setTimeout of equivalent function which causes the logic to execute on the next application lifecycle, or by moving the logic out of theapp.component.ts
file and into the initialization logic of a child component which is loaded after the route resolves. Note that prior to version 5.1.2 of @cinchy-co/angular-sdk, the query parameters would be stripped after the logic redirect cycle. If you are using an older version, please contact your Cinchy support team for help in crafting a solution that suits your needs.