Cinchy onKubernetes: You can now deploy Cinchy v5 on the Kubernetes system. Kubernetes is an open-source system that manages and automates the full lifecycle of container-based applications. You now have the ability to deploy Cinchy v5 on Kubernetes, and with it comes a myriad of features that help to simplify your deployment and enhance your scaling. Kubernetes can maximize your container capacity and easily scale up/down with your current operations.
Fluent Bit,OpenSearch: Available to those who deploy on Kubernetes, Fluent Bit collects logs which are then displayed through the OpenSearch visual dashboard, for all pods that write to stdout. This streamlines your search for information by putting the control into your hands and compiling your logs in one easy to access place—you can now easily write a query against all of your logs, in all of your environments. You will have access to a default configuration out of the box, but you can also customize your dashboards as well.
GrafanaandPrometheus: With the Kubernetes addition, you now have access to Prometheus to collect your metrics. Prometheus records real-time metrics in a time series database used for event monitoring and alerting. You can then create custom dashboards through Grafana to display your data in an easy to use visual that makes reporting on your metrics easy, and even set up push alerts based on your custom needs.
PostgreSQL: You now have the option to deploy Cinchy on PostgreSQL, an open source alternative to the Microsoft SQL Server that can save you the cost of licensing fees. It's standards-compliant, reliable, highly programmable, and allows for concurrency. Utilizing this deployment makes Cinchy more affordable and scalable. We recommend Amazon Aurora for AWS users.
Kafka: Kafka is an open-source event streaming platform. This is designed to act as the middleware that allows for messaging between components through a queuing mechanism.
Redis: Redis is currently being used to facilitate a distributed lock using RedLock, which guarantees lock synchronizations across Cinchy instances. It's also a storage location for the execution output when running batch data syncs.
All components have been transitioned from Log4Net to Serilog.
The BuildIdentifier property from the appsettings.json will now appear in the healthcheck endpoint at the root level of the JSON payload, with a key of buildidentifier.
Added the ability to ingest S3 data sources for delimited or parquet files.
Improved performance of the Connection UI when there is a large (250+) number of columns/mappings.
Optimization of bulk UPSERT performance.
Added the following UI optimizations for handling large tables: default views set to collapsed, page size limited to 1k records, and added a button for getting the row count. Added the ability for Connections to read/write error files to S3 when an S3 bucket is specified.
Added accessibility fixes.
We've added support for PUT, PATCH, and DELETE in UDF Extensions in addition to GET/POST.
You are able to override a Kafka topic within the appsettings.json for Connections, the Worker, and the Event Listener.
We've added support for the INSERT INTO SELECT statement, which copies data from one table and inserts it into another table. Click here for more information on INSERT INTO SELECT.
When using PostGres, the SELECT @cinchy_row_id function will fail in queries. Instead, use the OUTPUT clause with INSERT, UPDATE, DELETE. Click here for more information on OUTPUT.
We've added support for the TRUNCATE TABLE statement, which removes all rows from a table, without logging the individual row deletions. Click here for more information on TRUNCATE TABLE. (Please note that we don't support the "With Partitions" argument.)