MongoDB

Overview

The MongoDB stream source works similar to Cinchy's Change Data Capture functionality. The listener subscribes to monitor the change stream of a specific collection in the database of the MongoDB server. Any actions performed on document(s) inside of that collection are picked up by the listener and sent to the queue.

Limitations

In order to use change streams in MongoDB, there are a few requirements your environment must meet.

• The database must be in a replica set or sharded cluster.
• The database must use the WiredTiger storage engine.
• The replica set or sharded cluster must use replica set protocol version 1.

Configuring the Sync

All of the configuration parameters and variables for this real-time sync are the same as those outlined in the Mongo Event, MongoDB Collection, and MongoDB Collection Event Triggered documentation. The only difference is that you must also set up your listener configuration as per the next section.

The Listener Configuration

info

In Cinchy v5.7+, configuring the listener can be done directly in the Connections UI, however for multiple listener requirements you must still add additional configurations in the Listener Config table.

To set up an Stream Source, you must set up a Listener Configuration. The below table describes the parameters and their relevant descriptions.

Topic

The below table can be used to help create your Topic JSON needed to set up a real-time sync.

Parameter	Description	Example
Database	Mandatory. The name of your MongoDB database.	Cinchy
Collection	Mandatory. The name of your MongoDB collection.	Employee
Pipeline Stages	Optional. Specify pipeline stages with filters in MongoDB. An aggregation pipeline can have multiple stages for filtering, grouping, and calculating document values. Learn more about pipeline stages.	See the Example Topic JSON below.

Example Topic JSON

The example config uses a filter to return documents with an ID between 0 and 10,000 AND documents with the location set to Montreal, OR where the operation type is delete.

{
	"database": "cinchy",
	"collection": "employee",
        "changeStreamSettings": {
		"pipelineStages": [
			"{ $match: {
                                       $or: [
                                           { $and: [
                                                { 'fullDocument.id': { $gt: 0, $lt: 10000 } },
                                                { 'fullDocument.location': 'Montreal' } ]
                                           },
                                           { 'operationType': 'delete' }
                                      ]
                                } }"
		]
	}
}

Connection Attributes

The below table can be used to help create your Connection Attributes JSON needed to set up a real-time sync.

Parameter	Description	Example
connectionString	Mandatory. Your MongoDB connection string.	mongodb://localhost:9877

{
	"connectionString": "mongodb://localhost:9877"