Product Concepts

Welcome to the Grouparoo Dictionary! This is the collection of terms we use throughout the Grouparoo application to help you sync customer data between your source(s) of truth and business applications.

Table of Contents

• Aggregation Methods
• App
• API Key
• App Refresh Query
• Application Database
• Batch Generation
• Columnar Source
• Destination
• Filter
• Group
• High Watermark
• Mapping
• Model
• Plugin
• Primary Key
• Record
• Record Property
• Property
• Query Source
• Run
• Schedule
• Source
• Sync Mode
• Table Source

Aggregation Methods

Aggregation methods define how we combine the data. When adding multiple properties from a Source, the default aggregation is exact. This means that the imported Grouparoo Record will retain the exact value of each row for that property.

When adding properties individually, one of the required options to choose is the aggregation method.

Not all source types are compatible with all aggregation methods, but most are compatible with the following standard list:

• exact: use the value directly
• average: take the average
• count: count the occurrences
• sum: add it up
• min: find the smallest value
• max: find the largest value
• most recent value: use the value of the newest record
• least recent value: use the value of the oldest record

App

An App in Grouparoo represents a integration, or a connection to some service. Every Source and Destination requires an App. See this page for a full list of available integrations.

API Key

Grouparoo uses API Keys to allow for programmatic access to API provided by our Grouparoo instance. Each API Key can have different read and write permissions to various sections of thr Grouparoo API.

You can learn more about how to use and generate API Keys from the REST API documentation.

App Refresh Query

An App Refresh Query can be used to direct Grouparoo to a particular place to check for data. This may looking for new data in a column of one of your sources, or it may refer to a table that you do not use as a source. If the Query you configure returns a value different from what you had stored, Grouparoo will enqueue all Schedules(#schedule) on all Sources(#source) that have refresh enabled.

Application Database

The Application Database, sometimes referred to as the Grouparoo database, is the database created and used by the Grouparoo application. This is not to be confused with any database source that in which you store customer data.

Batch Generation

Some versions of the generate command support creating multiple config files from a single command. This is what we call Batch Generation.

Columnar Source

A Columnar Source is a specific type of Source that imports data from a column-based mechanism, like a database. Sources of this type share most of their behavior in common.

Destination

Destinations are where you want Grouparoo Records and Groups to be sent. Often Destinations are customer marketing tools such as email marketing or push notification services, though databases and data warehouses can also function at Destinations.

When you set up a Destination in Grouparoo, you will choose which Groups as well as which Record Properties you will send to each Destination. You will have control and visibility over what data gets sent to each Destination.

Grouparoo will keep the Record and Group data in your Destinations up-to-date in real time.

See our Integrations page for an up-to-date list of our supported Destinations. We are always adding support for more Destinations. If you have a request, please let us know!

Filter

Filters are a series of rules used to filter data into some desired subset. Sometimes you may want to specifically include or exclued rows that meet a criteria. This is where Filters come in.

Each filter rule has a key and operation. The key is a property name, and the operation is a comparison operator.

The following operations are available to use as Grouparoo filters. Each operation has a shortened version that is used in the configuration files:

• equals: "eq"
• does not equal: "ne"
• does not exist: "notExists"
• greater than: "gt"
• greater than or equal to: "gte"
• less than: "lt"
• less than or equal to: "lte"
• contains: "substring"
• does not contain: "notSubstring"
• in a given set: "in"

Group

Groups are a collection of Records that you can define based on rules. These rules (or filters) are built to be easy-to-understand by non-technical users.

Let's say we wanted to make a Group of high-value customers who recently abandoned the checkout process. Here is an example of the rules to define this Group:

• [Total Customer Value] [is greater than] [$50]
• [Abandoned Cart] [in the last] [3 days]

With these rules, you have a Group consisting of every user who has purchased more than $50 and recently abandoned their cart. You can then synchronize that Group in real-time to all of your Destinations to then communicate with them.

High Watermark

A High Watermark is a value used for comparison during Runs that are scheduled. Grouparoo will only import items where the High Watermark column is greater than the largest previously imported High Watermark value. For example, let's say you use updated_at as your High Watermark. Grouparoo will run an import and Record the highest value from that column (presumably from the most recently updated Record). The next time the import runs, Grouparoo will only consider records where updated_at is greater than the previous High Watermark value. And it doesn't have to be a date, it just has to be comparable. Another example would be a numeric field, like version, in which the value is incremented when changes are made to the Record in the Source database.

Mapping

When importing data from a Source, Grouparoo needs to know how to relate the data its importing to the Records already in Grouparoo. This is what we call a Mapping. It operates like a foreign key in a database.

Consider that you have a Property called email, while your Source has an attribute called emailAddress, which is a unique field in that Source. You could map emailAddress from your source to email in Grouparoo. This tells Grouparoo that when anything changes on the Source, it knows which Record to update.

Every Source needs a Mapping, as Grouparoo must know how to associate imported information.

Model

A Model defines the shape of your Records. Each Model has its own Sources, Records, and Properties. Together, this allows you to do things like create generate different types of Records such as Leads, Customers, Users, and Visitors that contain different Properties. For example, a Visitor Model may have a number_of_sessions Property, while a Customer may have a ltv Property. Models allow you to only add the Properties you need to keep your Records clean.

Plugin

The core Grouparoo application is the engine that drives syncing data from Sources to Destinations. The application relies on plugins to provide you with only the functionality you require to sync data between your preferred services.

Being that Grouparoo is a Node.js application, the plugins are available as JavaScript packages through NPM. The majority of plugins are open-source and available for free, though we do have also have paid plugins.

Learn more about existing plugins, or develop your own Plugin.

Primary Key

The Primary Key for a Model is the Property that serves as the unique identifier Mapping for the Primary Source in that Model.

For example, in a User Model with a Postgres table import as the Primary Source, the Primary Key may be the userId Property mapped to the id column in the Postgres users table.

Record

Records represent individual people. For your specific company or organization, that may mean:

• Leads
• Customers
• Users
• Visitors to your site or App

Record Property

Record Properties are data associated with a Record. You can define Record Properties based on data that exists in your data Sources.

Record Properties are given a specific type, letting us know if they are a number, date, or string. Having a type helps Grouparoo know how to display this data and send it to Destinations.

Record Properties can also be Unique or Not Unique. Defining Uniqueness is needed in order to create and merge Records from various sources without creating duplicate Records.

Common examples of Unique Record Properties are:

• Email Address
• User ID

Common examples of Not Unique Record Properties are:

• First Name
• Last Name
• Date of Last Purchase

Property

Properties define how a Record Property gets pulled from a Source. Every Source might be structured in slightly different ways, so Grouparoo Plugins make it easy for you to define Properties.

For example, you might define a Property called first_name which you are pulling from your product database via the @grouparoo/postgres Plugin.

One Property in the Model is set as the Primary Key.

Query Source

A Query Source is an implementation of a Columnar Source that enables you to write custom queries to extract information from your database. This is a more flexible version when compared with a Table Source.

Run

A Run is a series of imports and/or exports. Runs are triggered by actions in Grouparoo, such as creating a new Group. Runs can also be triggered by Schedules.

Schedule

Schedules define a frequency that Grouparoo should check your Sources for new data, creating a Run. Whenever possible, Schedules use High watermarks to scan for only new and updated data in your Data Warehouse. Schedules can also be triggered manually or via the API.

Though technically separate objects, Schedules are often used in conjunction with Sources.

Source

Sources are the objects within Apps that Grouparoo will connect with to pull in customer data. A Source can be anything that contains customer data such as data warehouses, databases, and CRMs.

When you add a new Source, you will define how that Source connects with the Records that already exist in Grouparoo through a Unique Record Property.

Sources belong to Models, and each Model has a Primary Source and, optionally, multiple Secondary Sources. The Primary Source is the data source where the bulk of the Record data will be imported for the Model, and Secondary Sources serve additional data. The Mapping for the Primary Source is set via the Model's Primary Key.

See our Integrations page for an up-to-date list of our supported Sources. We are always adding support for more Sources. If you have a request, please let us know!

Sync Mode

When creating a Grouparoo Destination, you will be asked to choose a Sync Mode. A Sync Mode tells the Grouparoo application how to create and delete Records on this Destination. The types of Sync Modes are:

• Sync: Add, update and remove records from the Destination as needed.
• Additive: Add new records and update those that already exist in the Destination, but never remove anyone.
• Enrich: Update records that already exist in the Destination, enriching them with additional data and Group membership information. No records are added or removed.

You can learn more about in our blog posts introducing Sync Modes and how to use Sync Modes

Table Source

A Query Source is an implementation of a Columnar Source that makes it easy to import data from a database by using aggregation methods to extract data from a column in your database. This is a more rigid version when compared with a Query Source, but often enables you to move faster when you either want to import data directly or perform minor operations or transformations of that data.