Datastores
Governing Data Plane Access: Integrating RBAC and Sharing
In modern data environments, security is not just about who can log in to an application (Control Plane), but specifically what data they can touch during computation (Data Plane). Syntasa’s Data Plane Access Control feature bridges this gap by ensuring that the permissions you define in the…
Syntasa Data Plane Access Control: AWS & GCP
In a modern data ecosystem, security is defined by the Principle of Least Privilege. Syntasa’s Data Plane Access Control is a robust security framework designed to ensure that compute resources—such as Spark clusters and JupyterLab notebooks—have access only to the specific data they need to…
Event Stores Overview
An Event Store serves as a comprehensive repository comprising databases and file directories utilized by Syntasa applications to store and manage data generated through various processes. The primary purpose of an Event Store is to facilitate the efficient storage and retrieval of data, enabling…
Getting Started with the Datastores
Syntasa writes all the data to specified folders and databases within the environment where the data is processed. Datastores serve as the storage and retrieval points for this information. Syntasa applications utilize three types of data stores: Event Store, ID Store, and Intelligence Hub. Among…
Creating and Editing an Event Store
This article will cover the following topics:
Event Stores Datasets
When an event store is used by any app, a database is created with its tables depending on the type of process we have used in the application. The database for development and production are created separately. If you want to have a look at these tables and their data, you can check it within the…
Datastores Dependencies
If you're using a single event store across multiple apps and need to identify which apps are utilizing it, we've got you covered! Simply navigate to the 'Dependencies' tab within the Datastore page. Here, you'll find a comprehensive list of all the apps and/or DataLayers that are currently making…
ID Stores
The Syntasa platform includes a Type of Datastore called ID Store, which consists of databases and file directories for Syntasa app data storage. ID stores serve as a useful way to organize your apps on the backend and determine where your data is stored. They also help distinguish between your…
Intelligence Hub
An intelligence hub is a NoSQL database storage location (i.e. Big Table, Hbase, DynamoDB). The main purpose of using an intelligence hub is to post Composer model customer attributes into a centralized location.
External Datasets - An Overview
In the Syntasa application, when any process is executed, the transformed data (output) is written into a dataset that resides as a Hive table within the Syntasa-managed environment. This behavior was consistent across all application processes prior to the 9.0 release. Users did not have the…
Create External Dataset
Creating a reference to an external dataset in Syntasa is a simple and guided process. The Create Dataset screen allows you to define a dataset that points to a Hive table located outside the Syntasa-managed environment, such as cloud object storage. Once created, this external dataset can be used…
Choose an External Dataset for an App Process
Once an external dataset is created, it can be easily used as the output destination for an application process in Syntasa. This allows the transformed data generated by a process (for example, a Spark processor) to be written directly to an external location, based on the workflow configuration.