Dataherald is built on a modular architecture and provides standaridized, extensible interfaces for anyone to replace any of these modules with their own implementation. This section outlines these modules.
The system is built on the following modules, and you can implement your own and replace the default implementation from the
.env file. In many instances the codebase already has multiple implementations which can be selected. We encourage the community to build their own modules and submit them for inclusion in the codebase.
The following are the core modules which make up the Dataherald engine.
Context Store: Which stores the relevant business and data context used in few-shot prompting.
Text-to-SQL Engine: The module that translates the Natural Language question to SQL
API Server: The server that exposes the API
Evaluator: The module that assigns a confidence score to the generated SQL
Vector Store: The vector store which stores application embeddings.
Database Integration: Used to store and persist application data.
The following diagram illustrates the overall system architecture.