API ======================= The Dataherald Engine exposes RESTful APIs that can be used to: * 🔌 Connect to and manage connections to databases * 🔑 Add context to the engine through scanning the databases, adding database level instructions, adding descriptions to tables and columns and adding golden records * 🙋‍♀️ Ask natural language questions from the relational data Our APIs have resource-oriented URL built around standard HTTP response codes and verbs. The core resources are described below. Database Connections ------------------------------ The ``database-connections`` object allows you to define connections to your relational data stores. Related endpoints are: * :doc:`Create database connection ` -- ``POST api/v1/database-connections`` * :doc:`List database connections ` -- ``GET api/v1/database-connections`` * :doc:`Update a database connection ` -- ``PUT api/v1/database-connections/{db_connection_id}`` **Database connection resource example:** .. code-block:: json { "alias": "string", "use_ssh": false, "connection_uri": "string", "path_to_credentials_file": "string", "llm_api_key": "string", "ssh_settings": { "db_name": "string", "host": "string", "username": "string", "password": "string", "remote_host": "string", "remote_db_name": "string", "remote_db_password": "string", "private_key_password": "string", "db_driver": "string" } } Table Descriptions --------------------- The ``table-descriptions`` object is used to add context about the tables and columns in the relational database. These are then used to help the LLM build valid SQL to answer natural language questions. Related endpoints are: * :doc:`Scan table description ` -- ``POST api/v1/table-descriptions/sync-schemas`` * :doc:`Update a table description ` -- ``PATCH api/v1/table-descriptions/{table_description_id}`` * :doc:`List table description ` -- ``GET api/v1/table-descriptions`` * :doc:`Get a description ` -- ``GET api/v1/table-descriptions/{table_description_id}`` * :doc:`Refresh table descriptions ` -- ``GET api/v1/table-descriptions/refresh`` **Table description resource example:** .. code-block:: json { "columns": [{}], "db_connection_id": "string", "description": "string", "examples": [{}], "table_name": "string", "table_schema": "string" } Database Instructions --------------------- The ``database-instructions`` object is used to set constraints on the SQL that is generated by the LLM. These are then used to help the LLM build valid SQL to answer natural language questions based on your business rules. Related endpoints are: * :doc:`Add database instructions ` -- ``POST api/v1/{db_connection_id}/instructions`` * :doc:`List database instructions ` -- ``GET api/v1/{db_connection_id}/instructions`` * :doc:`Update database instructions ` -- ``PUT api/v1/{db_connection_id}/instructions/{instruction_id}`` * :doc:`Delete database instructions ` -- ``DELETE api/v1/{db_connection_id}/instructions/{instruction_id}`` **Instruction resource example:** .. code-block:: json { "db_connection_id": "string", "instruction": "string", } Prompts --------------------- The ``prompt`` object is used to ask questions or pass any natural language text to the LLM. Related endpoints are: * :doc:`Create a prompt ` -- ``POST api/v1/prompts`` * :doc:`List prompts ` -- ``GET api/v1/prompts`` * :doc:`Get a prompt ` -- ``GET api/v1/prompts/{prompt_id}`` **prompt resource example:** .. code-block:: json { "id": "str" "db_connection_id": "str" "text": "str" "created_at": "datetime", "metadata": "dict | None" } SQL generations --------------------- Given a prompt, Dataherald AI agents can generate SQL queries to efficiently answer the question or provide the necessary information. Related endpoints are: * :doc:`create a SQL generation for a given prompt ` -- ``POST api/v1/prompts/{prompt_id}/sql-generations`` * :doc:`create a prompt and a SQL generation ` -- ``POST api/v1/prompts/sql-generations`` * :doc:`list SQL generations ` -- ``GET api/v1/sql-generations`` * :doc:`get a SQL generation ` -- ``GET api/v1/sql-generations/{sql_generation_id}`` * :doc:`get a SQL a csv file of SQL query results ` -- ``GET api/v1/sql-generations/{sql_generation_id}/csv-file`` * :doc:`execute a SQL generation ` -- ``GET api/v1/sql-generations/{sql_generation_id}/execute`` **SQL generation resource example:** .. code-block:: json { "id": "str" "prompt_id": "str" "finetuning_id": "str" "evaluate": "bool" "llm_config": { "llm_name": "str", "api_base": "str" }, "sql": "str" "status": "str" "completed_at": "datetime" "tokens_used": "int" "confidence_score": "float" "error": "str" "created_at": "datetime", "metadata": "dict | None" } NL generations --------------------- For each SQL generation, the LLMs can generate a natural language response based on the SQL query results. Related endpoints are: * :doc:`create a NL generation for a given SQL generation ` -- ``POST api/v1/sql-generations/{sql_generation_id}/nl-generations`` * :doc:`create a SQL generation and a NL generation for a given prompt ` -- ``POST api/v1/prompts/{prompt_id}/sql-generations/nl-generations`` * :doc:`create a prompt and a SQL generation and a NL generation ` -- ``POST api/v1/prompts/sql-generations/nl-generations`` * :doc:`list NL generations ` -- ``GET api/v1/nl-generations`` * :doc:`get a NL generation ` -- ``GET api/v1/nl-generations/{nl_generation_id}`` **Nl generation resource example:** .. code-block:: json { "id": "str" "sql_generation_id": "str" "llm_config": { "llm_name": "str", "api_base": "str" }, "text": "str" "created_at": "datetime", "metadata": "dict | None" } Finetuning jobs --------------------- The ``finetuning`` object is used to finetune the LLM to your data. This is an asynchronous process that uploads your golden records to model provider servers and creates a finetuning job. The finetuned model is going to be used inside an agent for generating SQL queries. Related endpoints are: * :doc:`Finetuning job create ` -- ``POST api/v1/finetunings`` * :doc:`Finetuning job get ` -- ``GET api/v1/finetunings/{finetuning_id}`` * :doc:`Finetuning job cancel ` -- ``POST api/v1/finetunings/{finetuning_id}/cancel`` * :doc:`Finetuning job delete ` -- ``DELETE api/v1/finetunings/{finetuning_id}`` * :doc:`Finetuning job list ` -- ``GET api/v1/finetunings`` **Finetuning resource example:** .. code-block:: json { "id": "finetuing-job-id", "db_connection_id": "database_connection_id", "alias": "model name", "status": "finetuning_job_status", // Possible values: queued, running, succeeded, validating_files, failed, or cancelled "error": "The error message if the job failed", // Optional, default is None "base_llm": { "model_provider": "model_provider_name", // Currently, only 'openai' "model_name": "model_name", // Supported: gpt-3.5-turbo, gpt-4 "model_parameters": { "n_epochs": "int or string", // Optional, default 3 "batch_size": "int or string", // Optional, default 1 "learning_rate_multiplier": "int or string" // Optional, default "auto" } }, "finetuning_file_id": "File ID for finetuning file", "finetuning_job_id": "Finetuning job ID", "model_id": "Model ID after finetuning", "created_at": "datetime", "golden_sqls": "array[ids]", // Default is None, meaning use all golden records "metadata": "dict | None" // Optional, default None } Error Codes --------------------- Certain errors are accompanied by an error code and an explanatory message. These errors trigger an HTTP 400 response code. **DB connection error code response example:** .. code-block:: json { "error_code": "invalid_database_uri_format", "message": "Invalid URI format: foo", "description": null, "detail": { "alias": "foo", "use_ssh": false, "connection_uri": "gdfgdgAABl5e-dfg_-wErFJdFZeVXwnmew_dfg__WU-dfgdfa==" } } .. toctree:: :hidden: api.create_database_connection api.list_database_connections api.update_database_connection api.scan_table_description api.list_table_description api.get_table_description api.refresh_table_description api.update_table_descriptions api.list_query_history api.add_instructions api.list_instructions api.update_instructions api.delete_instructions api.golden_sql api.create_prompt api.list_prompts api.get_prompt api.create_sql_generation api.create_prompt_sql_generation api.list_sql_generations api.get_sql_generation api.get_csv_file api.execute_sql_generation api.create_nl_generation api.create_sql_generation_nl_generation api.create_prompt_sql_generation_nl_generation api.list_nl_generations api.get_nl_generation api.finetuning api.get_finetuning api.cancel_finetuning api.delete_finetuning api.list_finetunings api.error_codes