Michael McGuinness
73 lines
3.0 KiB
Markdown
73 lines
3.0 KiB
Markdown
# Query Orchestration
|
|
|
|
This repository contains the DoczyAI code.
|
|
|
|
Using the following project as a baseline: `https://github.com/golang-standards/project-layout`.
|
|
|
|
## Installation and Usage
|
|
|
|
- Install devbox: `https://www.jetify.com/docs/devbox/installing_devbox`
|
|
- Ubuntu/MacOS: `curl -fsSL https://get.jetify.com/devbox | bash`
|
|
- NixOS: `devbox`
|
|
- Install docker: `https://docs.docker.com/engine/install/`
|
|
- (IF NECESSARY) Ensure user in docker group and docker group is in sudo group
|
|
|
|
```sh
|
|
sudo groupadd docker
|
|
sudo usermod -aG docker $USER
|
|
```
|
|
|
|
- Run `touch .env` to create `.env` file for custom environment variables
|
|
- Run `devbox shell` to enter the environment
|
|
- Run `task fullsuite:ci` to run all tests that ensure the current state
|
|
|
|
To find new packages: `https://search.nixos.org/packages`
|
|
|
|
For regular usage, please use `task` scripts in order run the most appropriate configuration.
|
|
|
|
## Environment Variables
|
|
|
|
When using environment variables, there is a hiearchy that will be respected. (First will override the next)
|
|
|
|
1. Within the devbox.json file, a variable added to "env"
|
|
2. Within the devbox.json file, a variable exported in "init_hook"
|
|
3. A variable added to .env
|
|
|
|
## Testing
|
|
|
|
For testing endtoends such as queries against a database, or interactions with an SQS queue, this repository employs the use of docker testcontainers.
|
|
|
|
`https://testcontainers.com/`
|
|
|
|
This enables tests to be fully self-contained, thus allowing allowing tests to be reliable and replicable.
|
|
|
|
### Execution
|
|
|
|
The simplest way of ensuring the validity of the full project is by running `task fullsuite` when in the devbox shell.
|
|
|
|
This will:
|
|
|
|
1. Generate the latest version of the specs.
|
|
2. Run the linting commands.
|
|
3. Run the unit tests.
|
|
4. Run the endtoend tests.
|
|
|
|
## Naming Conventions
|
|
|
|
**Service** - This term is used internally in order to refer to commands that get deployed as APIs.
|
|
**Runner** - This term is used internally in order to refer to commands that get deployed as consumers of a queue.
|
|
|
|
### Creating a new API
|
|
|
|
1. Add a file named `<api_name>.yml`. The api name is important, as it will be used as a reference throughout the codebase. The api name must be in the format `<functionality>API`. E.g. queryAPI, ClientAPI
|
|
2. Run `task openapi:generate`. This will generate the following:
|
|
a. A location to implement the server-side functions in `./api/<api_name>/`, as well as generated functions, models and swagger docs.
|
|
b. A location with the client-side code in `./pkg/<api_name>/`
|
|
3. Implement the controllers in `./api/<api_name>/`
|
|
4. Create a command in `./cmd/<api_name>/main.go` which creates a new instance from `./internal/api`
|
|
|
|
### Creating a new Runner
|
|
|
|
1. Implement the controller in `./api/<runner_name>/` which implements the `Controller` interface in `./internal/queue`. The runner name must be in the format `<functionality>Runner`. E.g. queryRunner, csvExportRunner
|
|
2. Create a command in `./cmd/<runner_name>/main.go` which creates a new instance from `./internal/queue`
|