data_pipelines_cli.cli_commands package

Subpackages

Submodules

data_pipelines_cli.cli_commands.clean module

clean() None[source]

Delete local working directories.

data_pipelines_cli.cli_commands.compile module

compile_project(env: str, docker_tag: Optional[str] = None, docker_build: bool = False, docker_build_args: Optional[Dict[str, str]] = None) None[source]

Create local working directories and build artifacts.

Parameters
  • env (str) – Name of the environment

  • docker_tag (Optional[str]) – Image tag of a Docker image to create

  • docker_build (bool) – Whether to build a Docker image

  • bi_build – Whether to generate a BI codes

Raises

DataPipelinesError

replace_image_settings(image_tag: str) None[source]

data_pipelines_cli.cli_commands.create module

data_pipelines_cli.cli_commands.deploy module

class DeployCommand(env: str, docker_push: bool, dags_path: Optional[str], provider_kwargs_dict: Optional[Dict[str, Any]], datahub_ingest: bool, bi_git_key_path: str, auth_token: Optional[str])[source]

Bases: object

A class used to push and deploy the project to the remote machine.

auth_token: Optional[str]

Authorization OIDC ID token for a service account to communication with Airbyte instance

bi_git_key_path: str
blob_address_path: str

URI of the cloud storage to send build artifacts to

datahub_ingest: bool

Whether to ingest DataHub metadata

deploy() None[source]

Push and deploy the project to the remote machine.

Raises
docker_args: Optional[data_pipelines_cli.data_structures.DockerArgs]

Arguments required by the Docker to make a push to the repository. If set to None, deploy() will not make a push

env: str
provider_kwargs_dict: Dict[str, Any]

Dictionary of arguments required by a specific cloud storage provider, e.g. path to a token, username, password, etc.

data_pipelines_cli.cli_commands.docs module

docs(env: str, port: int) None[source]

Generate and serve dbt documentation.

Parameters
  • env (str) – Name of the environment

  • port (int) – Port to serve dbt documentation on.

data_pipelines_cli.cli_commands.init module

data_pipelines_cli.cli_commands.prepare_env module

prepare_env(env: str) None[source]

Prepare local environment for use with dbt-related applications.

Prepare local environment for use with applications expecting a “traditional” dbt structure, such as plugins to VS Code. If in doubt, use dp run and dp test instead.

Parameters

env (str) – Name of the environment

data_pipelines_cli.cli_commands.publish module

create_package() pathlib.Path[source]

Create a dbt package out of the built project.

Raises

DataPipelinesError – There is no model in ‘manifest.json’ file.

publish_package(package_path: pathlib.Path, key_path: str, env: str) None[source]

data_pipelines_cli.cli_commands.run module

run(env: str) None[source]

Run the project on the local machine.

Parameters

env (str) – Name of the environment

data_pipelines_cli.cli_commands.seed module

seed(env: str) None[source]

Run the project on the local machine.

Parameters

env (str) – Name of the environment

data_pipelines_cli.cli_commands.template module

list_templates() None[source]

Print a list of all templates saved in the config file.

data_pipelines_cli.cli_commands.test module

test(env: str) None[source]

Run tests of the project on the local machine.

Parameters

env (str) – Name of the environment

data_pipelines_cli.cli_commands.update module