data_pipelines_cli.cli_commands package
Subpackages
Submodules
data_pipelines_cli.cli_commands.clean module
data_pipelines_cli.cli_commands.compile module
- compile_project(env: str, docker_tag: Optional[str] = None, docker_build: bool = False, docker_build_args: Optional[Dict[str, str]] = None) None [source]
Create local working directories and build artifacts.
- Parameters
env (str) – Name of the environment
docker_tag (Optional[str]) – Image tag of a Docker image to create
docker_build (bool) – Whether to build a Docker image
bi_build – Whether to generate a BI codes
- Raises
data_pipelines_cli.cli_commands.create module
data_pipelines_cli.cli_commands.deploy module
- class DeployCommand(env: str, docker_push: bool, dags_path: Optional[str], provider_kwargs_dict: Optional[Dict[str, Any]], datahub_ingest: bool, bi_git_key_path: str, auth_token: Optional[str])[source]
Bases:
object
A class used to push and deploy the project to the remote machine.
- auth_token: Optional[str]
Authorization OIDC ID token for a service account to communication with Airbyte instance
- bi_git_key_path: str
- blob_address_path: str
URI of the cloud storage to send build artifacts to
- datahub_ingest: bool
Whether to ingest DataHub metadata
- deploy() None [source]
Push and deploy the project to the remote machine.
- Raises
DependencyNotInstalledError – DataHub or Docker not installed
DataPipelinesError – Error while pushing Docker image
- docker_args: Optional[data_pipelines_cli.data_structures.DockerArgs]
Arguments required by the Docker to make a push to the repository. If set to None,
deploy()
will not make a push
- env: str
- provider_kwargs_dict: Dict[str, Any]
Dictionary of arguments required by a specific cloud storage provider, e.g. path to a token, username, password, etc.
data_pipelines_cli.cli_commands.docs module
data_pipelines_cli.cli_commands.init module
data_pipelines_cli.cli_commands.prepare_env module
- prepare_env(env: str) None [source]
Prepare local environment for use with dbt-related applications.
Prepare local environment for use with applications expecting a “traditional” dbt structure, such as plugins to VS Code. If in doubt, use
dp run
anddp test
instead.- Parameters
env (str) – Name of the environment
data_pipelines_cli.cli_commands.publish module
- create_package() pathlib.Path [source]
Create a dbt package out of the built project.
- Raises
DataPipelinesError – There is no model in ‘manifest.json’ file.