Changelog
Unreleased
0.7.0 - 2021-12-29
Added
Add documentation in the style of Read the Docs.
Exception classes in
errors.py
, deriving fromDataPipelinesError
base exception class.Unit tests to massively improve code coverage.
--version
flag to dp command.Add
dp prepare-env
command that prepares local environment for standalone dbt (right now, it only generates and savesprofiles.yml
in$HOME/.dbt
).
Changed
dp compile
:--env
option has a default value:base
,--datahub
is changed to--datahub-gms-uri
,--repository
is changed to--docker-repository-uri
.
dp deploy
’s--docker-push
is not a flag anymore and requires a Docker repository URI parameter;--repository
got removed then.dp run
anddp test
rundp compile
before actual dbt command.Functions raise exceptions instead of exiting using
sys.exit(1)
;cli.cli()
entrypoint is expecting exception and exits only there.dp deploy
raises an exception if there is no Docker image to push orbuild/config/dag
directory does not exist.Rename
gcp
togcs
in requirements (now one should runpip install data-pipelines-cli[gcs]
).
0.6.0 - 2021-12-16
Modified
dp saves generated
profiles.yml
in eitherbuild/local
orbuild/env_execution
directories. dbt gets executed withenv_execution
as the target.
0.5.1 - 2021-12-14
Fixed
_dbt_compile
is no longer removing replaced<IMAGE_TAG>
.
0.5.0 - 2021-12-14
Added
echo_warning
function prints warning messages in yellow/orange color.
Modified
Docker image gets built at the end of
compile
command.dbt-related commands do not fail if no
$HOME/.dp.yml
exists (e.g.,dp run
).
Removed
Dropped
dbt-airflow-manifest-parser
dependency.
0.4.0 - 2021-12-13
Added
dp run
anddp test
commands.dp clean
command for removingbuild
andtarget
directories.File synchronization tests for Google Cloud Storage using
gcp-storage-emulator
.Read vars from config files (
$HOME/.dp.yml
,config/$ENV/dbt.yml
) and pass todbt
.
Modified
profiles.yml
gets generated and saved inbuild
directory indp compile
, instead of relying on a local one in the main project directory.dp dbt <command>
generatesprofiles.yml
inbuild
directory by default.dp init
is expectingconfig_path
argument to download config template with the help of thecopier
and save it in$HOME/.dp.yml
.dp template list
is renamed asdp template-list
.dp create
allows for providing extra argument calledtemplate-path
, being either name of one of templates defined in.dp.yml
config file or direct link to Git repository.
Removed
Support for manually created
profiles.yml
in main project directory.dp template new
command.username
field from$HOME/.dp.yml
file.
0.3.0 - 2021-12-06
Run
dbt deps
alongside rest ofdbt
commands indp compile
0.2.0 - 2021-12-03
Add support for GCP and S3 syncing in
dp deploy
0.1.2 - 2021-12-02
Fix: do not use styled
click.secho
for Docker push response, as it may not be astr
0.1.1 - 2021-12-01
Fix Docker SDK for Python’s bug related to tagging, which prevented Docker from pushing images.
0.1.0 - 2021-12-01
Added
Draft of
dp init
,dp create
,dp template new
,dp template list
anddp dbt
Draft of
dp compile
anddp deploy