Prepare release 0.13.0 (#1172)

* Bump api spec to 0.13.0 Signed-off-by: wslulciuc <[email protected]> * Bump python client and airflow integration to 0.13.0 Signed-off-by: wslulciuc <[email protected]> * continued: Bump python client and airflow integration to 0.13.0 Signed-off-by: wslulciuc <[email protected]> * Bump chart to 0.13.0 Signed-off-by: wslulciuc <[email protected]> * Bump java and spark integration to 0.13.0 Signed-off-by: wslulciuc <[email protected]> * Update CHANGELOG.md for 0.13.0 Signed-off-by: wslulciuc <[email protected]>
MarquezProject · Mar 30, 2021 · 2aa8b92 · 2aa8b92
1 parent 4507b26
commit 2aa8b92
Show file tree

Hide file tree

Showing 18 changed files with 72 additions and 43 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,6 +1,35 @@
 # Changelog
 
-## [Unreleased](https://github.com/MarquezProject/marquez/compare/0.12.2...HEAD)
+## [Unreleased](https://github.com/MarquezProject/marquez/compare/0.13.0...HEAD)
+
+## [0.13.0](https://github.com/MarquezProject/marquez/compare/0.12.2...0.13.0) - 2021-03-30
+
+### Added
+
+* Secret support for helm chart [@KevinMellott91](https://github.com/KevinMellott91)
+* **New** `seed` cmd to populate `marquez` database with source, dataset, and job metadata allowing users to try out features of Marquez (data lineage, view job run history, etc) :tada:
+* Docs on applying db migrations manually
+* **New** Lineage API to support data lineage queries :tada:
+* Support for logging errors via [sentry](https://sentry.io/)
+* **New** Airflow [example](https://github.com/MarquezProject/marquez/tree/main/examples/airflow) with Marquez :tada:
+
+### Changed
+
+* Update OpenLinageDao to stop converting URI structures to contain underscores instead of colons and slashes [@collado-mike](https://github.com/collado-mike)
+* Bump testcontainers dependency to `v1.15.2` [@ ShakirzyanovArsen](https://github.com/ShakirzyanovArsen)
+* Register output datasets for a run lazily [@henneberger](https://github.com/henneberger)
+* Refactor spark plan traversal to find input/output datasets from datasources [@collado-mike](https://github.com/collado-mike)
+* Web UI project settings and default marquez port [@phixMe](https://github.com/phixMe)
+* Associate dataset inputs on run start [@henneberger](https://github.com/henneberger)
+
+### Fixed
+
+* Dataset description is not overwritten on update [@henneberger](https://github.com/henneberger)
+* Latest tags are returned from dataset [@henneberger](https://github.com/henneberger)
+* Airflow integration tests on forked PRs [@mobuchowski](https://github.com/mobuchowski)
+* Empty nominal end time support [@henneberger](https://github.com/henneberger)
+* Ensure valid dataset fields for OpenLineage [@henneberger](https://github.com/henneberger)
+* Ingress context templating for helm chart [@KulykDmytro](https://github.com/KulykDmytro)
 
 ## [0.12.2](https://github.com/MarquezProject/marquez/compare/0.12.0...0.12.2) - 2021-03-16
 
@@ -23,7 +52,7 @@
 *  New `/lineage` endpoint for [OpenLineage](https://github.com/OpenLineage/OpenLineage) support [@henneberger](https://github.com/henneberger)
 *  New graphql endpoint [@henneberger](https://github.com/henneberger)
 *  New spark integration [@henneberger](https://github.com/henneberger)
-*  New API to list versions for a dataset 
+*  New API to list versions for a dataset
 
 ### Changed
 
@@ -54,7 +83,7 @@
 
 * Always migrate db schema on app start in development config
 * Update default db username / password
-* Use [`marquez.dev.yml`](https://github.com/MarquezProject/marquez/blob/main/marquez.dev.yml) in on docker compose `up` 
+* Use [`marquez.dev.yml`](https://github.com/MarquezProject/marquez/blob/main/marquez.dev.yml) in on docker compose `up`
 
 ## [0.11.1](https://github.com/MarquezProject/marquez/compare/0.11.0...0.11.1) - 2020-08-19
 
@@ -194,7 +223,7 @@
 
 ### Added
 
-* Link dataset versions with run inputs 
+* Link dataset versions with run inputs
 * Add schema required by tagging
 * More tests for class `common.Utils`
 * Add `ColumnsTest`
@@ -212,7 +241,7 @@
 ### Added
 
 * Add `Job.latestRun`
-* Add debug logging 
+* Add debug logging
 
 ### Changed
 
@@ -248,7 +277,7 @@
 
 ### Changed
 
-* Rename guid column to uuid 
+* Rename guid column to uuid
 * Use admin ping and health
 * Update `owner` to `ownerName`
 
@@ -258,7 +287,7 @@
 
 ### Fixed
 
-* Fix `marquez.jar` rename on `COPY` 
+* Fix `marquez.jar` rename on `COPY`
 
 ## [0.4.0](https://github.com/MarquezProject/marquez/compare/0.3.4...0.4.0) - 2019-06-04
 
@@ -277,19 +306,19 @@
 
 ### Changed
 
-* Set timestamps to `CURRENT_TIMESTAMP` 
+* Set timestamps to `CURRENT_TIMESTAMP`
 
 ## [0.3.2](https://github.com/MarquezProject/marquez/compare/0.3.1...0.3.2) - 2019-05-14
 
 ### Changed
 
-* Set `job_versions.updated_at` to `CURRENT_TIMESTAMP` 
+* Set `job_versions.updated_at` to `CURRENT_TIMESTAMP`
 
 ## [0.3.1](https://github.com/MarquezProject/marquez/compare/0.3.0...0.3.1) - 2019-05-14
 
 ### Added
 
-* Handle `Flyway.repair()` error 
+* Handle `Flyway.repair()` error
 
 ## [0.3.0](https://github.com/MarquezProject/marquez/compare/0.2.1...0.3.0) - 2019-05-14
 
@@ -299,7 +328,7 @@
 
 ### Changed
 
-* Return timestamp strings as ISO format 
+* Return timestamp strings as ISO format
 
 ### Removed
 

diff --git a/api/gradle.properties b/api/gradle.properties
@@ -1 +1 @@
-version=0.13.0-SNAPSHOT
+version=0.13.0
diff --git a/chart/Chart.yaml b/chart/Chart.yaml
@@ -25,4 +25,4 @@ name: marquez
 sources:
   - https://github.com/MarquezProject/marquez
   - https://marquezproject.github.io/marquez/
-version: 0.2.0
+version: 0.13.0
diff --git a/chart/README.md b/chart/README.md
@@ -40,7 +40,7 @@ helm delete marquez
 | `marquez.replicaCount`       | Number of desired replicas       | `1`                      |
 | `marquez.image.registry`     | Marquez image registry           | `docker.io`              |
 | `marquez.image.repository`   | Marquez image repository         | `marquezproject/marquez` |
-| `marquez.image.tag`          | Marquez image tag                | `0.12.2`                 |
+| `marquez.image.tag`          | Marquez image tag                | `0.13.0`                 |
 | `marquez.image.pullPolicy`   | Image pull policy                | `IfNotPresent`           |
 | `marquez.existingSecretName` | Name of an existing secret containing db password ('marquez-db-password' key) | `nil` |
 | `marquez.db.host`            | PostgreSQL host                  | `localhost`              |
@@ -63,7 +63,7 @@ helm delete marquez
 | `web.replicaCount`       | Number of desired replicas      | `1`            |
 | `web.image.registry`     | Marquez Web UI image registry   | `docker.io`    |
 | `web.image.repository`   | Marquez Web UI image repository | `marquez-web`  |
-| `web.image.tag`          | Marquez Web UI image tag        | `0.12.2`       |
+| `web.image.tag`          | Marquez Web UI image tag        | `0.13.0`       |
 | `web.image.pullPolicy`   | Image pull policy               | `IfNotPresent` |
 | `web.port`               | Marquez Web host port           | `5000`         |
 | `web.resources.limits`   | K8s resource limit overrides    | `nil`          |

diff --git a/chart/values.yaml b/chart/values.yaml
@@ -16,7 +16,7 @@ marquez:
   image:
     registry: docker.io
     repository: marquezproject/marquez
-    tag: 0.12.2
+    tag: 0.13.0
     pullPolicy: IfNotPresent
   ## Name of the existing secret containing credentials for the Marquez installation.
   ## When this is specified, it will take precedence over the values configured in the 'db' section.
@@ -70,7 +70,7 @@ web:
   image:
     registry: docker.io
     repository: marquezproject/marquez-web
-    tag: 0.12.2
+    tag: 0.13.0
     pullPolicy: IfNotPresent
   ## Marquez website will run on this port
   ##

diff --git a/clients/java/README.md b/clients/java/README.md
@@ -10,14 +10,14 @@ Maven:
 <dependency>
     <groupId>io.github.marquezproject</groupId>
     <artifactId>marquez-java</artifactId>
-    <version>0.12.2</version>
+    <version>0.13.0</version>
 </dependency>
 ```
 
 or Gradle:
 
 ```groovy
-implementation 'io.github.marquezproject:marquez-java:0.12.2'
+implementation 'io.github.marquezproject:marquez-java:0.13.0'
 ```
 
 ## Usage
@@ -33,6 +33,6 @@ NamespaceMeta meta = NamespaceMeta().builder()
     .description("My first namespace!")
     .build()
 
-// Create namespace 
+// Create namespace
 Namespace namespace = client.createNamespace("my-namespace", meta);
 ```
diff --git a/clients/java/gradle.properties b/clients/java/gradle.properties
@@ -1,2 +1,2 @@
-version=0.13.0-SNAPSHOT
+version=0.13.0
 jdk8.build=true
diff --git a/clients/python/marquez_client/version.py b/clients/python/marquez_client/version.py
@@ -10,4 +10,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-VERSION = '0.12.2'
+VERSION = '0.13.0'
diff --git a/clients/python/setup.cfg b/clients/python/setup.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.12.2
+current_version = 0.13.0
 
 [metadata]
 description-file = README.md
diff --git a/docs/openapi.html b/docs/openapi.html
diff --git a/gradle.properties b/gradle.properties
@@ -1,4 +1,4 @@
 org.gradle.internal.http.connectionTimeout=60000
 org.gradle.internal.http.socketTimeout=60000
 
-version=0.13.0-SNAPSHOT
+version=0.13.0
diff --git a/integrations/airflow/marquez_airflow/version.py b/integrations/airflow/marquez_airflow/version.py
@@ -10,4 +10,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-VERSION = "0.12.2"
+VERSION = "0.13.0"
diff --git a/integrations/airflow/setup.cfg b/integrations/airflow/setup.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.12.2
+current_version = 0.13.0
 
 [metadata]
 description-file = README.md
diff --git a/integrations/airflow/setup.py b/integrations/airflow/setup.py
@@ -48,7 +48,7 @@ def get_version(rel_path):
     url="https://github.com/MarquezProject/marquez-airflow",
     packages=find_packages(),
     install_requires=[
-        "marquez-python==0.12.2",
+        "marquez-python==0.13.0",
         "sqlparse==0.4.1"
     ]
 )
diff --git a/integrations/airflow/test-requirements.txt b/integrations/airflow/test-requirements.txt
@@ -4,7 +4,7 @@ apache-airflow[gcp_api]==1.10.12
 apache-airflow[google]==1.10.12
 apache-airflow[postgres]==1.10.12
 codecov==2.1.11
-marquez-python==0.12.2
+marquez-python==0.13.0
 pytest
 pytest-cov==2.10.1
 requests==2.24.0

diff --git a/integrations/spark/README.md b/integrations/spark/README.md
@@ -1,6 +1,6 @@
 # Marquez Spark Agent
 
-The Marquez Spark Agent uses jvm instrumentation to emit OpenLineage metadata to Marquez. 
+The Marquez Spark Agent uses jvm instrumentation to emit OpenLineage metadata to Marquez.
 
 ## Installation
 
@@ -10,21 +10,21 @@ Maven:
 <dependency>
     <groupId>io.github.marquezproject</groupId>
     <artifactId>marquez-spark</artifactId>
-    <version>0.12.2</version>
+    <version>0.13.0</version>
 </dependency>
 ```
 
 or Gradle:
 
 ```groovy
-implementation 'io.github.marquezproject:marquez-spark:0.12.2'
+implementation 'io.github.marquezproject:marquez-spark:0.13.0'
 ```
 
 ## Getting started
 
 ### Dataproc
 
-Dataproc requires two things: a uri to the marquez java agent jar in the `files` parameter and 
+Dataproc requires two things: a uri to the marquez java agent jar in the `files` parameter and
 an additional spark property. Dataproc will copy the agent jar to the current working directory of the
 executor and the `-javaagent` parameter will load it on execution.
 
@@ -34,8 +34,8 @@ import os
 ...
 job_name = 'job_name'
 
-jar = 'marquez-spark-0.12.2.jar'
-files = [f"https://repo1.maven.org/maven2/io/github/marquezproject/marquez-spark/0.12.2/marquez-spark-0.12.2.jar"]
+jar = 'marquez-spark-0.13.0.jar'
+files = [f"https://repo1.maven.org/maven2/io/github/marquezproject/marquez-spark/0.13.0/marquez-spark-0.13.0.jar"]
 properties = {
   'spark.driver.extraJavaOptions':
     f"-javaagent:{jar}={os.environ.get('MARQUEZ_URL')}/api/v1/namespaces/{os.getenv('MARQUEZ_NAMESPACE', 'default')}/jobs/{job_name}/runs/{uuid4()}?api_key={os.environ.get('MARQUEZ_API_KEY')}"
@@ -56,8 +56,8 @@ t1 = DataProcPySparkOperator(
 
 ## Arguments
 
-The java agent accepts an argument in the form of a uri. It includes the location of Marquez, the 
-namespace name, the job name, and a unique run id. The run id will be emitted as a parent run 
+The java agent accepts an argument in the form of a uri. It includes the location of Marquez, the
+namespace name, the job name, and a unique run id. The run id will be emitted as a parent run
 facet.
 ```
 {marquez_home}/api/v1/namespaces/{namespace}/job/{job_name}/runs/{run_uuid}?api_key={api_key}"
@@ -75,7 +75,7 @@ Tested and compatible for Spark `2.4.7` only. Other spark versions may cause the
 
 ## Java 8
 
-Testing requires a Java 8 JVM to test the scala spark components. 
+Testing requires a Java 8 JVM to test the scala spark components.
 
 `export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
 

diff --git a/integrations/spark/gradle.properties b/integrations/spark/gradle.properties
@@ -1,2 +1,2 @@
-version=0.13.0-SNAPSHOT
-jdk8.build=true
+version=0.13.0
+jdk8.build=true
diff --git a/spec/openapi.yml b/spec/openapi.yml
@@ -1,7 +1,7 @@
 openapi: 3.0.2
 info:
   title: Marquez
-  version: 0.12.2
+  version: 0.13.0
   description: Marquez is an open source **metadata service** for the **collection**, **aggregation**,
     and **visualization** of a data ecosystem's metadata.
   license:
@@ -1733,4 +1733,4 @@ components:
           type: string
         destination:
           description: The ID of the destination node.
-          type: string
+          type: string