Justin Taras – Medium

Justin Taras

Published in

Migrating Delta Lake Tables to Google Cloud Storage using Storage Transfer Service

The Scenario

Oct 29

Migrating Delta Lake Tables to Google Cloud Storage using Storage Transfer Service

Oct 29

Dataplex Catalog Export to BigQuery

TLDR; Dataplex recently introduced the ability to export catalog data to Google Cloud Storage. This feature is valuable for several…

Sep 11

Dataplex Catalog Export to BigQuery

Sep 11

Deciphering Dataplex Consumption in Google Cloud Billing

Overview

Jul 31

Deciphering Dataplex Consumption in Google Cloud Billing

Jul 31

Published in

BigQuery Advanced Runtime: Testing Performance Part 2

TL;DR Building on the previous analysis of the Advanced Runtime, this next test measures the effect of clustering on performance…

Jul 21

BigQuery Advanced Runtime: Testing Performance Part 2

Jul 21

Published in

BigQuery Advanced Runtime: Testing Performance Part 1

TL;DR BigQuery’s new Advanced Runtime is an under-the-hood upgrade that automatically makes your queries faster without requiring any…

Jun 30

BigQuery Advanced Runtime: Testing Performance Part 1

Jun 30

Logging new Airflow DAG entires in Cloud Composer

DAG Upload Audit

Aug 2, 2024

Logging new Airflow DAG entires in Cloud Composer

Aug 2, 2024

Dataproc Serverless: Python Package Management through Conda

TL;DR Use Conda to package up python dependencies for your Dataproc Serverless jobs

May 17, 2024

Dataproc Serverless: Python Package Management through Conda

May 17, 2024

Cloud Data Fusion: Tracking Pipeline Spend

TL;DR Using cluster labels in compute profiles is a great way to track spend at a pipeline level.

Apr 19, 2024

Cloud Data Fusion: Tracking Pipeline Spend

Apr 19, 2024

Cloud Data Fusion: Using Spark SQL for Column Transformations

TL;DR: While data transformation tools like Wrangler offer extensive features, you may occasionally require custom functionality, such as…

Feb 26, 2024

Cloud Data Fusion: Using Spark SQL for Column Transformations

Feb 26, 2024

Cloud Data Fusion: Using RBAC to Enforce Data Access

TL;DR You can use a combination of RBAC and Pipeline Service Accounts to scope data access for teams/project to just the data required for…

Feb 1, 2023

Cloud Data Fusion: Using RBAC to Enforce Data Access

Feb 1, 2023

Justin Taras

Justin Taras

I’m a Google Customer Engineer interested in all things data. I love helping customers leverage their data to build new and powerful data driven applications!

Following

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech