Justin Taras – Medium

Justin Taras

Cloud Data Fusion: Tracking Pipeline Spend

TL;DR Using cluster labels in compute profiles is a great way to track spend at a pipeline level.

3 min read17 hours ago

--

Cloud Data Fusion: Tracking Pipeline Spend

--

Justin Taras

Cloud Data Fusion: Using Spark SQL for Column Transformations

TL;DR: While data transformation tools like Wrangler offer extensive features, you may occasionally require custom functionality, such as…

3 min readFeb 26, 2024

--

Cloud Data Fusion: Using Spark SQL for Column Transformations

--

Justin Taras

Cloud Data Fusion: Using RBAC to Enforce Data Access

TL;DR You can use a combination of RBAC and Pipeline Service Accounts to scope data access for teams/project to just the data required for…

8 min readFeb 1, 2023

--

Cloud Data Fusion: Using RBAC to Enforce Data Access

--

Justin Taras

Cloud Data Fusion: Building Job Metadata Pipelines

TL;DR Data Fusion creates a wealth of metadata related to pipeline performance and configuration. This article will explore building a…

12 min readNov 2, 2022

--

1

Cloud Data Fusion: Building Job Metadata Pipelines

--

1

Justin Taras

Cloud Data Fusion: Connecting to CloudSQL via SSL/TLS

TL;DR How to configure your Data Fusion pipelines to support SSL/TLS connectivity between your Data Fusion pipelines and CLoudSQL MySQL.

5 min readOct 25, 2022

--

Cloud Data Fusion: Connecting to CloudSQL via SSL/TLS

--

Justin Taras

Cloud Data Fusion: Using Terraform to run ephemeral Data Fusion Instances

TL;DR Some users of Data Fusion only have a small number of pipelines to run on a daily basis. This can make running an always on Data…

6 min readJun 3, 2022

--

2

Cloud Data Fusion: Using Terraform to run ephemeral Data Fusion Instances

--

2

Justin Taras

Cloud Data Fusion: Reverse ETL from BigQuery to CloudSQL

TL;DR Traditional ETL is all about moving data from operational systems into a system of truth like a data warehouse. The reverse ETL model…

9 min readMay 19, 2022

--

1

Cloud Data Fusion: Reverse ETL from BigQuery to CloudSQL

--

1

Justin Taras

Cloud Data Fusion: Adding a Service Account to the Secure Store

Storing Service Account JSON keys in plain text is not ideal to say the least. To protect that sensitive information, it is recommended the…

4 min readMay 2, 2022

--

Cloud Data Fusion: Adding a Service Account to the Secure Store

--

Justin Taras

Cloud Data Fusion: Upload UDD’s through the Rest API

TL;DR Use the Rest API to upload UDD’s across namespaces in a programatic way. While the Data Fusion UI has the ability to do this, it can…

3 min readMar 21, 2022

--

1

Cloud Data Fusion: Upload UDD’s through the Rest API

--

1

Justin Taras

Building Apache Livy 0.8.0 for Spark 3.x

TL;DR The 0.7.0 version of Livy was built for Spark 2.4.5 and Scala 11. Newer versions of Spark build on Scala 12 will not run the current…

3 min readMar 21, 2022

--

1

Building Apache Livy 0.8.0 for Spark 3.x

--

1

Justin Taras

Justin Taras

I’m a Google Customer Engineer interested in all things data. I love helping customers leverage their data to build new and powerful data driven applications!

Following

Help
Status
About
Careers
Blog
Privacy
Terms
Text to speech
Teams