GCS

Availability: Airbyte Cloud: ✅, Airbyte OSS: ✅
Support Level: Community
Latest Version: 0.3.0
Definition Id: 2a8c41ae-8c23-4be0-a73f-2ab10ca1a820

This page guides you through the process of setting up the GCS source connector. This connector supports loading multiple CSV files (non compressed) from a GCS directory. The conntector will check for all files ending in .csv, even nested files.

info

Cloud storage may incur egress costs. Egress refers to data that is transferred out of the cloud storage system, such as when you download files or access them from a different location. For more information, see the Google Cloud Storage pricing guide.

Prerequisites

JSON credentials for the service account that has access to GCS. For more details check instructions
GCS bucket
Path to file(s)

Set up Source

Create a Service Account

First, you need to select existing or create a new project in the Google Cloud Console:

Sign in to the Google Account.
Go to the Service Accounts page.
Click Create service account.
Create a JSON key file for the service user. The contents of this file will be provided as the service_account in the UI.

Grant permisison to GCS

Use the service account ID from above, grant read access to your target bucket. Click here for more details.

Set up the source in Airbyte UI

Paste the service account JSON key to service_account
Enter your GCS bucket name to gcs_bucket
Enter path to your file(s) to gcs_path

Changelog

Version	Date	Pull Request	Subject
0.3.0	2023-10-11	31212	Migrated to file based CDK
0.2.0	2023-06-26	27725	License Update: Elv2
0.1.0	2023-02-16	23186	New Source: GCS

Prerequisites​

Set up Source​

Create a Service Account​

Grant permisison to GCS​

Set up the source in Airbyte UI​

Changelog​