Google¶

Airflow 支持 Google 服务。

所有钩子都基于 airflow.providers.google.common.hooks.base_google.GoogleBaseHook。一些集成还使用 airflow.providers.google.common.hooks.discovery_api。

请参阅 Google Cloud 连接类型文档，以配置与 Google 服务的连接。

Google Cloud¶

Airflow 广泛支持 Google Cloud。

注意

你可以通过分析 Google Cloud 示例 DAG 的源代码来了解如何使用 Google Cloud 集成

服务¶

这些集成允许你在 Google Cloud 中执行各种操作。

传输¶

这些集成允许您从/向 Google Cloud 复制数据。

Google Cloud Storage (GCS) 到 Amazon Simple Storage Service (S3)¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Amazon Simple Storage Service (S3)
操作指南: Google Cloud Storage 到 Amazon S3 传输操作符
Python API: airflow.providers.amazon.aws.transfers.gcs_to_s3
提供商: apache-airflow-providers-amazon

Amazon Glacier 到 Google Cloud Storage (GCS)¶

源产品文档: Amazon Glacier
目标产品文档: Google Cloud Storage (GCS)
操作指南: Amazon S3 Glacier 到 GCS
Python API: airflow.providers.amazon.aws.transfers.glacier_to_gcs
提供商: apache-airflow-providers-amazon

Presto 到 Google Cloud Storage (GCS)¶

源产品文档: Presto
目标产品文档: Google Cloud Storage (GCS)
操作指南: Presto 到 Google Cloud Storage 传输操作符
Python API: airflow.providers.google.cloud.transfers.presto_to_gcs
提供商: apache-airflow-providers-google

Trino 到 Google Cloud Storage (GCS)¶

源产品文档: Trino
目标产品文档: Google Cloud Storage (GCS)
操作指南: Trino 到 Google Cloud Storage 传输操作符
Python API: airflow.providers.google.cloud.transfers.trino_to_gcs
提供商: apache-airflow-providers-google

通用 SQL 到 Google Cloud Storage (GCS)¶

源产品文档: 通用 SQL
目标产品文档: Google Cloud Storage (GCS)
Python API: airflow.providers.google.cloud.transfers.sql_to_gcs
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到 Google Drive¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Google Drive
操作指南: Google Cloud Storage 到 Google Drive 传输算子
Python API: airflow.providers.google.suite.transfers.gcs_to_gdrive
提供商: apache-airflow-providers-google

Google Drive 到 Google Cloud Storage (GCS)¶

源产品文档: Google Drive
目标产品文档: Google Cloud Storage (GCS)
操作指南: Google Drive 到 Google Cloud Storage 传输算子
Python API: airflow.providers.google.cloud.transfers.gdrive_to_gcs
提供商: apache-airflow-providers-google

Microsoft Azure FileShare 到 Google Cloud Storage (GCS)¶

源产品文档: Microsoft Azure FileShare
目标产品文档: Google Cloud Storage (GCS)
操作指南: 将数据从 Azure FileShare 存储传输到 Google Cloud Storage
Python API: airflow.providers.google.cloud.transfers.azure_fileshare_to_gcs
提供商: apache-airflow-providers-google

Apache Cassandra 到 Google Cloud Storage (GCS)¶

源产品文档: Apache Cassandra
目标产品文档: Google Cloud Storage (GCS)
Python API: airflow.providers.google.cloud.transfers.cassandra_to_gcs
提供商: apache-airflow-providers-google

Google 日历到 Google Cloud Storage (GCS)¶

源产品文档: Google 日历
目标产品文档: Google Cloud Storage (GCS)
操作指南: Google 日历到 Google Cloud Storage 传输算子
Python API: airflow.providers.google.cloud.transfers.calendar_to_gcs
提供商: apache-airflow-providers-google

Google 电子表格到 Google Cloud Storage (GCS)¶

源产品文档: Google 电子表格
目标产品文档: Google Cloud Storage (GCS)
操作指南: Google 表格到 Google Cloud Storage 传输算子
Python API: airflow.providers.google.cloud.transfers.sheets_to_gcs
提供商: apache-airflow-providers-google

Amazon Simple Storage Service (S3) 到 Google Cloud Storage (GCS)¶

源产品文档: Amazon Simple Storage Service (S3)
目标产品文档: Google Cloud Storage (GCS)
操作指南: 将数据从 Amazon S3 传输到 Google Cloud Storage
Python API: airflow.providers.google.cloud.transfers.s3_to_gcs
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到 SSH 文件传输协议 (SFTP)¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: SSH 文件传输协议 (SFTP)
操作指南: Google Cloud Storage 传输算子到 SFTP
Python API: airflow.providers.google.cloud.transfers.gcs_to_sftp
提供商: apache-airflow-providers-google

PostgreSQL 到 Google Cloud Storage (GCS)¶

源产品文档: PostgreSQL
目标产品文档: Google Cloud Storage (GCS)
操作指南: Postgres 到 Google Cloud Storage 算子
Python API: airflow.providers.google.cloud.transfers.postgres_to_gcs
提供商: apache-airflow-providers-google

Google BigQuery 到通用 SQL¶

源产品文档: Google BigQuery
目标产品文档: 通用 SQL
Python API: airflow.providers.google.cloud.transfers.bigquery_to_sql
提供商: apache-airflow-providers-google

Google BigQuery 到 MySQL¶

源产品文档: Google BigQuery
目标产品文档: MySQL
操作指南: Google Cloud BigQuery 传输算子到 MySQL
Python API: airflow.providers.google.cloud.transfers.bigquery_to_mysql
提供商: apache-airflow-providers-google

Google BigQuery 到 Microsoft SQL Server (MSSQL)¶

源产品文档: Google BigQuery
目标产品文档: Microsoft SQL Server (MSSQL)
操作指南: Google Cloud BigQuery 传输算子到 Microsoft SQL Server
Python API: airflow.providers.google.cloud.transfers.bigquery_to_mssql
提供商: apache-airflow-providers-google

Google BigQuery 到 PostgreSQL¶

源产品文档: Google BigQuery
目标产品文档: PostgreSQL
操作指南: Google Cloud BigQuery 传输算子到 Postgres
Python API: airflow.providers.google.cloud.transfers.bigquery_to_postgres
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到 Google BigQuery¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Google BigQuery
操作指南: Google Cloud Storage 传输算子到 BigQuery
Python API: airflow.providers.google.cloud.transfers.gcs_to_bigquery
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到 Google Cloud Storage (GCS)¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Google Cloud Storage (GCS)
操作指南: 在 Google Cloud Storage 中传输数据
Python API: airflow.providers.google.cloud.transfers.gcs_to_gcs
提供商: apache-airflow-providers-google

Facebook 广告到 Google Cloud Storage (GCS)¶

源产品文档: Facebook 广告
目标产品文档: Google Cloud Storage (GCS)
操作指南: Facebook 广告到 GCS 算子
Python API: airflow.providers.google.cloud.transfers.facebook_ads_to_gcs
提供商: apache-airflow-providers-google

SSH 文件传输协议 (SFTP) 到 Google Cloud Storage (GCS)¶

源产品文档: SSH 文件传输协议 (SFTP)
目标产品文档: Google Cloud Storage (GCS)
操作指南: SFTP 到 Google Cloud Storage 传输算子
Python API: airflow.providers.google.cloud.transfers.sftp_to_gcs
提供商: apache-airflow-providers-google

Microsoft Azure Data Lake Storage 到 Google Cloud Storage (GCS)¶

源产品文档: Microsoft Azure Data Lake Storage
目标产品文档: Google Cloud Storage (GCS)
Python API: airflow.providers.google.cloud.transfers.adls_to_gcs
提供商: apache-airflow-providers-google

Google BigQuery 到 Google BigQuery¶

源产品文档: Google BigQuery
目标产品文档: Google BigQuery
操作指南: Google Cloud BigQuery 传输算子到 BigQuery
Python API: airflow.providers.google.cloud.transfers.bigquery_to_bigquery
提供商: apache-airflow-providers-google

MySQL 到 Google Cloud Storage (GCS)¶

源产品文档: MySQL
目标产品文档: Google Cloud Storage (GCS)
操作指南: MySQL 到 Google Cloud Storage 算子
Python API: airflow.providers.google.cloud.transfers.mysql_to_gcs
提供商: apache-airflow-providers-google

Oracle 到 Google Cloud Storage (GCS)¶

源产品文档: Oracle
目标产品文档: Google Cloud Storage (GCS)
操作指南: Oracle 到 Google Cloud Storage 算子
Python API: airflow.providers.google.cloud.transfers.oracle_to_gcs
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到 Google 电子表格¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Google 电子表格
操作指南: Google Cloud Storage 到 Google 表格传输算子
Python API: airflow.providers.google.suite.transfers.gcs_to_sheets
提供商: apache-airflow-providers-google

本地到 Google Cloud Storage (GCS)¶

目标产品文档: Google Cloud Storage (GCS)
操作指南: 将数据从本地文件系统上传到 Google Cloud Storage
Python API: airflow.providers.google.cloud.transfers.local_to_gcs
提供商: apache-airflow-providers-google

Google BigQuery 到 Google Cloud Storage (GCS)¶

源产品文档: Google BigQuery
目标产品文档: Google Cloud Storage (GCS)
操作指南: Google Cloud BigQuery 传输算子到 Google Cloud Storage
Python API: airflow.providers.google.cloud.transfers.bigquery_to_gcs
提供商: apache-airflow-providers-google

Google Cloud Storage (GCS) 到本地¶

源产品文档: Google Cloud Storage (GCS)
操作指南: 将数据从 Google Cloud Storage 下载到本地文件系统
Python API: airflow.providers.google.cloud.transfers.gcs_to_local
提供商: apache-airflow-providers-google

Salesforce 到 Google Cloud Storage (GCS)¶

源产品文档: Salesforce
目标产品文档: Google Cloud Storage (GCS)
操作指南: Salesforce 到 GCS 算子
Python API: airflow.providers.google.cloud.transfers.salesforce_to_gcs
提供商: apache-airflow-providers-google

Google Ads 到 Google Cloud Storage (GCS)¶

源产品文档: Google Ads
目标产品文档: Google Cloud Storage (GCS)
Python API: airflow.providers.google.ads.transfers.ads_to_gcs
提供商: apache-airflow-providers-google

Microsoft SQL Server (MSSQL) 到 Google Cloud Storage (GCS)¶

源产品文档: Microsoft SQL Server (MSSQL)
目标产品文档: Google Cloud Storage (GCS)
操作指南: Microsoft SQL Server 到 Google Cloud Storage 算子
Python API: airflow.providers.google.cloud.transfers.mssql_to_gcs
提供商: apache-airflow-providers-google

Microsoft Azure Blob Storage 到 Google Cloud Storage (GCS)¶

源产品文档: Microsoft Azure Blob Storage
目标产品文档: Google Cloud Storage (GCS)
操作指南: Azure Blob Storage 到 Google Cloud Storage (GCS) 传输算子
Python API: airflow.providers.google.cloud.transfers.azure_blob_to_gcs
提供商: apache-airflow-providers-google

Microsoft Azure Blob Storage 到 Google Cloud Storage (GCS)¶

源产品文档: Microsoft Azure Blob Storage
目标产品文档: Google Cloud Storage (GCS)
操作指南: 将数据从 Azure Blob Storage 上传到 Google Cloud Storage（已移至 Google 提供商）
Python API: airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs
提供商: apache-airflow-providers-microsoft-azure

Google Cloud Storage (GCS) 到 Presto¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Presto
操作指南: Google Cloud Storage 到 Presto 传输算子
Python API: airflow.providers.presto.transfers.gcs_to_presto
提供商: apache-airflow-providers-presto

Google Cloud Storage (GCS) 到 Samba¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Samba
操作指南: Google Cloud Storage 传输算子到 Samba
Python API: airflow.providers.samba.transfers.gcs_to_samba
提供商: apache-airflow-providers-samba

Google Cloud Storage (GCS) 到 Snowflake¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Snowflake
Python API: airflow.providers.snowflake.transfers.copy_into_snowflake
提供商: apache-airflow-providers-snowflake

Google Cloud Storage (GCS) 到 Trino¶

源产品文档: Google Cloud Storage (GCS)
目标产品文档: Trino
操作指南: Google Cloud Storage 到 Trino 传输算子
Python API: airflow.providers.trino.transfers.gcs_to_trino
提供商: apache-airflow-providers-trino

Google Marketing Platform¶

注意

你可以通过分析示例 DAG 的源代码来了解如何使用 Google Marketing Platform 集成。

Google Ads¶

运算符: airflow.providers.google.ads.operators.ads.
钩子: airflow.providers.google.ads.hooks.ads.
指南: Google Ads 算子.
提供商: apache-airflow-providers-google
产品文档: Google Ads

Google Analytics (GA4)¶

运算符: airflow.providers.google.marketing_platform.operators.analytics_admin.
钩子: airflow.providers.google.marketing_platform.hooks.analytics_admin.
指南: Google Analytics (GA4) 管理员算子.
提供商: apache-airflow-providers-google
产品文档: Google Analytics (GA4)

Google Analytics360¶

运算符: airflow.providers.google.marketing_platform.operators.analytics.
钩子: airflow.providers.google.marketing_platform.hooks.analytics.
指南: Google Analytics 360 算子.
提供商: apache-airflow-providers-google
产品文档: Google Analytics360

Google Display&Video 360¶

运算符: airflow.providers.google.marketing_platform.operators.display_video.
钩子: airflow.providers.google.marketing_platform.hooks.display_video.
传感器: airflow.providers.google.marketing_platform.sensors.display_video.
指南: Google Display & Video 360 算子.
提供商: apache-airflow-providers-google
产品文档: Google Display&Video 360

Google Search Ads 360¶

运算符: airflow.providers.google.marketing_platform.operators.search_ads.
钩子: airflow.providers.google.marketing_platform.hooks.search_ads.
传感器: airflow.providers.google.marketing_platform.sensors.search_ads.
指南: Google Search Ads 算子.
提供商: apache-airflow-providers-google
产品文档: Google Search Ads 360

其他 Google 算子和挂钩¶

Google¶

钩子: airflow.providers.google.common.hooks.base_google.
提供商: apache-airflow-providers-google
产品文档: Google

Google 日历¶

钩子: airflow.providers.google.suite.hooks.calendar.
提供商: apache-airflow-providers-google
产品文档: Google 日历

Google Cloud Batch¶

运算符: airflow.providers.google.cloud.operators.cloud_batch.
钩子: airflow.providers.google.cloud.hooks.cloud_batch.
指南: Google Cloud Batch 运算符.
提供商: apache-airflow-providers-google
产品文档: Google Cloud Batch

Google Cloud Composer¶

运算符: airflow.providers.google.cloud.operators.cloud_composer.
钩子: airflow.providers.google.cloud.hooks.cloud_composer.
传感器: airflow.providers.google.cloud.sensors.cloud_composer.
指南: Google Cloud Composer 运算符.
提供商: apache-airflow-providers-google
产品文档: Google Cloud Composer

Google Cloud Dataform¶

运算符: airflow.providers.google.cloud.operators.dataform.
钩子: airflow.providers.google.cloud.hooks.dataform.
传感器: airflow.providers.google.cloud.sensors.dataform.
指南: Google Dataform 运算符.
提供商: apache-airflow-providers-google
产品文档: Google Cloud Dataform

Google Cloud Run¶

运算符: airflow.providers.google.cloud.operators.cloud_run.
钩子: airflow.providers.google.cloud.hooks.cloud_run.
指南: Google Cloud Run 运算符.
提供商: apache-airflow-providers-google
产品文档: Google Cloud Run

Google Discovery API¶

钩子: airflow.providers.google.common.hooks.discovery_api.
提供商: apache-airflow-providers-google
产品文档: Google Discovery API

Google 云端硬盘¶

钩子: airflow.providers.google.suite.hooks.drive.
传感器: airflow.providers.google.suite.sensors.drive.
提供商: apache-airflow-providers-google
产品文档: Google Drive

Google LevelDB¶

运算符: airflow.providers.google.leveldb.operators.leveldb.
钩子: airflow.providers.google.leveldb.hooks.leveldb.
指南: Google LevelDB 运算符.
提供商: apache-airflow-providers-google
产品文档: Google LevelDB

Google 电子表格¶

运算符: airflow.providers.google.suite.operators.sheets.
钩子: airflow.providers.google.suite.hooks.sheets.
指南: Google 电子表格运算符.
提供商: apache-airflow-providers-google
产品文档: Google 电子表格

Google¶

Google Cloud¶

服务¶

Google AutoML¶

Google BigQuery¶

Google BigQuery 数据传输服务¶

Google Bigtable¶

Google Campaign Manager¶

Google Cloud Build¶

Google Cloud Common¶

Google Cloud 数据丢失防护 (DLP)¶

Google Cloud Firestore¶

Google Cloud Functions¶

Google Cloud 密钥管理服务 (KMS)¶

Google Cloud 生命科学¶

Google Cloud Memorystore¶

Google Cloud 自然语言¶

Google Cloud OS 登录¶

Google Cloud Pub/Sub¶

Google Cloud 密钥管理器¶

Google Cloud Spanner¶

Google Cloud 语音转文本¶

Google Cloud SQL¶

Google Cloud Stackdriver¶

Google Cloud Storage (GCS)¶

Google Cloud Storage Transfer Service¶

Google Cloud Tasks¶

Google Cloud Text-to-Speech¶

Google Cloud Translation¶

Google Cloud Video Intelligence¶

Google Cloud Vision¶

Google Cloud Workflows¶

Google Compute Engine¶

Google Data Catalog¶

Google Data Fusion¶

Google Data Pipelines¶

Google Dataflow¶

Google Dataplex¶

Google Dataprep¶

Google Dataproc¶

Google Dataproc Metastore¶

Google Datastore¶

Google Deployment Manager¶

Google Kubernetes Engine¶

Google Looker¶

Google Machine Learning Engine¶

Google Vertex AI¶

传输¶

Google Cloud Storage (GCS) 到 Amazon Simple Storage Service (S3)¶

Amazon Glacier 到 Google Cloud Storage (GCS)¶

Presto 到 Google Cloud Storage (GCS)¶

Trino 到 Google Cloud Storage (GCS)¶

通用 SQL 到 Google Cloud Storage (GCS)¶

Google Cloud Storage (GCS) 到 Google Drive¶

Google Drive 到 Google Cloud Storage (GCS)¶

Microsoft Azure FileShare 到 Google Cloud Storage (GCS)¶

Apache Cassandra 到 Google Cloud Storage (GCS)¶

Google 日历到 Google Cloud Storage (GCS)¶

Google 电子表格到 Google Cloud Storage (GCS)¶

Amazon Simple Storage Service (S3) 到 Google Cloud Storage (GCS)¶

Google Cloud Storage (GCS) 到 SSH 文件传输协议 (SFTP)¶

PostgreSQL 到 Google Cloud Storage (GCS)¶

Google BigQuery 到通用 SQL¶

Google BigQuery 到 MySQL¶

Google BigQuery 到 Microsoft SQL Server (MSSQL)¶

Google BigQuery 到 PostgreSQL¶

Google Cloud Storage (GCS) 到 Google BigQuery¶

Google Cloud Storage (GCS) 到 Google Cloud Storage (GCS)¶

Facebook 广告到 Google Cloud Storage (GCS)¶

SSH 文件传输协议 (SFTP) 到 Google Cloud Storage (GCS)¶

Microsoft Azure Data Lake Storage 到 Google Cloud Storage (GCS)¶

Google BigQuery 到 Google BigQuery¶

MySQL 到 Google Cloud Storage (GCS)¶

Oracle 到 Google Cloud Storage (GCS)¶

Google Cloud Storage (GCS) 到 Google 电子表格¶

本地到 Google Cloud Storage (GCS)¶

Google BigQuery 到 Google Cloud Storage (GCS)¶

Google Cloud Storage (GCS) 到本地¶

Salesforce 到 Google Cloud Storage (GCS)¶

Google Ads 到 Google Cloud Storage (GCS)¶