This gem triggers pipelines to run on EMR Serverless fabrics.
| Parameter | Description |
|---|---|
| Pipeline to schedule | The pipeline in the project that will be triggered. |
| Fabric & Job size to run this pipeline | Defines the compute fabric and job size configuration for the pipeline execution. |
| AWS Connection Id | The Airflow connection ID for AWS credentials. Used to authenticate with AWS services. |
| Action on Failure | Defines the behavior when the pipeline execution fails. Options include continuing despite failure or other failure handling strategies. |
| EMR Serverless Application Id Override | Optional field to specify a custom EMR Serverless application ID. This way, you can run the job on a different cluster than defined in the fabric, but keep all the other fabric configurations. |
| Execution Role ARN | The Amazon Resource Name (ARN) of the IAM role that EMR Serverless will assume to execute the job. Required for permissions to access AWS resources. |
| Configuration Overrides | JSON configuration object for EMR Serverless job settings. Allows you to specify application-level logging and runtime configuration properties. Learn more. |
| Set Pipeline Configurations | Define pipeline configurations with Schema and Config tabs. |