![]() task_id ( string) – a unique, meaningful id for the task.Task dependencies shouldīe set by using the set_upstream and/or set_downstream methods. Which ultimately becomes a node in DAG objects. Instantiating aĬlass derived from this one results in the creation of a task object, This class is abstract and shouldn’t be instantiated. Operators (tasks) target specific operations, running specific scripts, Moves data from Hive to MySQL (Hive2MySqlOperator). Waits for a partition to land in Hive (HiveSensorOperator), or one that Operator that runs a Pig job (PigOperator), a sensor operator that Operators derived from this class should perform or trigger certain tasks The constructor as well as the ‘execute’ method. To derive this class, you are expected to override Since operators create objects thatīecome nodes in the dag, BaseOperator contains many recursive methods forĭag crawling behavior. BaseOperator ( task_id, owner='Airflow', email=None, email_on_retry=True, email_on_failure=True, retries=0, retry_delay=datetime.timedelta(0, 300), retry_exponential_backoff=False, max_retry_delay=None, start_date=None, end_date=None, schedule_interval=None, depends_on_past=False, wait_for_downstream=False, dag=None, params=None, default_args=None, adhoc=False, priority_weight=1, weight_rule='downstream', queue='default', pool=None, sla=None, execution_timeout=None, on_failure_callback=None, on_success_callback=None, on_retry_callback=None, trigger_rule='all_success', resources=None, run_as_user=None, task_concurrency=None, executor_config=None, inlets=None, outlets=None, *args, **kwargs ) ¶īases: _mixin.LoggingMixinĪbstract base class for all operators. To understand the primitive features that can be leveraged in yourĭAGs. It’s worth taking the time to understand the parameters of BaseOperator ![]() All operators are derived from BaseOperator and acquire muchįunctionality through inheritance.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |