

************
DataPipeline
************



======
Client
======



.. py:class:: DataPipeline.Client

  A low-level client representing AWS Data Pipeline
  

  AWS Data Pipeline configures and manages a data-driven workflow called a pipeline. AWS Data Pipeline handles the details of scheduling and ensuring that data dependencies are met so that your application can focus on processing the data.

   

  AWS Data Pipeline provides a JAR implementation of a task runner called AWS Data Pipeline Task Runner. AWS Data Pipeline Task Runner provides logic for common data management scenarios, such as performing database queries and running data analysis using Amazon Elastic MapReduce (Amazon EMR). You can use AWS Data Pipeline Task Runner as your task runner, or you can write your own task runner to provide custom data management.

   

  AWS Data Pipeline implements two main sets of functionality. Use the first set to create a pipeline and define data sources, schedules, dependencies, and the transforms to be performed on the data. Use the second set in your task runner application to receive the next task ready for processing. The logic for performing the task, such as querying the data, running data analysis, or converting the data from one format to another, is contained within the task runner. The task runner performs the task assigned to it by the web service, reporting progress to the web service as it does so. When the task is done, the task runner reports the final success or failure of the task to the web service.

  ::

    
    import boto3
    
    client = boto3.client('datapipeline')

  

These are the available methods:

.. toctree::
  :maxdepth: 1
  :titlesonly:

  datapipeline/client/activate_pipeline
  datapipeline/client/add_tags
  datapipeline/client/can_paginate
  datapipeline/client/close
  datapipeline/client/create_pipeline
  datapipeline/client/deactivate_pipeline
  datapipeline/client/delete_pipeline
  datapipeline/client/describe_objects
  datapipeline/client/describe_pipelines
  datapipeline/client/evaluate_expression
  datapipeline/client/get_paginator
  datapipeline/client/get_pipeline_definition
  datapipeline/client/get_waiter
  datapipeline/client/list_pipelines
  datapipeline/client/poll_for_task
  datapipeline/client/put_pipeline_definition
  datapipeline/client/query_objects
  datapipeline/client/remove_tags
  datapipeline/client/report_task_progress
  datapipeline/client/report_task_runner_heartbeat
  datapipeline/client/set_status
  datapipeline/client/set_task_status
  datapipeline/client/validate_pipeline_definition


==========
Paginators
==========


Paginators are available on a client instance via the ``get_paginator`` method. For more detailed instructions and examples on the usage of paginators, see the paginators `user guide <https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html>`_.

The available paginators are:

.. toctree::
  :maxdepth: 1
  :titlesonly:

  datapipeline/paginator/DescribeObjects
  datapipeline/paginator/ListPipelines
  datapipeline/paginator/QueryObjects
