AWS stack ( S3, GLUE , Py spark)

Job Description:

AWS Glue supports an extension of the PySpark Python dialect for scripting extract, transform, and load (ETL) jobs. This section describes how to use Python in ETL scripts and with the AWS Glue API.

• Setting up to use Python with AWS Glue
• Calling AWS Glue APIs in Python
• Using Python libraries with AWS Glue
• AWS Glue Python code samples

AWS Glue has created the following extensions to the PySpark Python dialect.

• Accessing parameters using getResolvedOptions
• PySpark extension types
• DynamicFrame class
• DynamicFrameCollection class
• DynamicFrameWriter class
• DynamicFrameReader class
• GlueContext class

AWS Glue has created the following transform Classes to use in PySpark ETL operations.

• GlueTransform base class
• ApplyMapping class
• DropFields class
• DropNullFields class
• ErrorsAsDynamicFrame class
• FillMissingValues class
• Filter class
• FindIncrementalMatches class
• FindMatches class
• FlatMap class
• Join class
• Map class
• MapToCollection class
• mergeDynamicFrame
• Relationalize class
• RenameField class
• ResolveChoice class
• SelectFields class
• SelectFromCollection class
• Spigot class
• SplitFields class
• SplitRows class
• Unbox class
• UnnestFrame class

Any Degree

Python/Unix scripting ,SQL/DWH , AWS stack ( S3, GLUE , Py spark)