Senior Python Data Engineer

Key Responsibilities

• Design, build and maintain Python plugins that encapsulate business rules and transformation logic such as custom aggregation/disaggregation, rule-based data enrichment, and time-grain manipulations.
  
• Implement data-cleaning, validation, mapping and merging routines using Pandas, NumPy and other Python libraries to prepare inputs for downstream analytics.
  
• Define clear interfaces and configuration schemes for plugins (for example, picklists for output grains and rule codes) and package them for easy consumption by implementation teams.
  
• Profile and optimize Python code paths—leveraging vectorized operations, efficient merges and in-memory transformations—to handle large datasets with low latency.
  
• Establish and enforce coding standards, write comprehensive unit and integration tests with pytest (or similar), and ensure high coverage for all new components.
  
• Triage and resolve issues in legacy scripts, refactor complex routines for readability and extensibility, and manage versioning across releases.
  

Must-Have Skills

• 5–7 years of hands-on experience writing well-structured Python code, with deep familiarity in both object-oriented and functional programming paradigms.
  
• Expert mastery of Pandas and NumPy for transformations (group-by aggregations, merges, column operations), plus strong comfort with Python’s datetime and copy modules.
  
• Proven experience designing modular Python packages, exposing configuration through parameters or picklists, and managing versioned releases.
  
• Ability to translate complex, rule-driven requirements (such as disaggregation rules, external-table merges, and priority ranking) into clean Python functions and classes.
  
• Proficiency with pytest (or equivalent), mocking, and integrating with CI pipelines (e.g., GitHub Actions, Jenkins) for automated testing.
  
• Skilled use of Python’s logging module to instrument scripts, manage log levels, and capture diagnostic information.
  
• Strong Git workflow experience, including branching strategies, pull requests, code reviews, and merge management.