Data Engineer

• Gains a thorough understanding of the requirements and ensure that work product aligns with customer requirements
• Works within the established development guidelines, standards, methodologies, and naming conventions
• Builds processes to ingest, process and store massive amount of data
• Assists with optimization the performance of bigdata ecosystems
• Wrangles data in support of data science projects
• Performs productionization of ML and statistical models for Data Scientists & Statisticians
• Constructs, tests and maintains scalable data solutions for structured and unstructured data to support reporting, analytics, ML and AI
• Assists with research and building of proof of concepts to test out theories recommended by Senior and Lead Data Engineers
• Collaborates and contributes in identifying project risks, design mitigation plans, develops estimates
• Contributes to the design, development of data-pipelines, and feature engineering of data solutions
• Established processes or methods are still relied on however Data Engineer will be required to come up with creative solutions to problems. Senior teammates are still needed to provide oversight on solutions to complex problems.

Requirements:

• Experience: Minimum of 5 years of proven experience as a Data Engineer or in a similar role.
• Robust Technical Skills in:
  • Microsoft Data Factory: Solid experience in designing and building data pipelines.
  • Azure Data Lake: Deep understanding of storing and managing data at scale.
  • Azure: Hands-on experience within the Microsoft Azure data services ecosystem.
  • Databricks: Proficiency in using the Databricks platform for data processing and solution development.
  • Python: Advanced proficiency in Python for data engineering, scripting, and application development.
  • SQL: Advanced experience with SQL for manipulating, querying, and optimizing relational databases.
• Problem-Solving Mindset: Demonstrated ability to approach and solve complex challenges with a solution-oriented focus.
• Inquisitive Mindset & Continuous Learning: A constant desire to explore new technologies, methodologies, and enhance skills.
• Excellent Communication Skills: Ability to communicate clearly and effectively with technical team members and non-technical stakeholders.
• Ability to work both independently and as part of a collaborative team.