Azure Data Engineer - Remote, Latin America

Azure Data Engineer - Remote, Latin America

Empresa: Bluelight Consulting, Llc

Tipo de empleo: Tiempo completo Remote / Work from Home

Bluelight es una consultora de software líder dedicada a diseñar y desarrollar tecnologías innovadoras que mejoran la vida de los usuarios. Con un compromiso firme con la entrega excepcional de servicios a nuestros clientes, Bluelight sobresale en su enfoque en la calidad y la satisfacción del cliente. Nuestra misión no solo es crear aplicaciones de vanguardia, sino también fomentar un entorno colaborativo y enriquecedor donde cada miembro del equipo puede crecer y prosperar. Con presencia en los Estados Unidos y América Central/Sur, Bluelight está en una fase emocionante de expansión, buscando constantemente talento excepcional para unirse a su comunidad dinámica y diversa. Como ETL Data Engineer, jugará un papel crítico en el equipo de ingeniería de datos de nuestros clientes, diseñando, desarrollando y manteniendo procesos de integración de datos principalmente usando Python (PySpark) y Azure Synapse Analytics para asegurar la precisión y disponibilidad de datos analíticos. Trabajando de cerca con científicos de datos, analistas y otras partes interesadas para entregar datos de alta calidad para insights y toma de decisiones, esta posición es ideal para un profesional de desarrollo de software apasionado que prospera en un entorno dinámico y acelerado donde las opiniones y los esfuerzos de todos son valorados. Al unirte al creciente consultorio de software de nuestros clientes, tendrás la oportunidad de contribuir a proyectos desafiantes y de alto nivel dentro de una comunidad colaborativa que valora profundamente el trabajo duro, el aprendizaje continuo, el crecimiento personal y el desarrollo profesional.

Responsibilities

• ETL Data Engineering: Develop and maintain ETL data engineering processes using Python (PySpark) within Azure Synapse Analytics Notebooks, and/or Azure Synapse Analytics Pipelines, to ensure efficient data extractions, transformation, and loading.
• Data Warehousing: Apply your expertise in data warehousing, understanding star schemas, facts, and dimensions, to design and build effective data storage structures in a Massively Parallel Processing (MPP) SWL Pool.
• Data Source Expertise: Extract data from various sources, including REST APIs, SWL database tables, and CSV files.
• Azure Synapse Analytics Expertise: Utilize your deep knowledge of Azure Synapse Analytics to design and optimize data notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to the implementation and understanding of other Data Fabric concepts, such as data lakes, lakehouses, delta lakes, and data cataloging, to enhance data management capabilities.
• Data Modeling: Collaborate with data architects to create data models and schemas that align with business requirements.
• Data Quality: Implement data quality checks and validation processes to maintain data accuracy and consistency.
• Performance Tuning: Identify and resolve performance bottlenecks and optimize ETL data notebooks/pipelines to meet SLAs.
• Monitoring and Troubleshooting: Monitoring ETL jobs, diagnose issues, and implement solutions to ensure data pipeline reliability.
• Documentation: Maintain comprehensive documentation of ETL data engineering processes, data flows, and data transformations.
• Collaboration: Work closely with cross-functional teams to understand data requirements and provide support for data-related initiatives.
• Security and Compliance: Ensure data security and compliance with data governance and privacy standards.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field; or equivalent work experience, with certifications related to data engineering or data science (e.g. Azure Data Engineer) being a plus.
• Proven experience in ETL data engineering with significant expertise in using Python (PySpark) to perform data extraction, transformation, and loading from REST APIs, SQL database tables, and CSV files.
• Proficiency in using Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault.
• Demonstrated ability to write complex SQL queries, optimize query performance, and work with both SparkSQL and MS SQL to effectively extract, transform, and load data.
• Knowledge of data integration best practices and tools.
• Experience with version control systems, such as Git (Azure DevOps).
• Strong problem-solving and analytical skills, with a keen attention to detail.
• Excellent communication skills, both verbal and written, with the ability to work collaboratively in a team environment with shifting priorities.
• Familiarity with big data technologies, machine learning, and data analysis preferred.
• Experience with data visualization tools (e.g. Power BI, Tableau) and Agile Methodologies a plus.

Sobre la empresa

Bluelight Consulting, Llc

Easily find highly skilled, carefully vetted Nearshore Tech Talent ready to take your most important software development projects from concept to reality.