Job Description
Role: Data Engineer with snowflake, vertica, ETL, MySQL, Airflow
Location: Remote
Duration: months
Responsibilities:
Design and implement robust ETL/ELT pipelines to ingest, transform, and load data from various sources.
Work with relational and columnar databases including MySQL, PL/SQL, and Vertica to manage large datasets.
Develop and schedule data workflows using Shell/Bash scripting and Python.
Build and maintain data pipelines on Snowflake using ELT best practices.
Collaborate with data analysts, data scientists, and other stakeholders to understand data needs.
Optimize existing processes for scalability and performance.
Use tools like Airflow, Fivetran, and DBT for orchestration, transformation, and data movement.
Monitor data pipeline performance and resolve issues in a timely manner.
Document...