Data Engineer

What You'll Do As a Data Engineer, you will be engaged from the first client conversation all the way through to delivery — gathering requirements, designing the solution, and seeing it through to completion. You will design, construct, install, test, and maintain highly scalable data management systems and robust data pipelines. Your work will ensure data quality, reliability, and accessibility for our AI/ML engineers and LLM applications, leveraging cloud platforms and modern data engineering practices, including workflow orchestration. Design, build, and optimize scalable ETL/ELT data pipelines using Python and cloud-native tools (on AWS, Azure, or GCP). Develop data models and schemas optimized for analytical and AI/ML workloads. Implement data quality checks and monitoring frameworks. Manage and administer data warehouses, data lakes, and databases (SQL/NoSQL). Implement and manage workflow orchestration tools (e.g., Airflow, Prefect, Dagster) for scheduling and monitoring data pipelines. Collaborate closely with AI/ML Engineers and LLM Engineers to understand their data requirements. Ensure data security and compliance standards are met. Optimize data storage and processing costs on Hyperscaler platforms. Write efficient and maintainable Python code for data processing tasks. Work independently to troubleshoot and resolve data-related issues. Requirements What you’ll Bring Strong proficiency in Python for data manipulation and pipeline development (e.g., Pandas, PySpark) Expertise in SQL and experience with relational and NoSQL databases Hands-on experience with cloud-based data services on at least one Hyperscaler (e.g., AWS S3/Glue/Redshift, Azure Data Factory/Synapse, GCP Cloud Storage/Dataflow/BigQuery) Experience building and managing data pipelines and ETL/ELT processes Familiarity with data warehousing concepts and data modeling Understanding of data quality principles Ability to work independently and take ownership of data infrastructure components Ho