Data Engineer needed for Marchon Partners LLC located in Boston, MA. Design and develop scalable meta-driven ETL/ELT frameworks using Medallion architecture (bronze, silver, gold) improving reusability, maintainability, and development efficiency across enterprise data platforms. Design and implement real-time batch data scalable ETL pipelines using Azure Data Factory, Azure Databricks, and Python, processing large volumes of semi-structured data into Data Lake using PySpark. Implement API-based integrations to support continuing increases in data volume and complexity. Optimized data warehouse performance through query tuning, code optimization, and proactive monitoring of pipeline and system performance. Perform data cleansing, transformation, and enrichment using Azure cloud services to ensure high-quality and consistent data. Implemented CI/CD automation using Databricks Asset Bundles (DAB) with Azure DevOps, including Azure Key Vault integration for enabling environment-agnostic deployments, version control, and release standardization. Will assist client located throughout the U.S. Master’s degree in Computer Science or Computer Information Systems and 3 years of experience performing the duties listed above. Must be willing to travel or relocate.
Send resumes to: spayeur@marchonpartners.com
