Data Engineer (PySpark + Databricks)
Experience Required: 3.5+ Years
Location: Infopark Phase 2, Kochi
Experience: 3+ Years
Company: CHISQUARE LABS
We are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.
Key Responsibilities:
Ownership and management of databricks platform. Building software with PySpark / Python in Databricks. Ad-hoc data engineering specific software development.
- Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on Databricks.
- Data Architecture & Engineering: Work on distributed computing solutions, optimize Spark jobs, and build efficient data models.
- Performance & Cost Optimization: Fine-tune Spark configurations, optimize Databricks clusters, and reduce compute/storage costs.
- Collaboration: Work closely with Data Scientists, Analysts, and DevOps teams to ensure data reliability.
- ETL & Data Warehousing: Implement scalable ETL processes for structured & unstructured data.
- Monitoring & Automation: Implement logging, monitoring, and alerting mechanisms for data pipeline health and fault tolerance.
Qualifications:
- Bachelor’s/Master’s in Computer Science, Data Engineering, or related field
- 3+ years of experience in Big Data Engineering with PySpark and Databricks
- Python, Databricks, PySpark, SQL and NoSQL databases,
- Backend software engineering, REST APIs, AWS Lambda
- Docker-based development and deployment.
- AWS ECS, AWS DMS, CI-CD understanding.
- Databricks certification is a plus
Why Join Us?
- Work on innovative projects with emerging technologies
- Collaborative and learning-oriented work culture
- Opportunities for growth and upskilling
If you’re passionate about automation and excited to grow in a future-ready tech environment, we’d love to hear from you!