Senior Data Engineer
Realoq is actively seeking a Senior Data Engineer to join our dynamic team, playing a crucial role in shaping and optimizing our data infrastructure. As a Senior Data Engineer, you will be at the forefront of Realoq’s data-driven initiatives, contributing to the design, development, and maintenance of robust data systems.
Hyderabad, India (Remote/Hybrid)
- Implement Logical and physical data models as per the business needs.
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
- Implementing ETL and ELT process.
- Discuss with stakeholders to understand the requirements and come up with timelines.
- Come up with strategies to test the pipelines.
- Do design and implementation for data collection, cleansing, enrichment, and maintenance.
- 5+ years of hands-on experience as a Data Engineer.
- Excellent knowledge of setting up ETL and ELT jobs.
- Job scheduling and reporting using Airflow/Prefect or relevant tools.
- Good understanding of strategies for data enrichment, data staging.
- Strong coding skills in Python is necessary.
- Experience in updating and optimizing the local and metadata models.
- Evaluate the implemented data system for variances, discrepancies, and efficiencies.
- Troubleshoot and optimize the existing data flows, models, and processing jobs by modularizing.
- Explore ways to enhance the data quality and reliability.
- Strong in writing UDF.
- Proficiency in writing clean and effective code using Python, Scala, and PySpark is essential, coupled with a strong understanding of the Spark framework and a comprehensive grasp of the dynamics of the Hadoop cluster ecosystem, including services like Spark jobs.
- Enthusiasm to solve any ongoing issues with operating the cluster.
- Good knowledge of Big Data querying tools, such as AWS Athena, and SparkSQL.
- Experience with integration of data from multiple data sources.
- Knowledge of various ETL techniques and frameworks, like AWS Glue, and AWS Lambda preferred.
Why to Join Us ?
Innovation and Creativity:
By joining Realoq, you become part of a team that thrives on pushing boundaries and finding unique solutions to complex problems. This environment not only allows you to exercise your creative muscles but also encourages you to think critically, challenge traditional norms, and foster a culture of innovation.
Opportunity to Make a Real Impact:
By joining Realoq, your contributions are highly visible and can directly shape the company’s direction and success. Realoq provides an environment where your ideas, skills, and hard work are recognized and valued.
Personal Growth and Skill Development:
You will be exposed to a wide range of responsibilities and roles, allowing you to develop a diverse skill set and you may have the opportunity to wear multiple hats and take on responsibilities outside your primary role. This exposure enables rapid personal and professional growth, fostering a steep learning curve that can propel your career forward.