Data / Big Data Testing Engineer

We are seeking a meticulous and experienced Data / Big Data Testing Engineer to join our growing team. In this role, you will be responsible for ensuring the quality, integrity, and reliability of our data pipelines, data warehouses, and analytical platforms. You will work closely with data engineers, data scientists, and business analysts to develop and execute comprehensive test plans and strategies for our big data solutions. Your expertise will be crucial in validating data accuracy, performance, and scalability across various data sources and processing frameworks.

Responsibilities:

1. Design, develop, and execute test cases and test scripts for data pipelines, ETL processes, and data warehousing solutions.
2. Perform data validation and reconciliation to ensure data accuracy and consistency across different systems.
3. Test the performance, scalability, and reliability of big data solutions, including data ingestion, processing, and querying.
4. Collaborate with data engineers and architects to understand data models, data flows, and system requirements.
5. Identify, document, and track defects, and work with development teams to resolve them.
6. Develop and maintain automated testing frameworks for big data applications.
7. Contribute to the continuous improvement of testing processes and methodologies.

Qualifications:

1. Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
2. Proven experience in data testing, ETL testing, and big data testing.
3. Strong understanding of data warehousing concepts, data modeling, and database principles.
4. Experience with SQL and proficiency in querying large datasets.
5. Familiarity with big data technologies such as Hadoop, Spark, Kafka, or similar.
6. Experience with testing tools and frameworks (e.g., pytest, Selenium, JUnit).
7. Knowledge of cloud platforms (AWS, Azure, GCP) and their data services is a plus.
8. Excellent analytical and problem-solving skills.
9. Strong communication and collaboration skills.