Data Engineer II
Mastercard•19h ago
Pune, IndiaOnsiteFull-timeMid Level2+ yrs exp
Top focus
Data EngineerVp DataData Warehouse EngineerSenior Data Engineer
- Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Data Engineer II Overview Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart
- accessible. Using secure data and networks, partnerships
- passion, our innovations help individuals, financial institutions, governments
- businesses realize their greatest potential. The Mastercard Services organization is a key differentiator, delivering cutting-edge solutions used by some of the world’s largest organizations to make critical business decisions. Focused on innovation and scale, Services provides data-driven capabilities across consulting, analytics, experimentation
- risk management. Team Overview As part of Mastercard’s Data Platform & Orchestration team, you will contribute to building next-generation data platforms that are critical to our global data ecosystem. Our team develops and operates scalable platform capabilities, including:
- Cloud-native infrastructure and application provisioning
- Standardized CI/CD pipelines and engineering tooling
- Reusable frameworks and data platform components These platforms enable teams to build, deploy
- operate data-driven solutions efficiently and securely at scale. Role Overview Data Platform & Orchestration is seeking a Data Engineer II to design and build next-generation, cloud-native data platforms supporting Mastercard’s global data ecosystem. In this role, you will lead the development of scalable batch and real-time data pipelines, enabling efficient data processing across Data Lakes and Data Warehouses. You will work at the intersection of data engineering, cloud platforms
- distributed systems, contributing to high-impact initiatives and driving engineering excellence. This role is ideal for someone who thrives in a fast-paced, collaborative environment, enjoys solving complex data challenges
- is passionate about building resilient, high-performance systems at scale. Role Overview As a Data Engineer II, you will design and develop scalable batch and near real-time data pipelines that power Mastercard’s analytics and operational systems. You will work across data engineering, cloud platforms
- distributed systems, building robust data solutions on top of Data Lakes and Data Warehouses. This role is highly hands-on and requires strong engineering fundamentals, with opportunities to influence design decisions and mentor junior engineers. You will contribute to building cloud-native data platforms, enabling reliable, high-performance data processing at scale. ________________________________________ Key Responsibilities
- Design and build scalable data pipelines and microservices using Java (Spring Boot), Spark, and cloud-native technologies
- Develop high-throughput, low-latency data processing systems for real-time and batch workloads
- Design and develop batch and near real-time data pipelines using Spark, Kafka, and Java-based frameworks
- Build and maintain ETL/ELT pipelines for structured and unstructured data
- Develop data processing solutions across Data Lakes and Data Warehouse environments
- Contribute to stream processing use cases (Kafka, and optionally Flink or Spark Structured Streaming)
- Ensure data quality, validation, and reliability across pipelines
- Optimize data processing workloads for performance and scalability
- Develop and deploy data pipelines on AWS, Azure, or GCP
- Leverage cloud-native services such as S3/ADLS/GCS, EMR/Databricks, BigQuery/Redshift/Snowflake
- Contribute to Infrastructure as Code (Terraform, CloudFormation, or equivalent)
- Build solutions with high availability, fault tolerance, and scalability
- Follow best practices for secure data processing and cloud resource utilization
- Follow best practices in coding, testing, and CI/CD pipelines
- Contribute to automation, monitoring, and observability of data pipelines
- Develop reusable components to improve engineering efficiency and consistency
- Participate in code reviews and design discussions
- Collaborate with architects, product teams, and cross-functional stakeholders
- Support production deployments and troubleshoot issues in distributed systems
- Contribute to a culture of continuous improvement and technical excellence
- Mentor junior engineers and share knowledge within the team Required Skills & Qualifications
- Strong proficiency in Java (JDK 8+), OOP/OOAD principles, Python is a plus. Experience with Spring Boot, REST APIs, Spring Security, Hibernate
- Hands-on experience with distributed systems, multithreading, and messaging systems (Kafka preferred)
- Hands-on experience with Apache Spark (Core, SQL, or Structured Streaming). Experience with Kafka or similar messaging/streaming platforms
- Understanding of ETL/ELT pipelines, batch and streaming architectures
- Familiarity with data formats (Parquet, Avro, ORC)
- Basic understanding of data modeling (star/snowflake schemas). Understanding of distributed systems and multithreading concepts
- Hands-on experience with at least one cloud platform: AWS, Azure, or GCP
- Experience using cloud storage and compute services (e.g., S3, ADLS, Databricks, EMR)
- Familiarity with containerization (Docker) and basic Kubernetes concepts
- Exposure to Infrastructure as Code tools is a plus
- Strong SQL skills and experience working with relational and analytical databases
- Exposure to Data Lakes and Data Warehousing platforms
- Familiarity with workflow orchestration tools (Airflow or similar)
- Experience with CI/CD tools (Jenkins, GitHub Actions, or similar)
- Good understanding of unit testing (JUnit or equivalent)
- Familiarity with monitoring tools (Splunk, Prometheus, Grafana, etc.)
- Awareness of secure development practices
- Experience with real-time processing frameworks -Spark Streaming, good to have knowledge on Apache Flink
- Exposure to performance testing tools (JMeter, Gatling)
- Familiarity with DevSecOps or SRE concepts
- Experience improving automation and developer productivity
- Strong problem-solving and analytical skills. Ownership mindset with ability to deliver independently
- Good communication and collaboration skills. Passion for learning new technologies and improving engineering practices
- Ability to work effectively in a fast-paced, global environment Education
- Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercard’s security policies and practices
- Ensure the confidentiality and integrity of the information being accessed
- Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.
Required skills
JavaSpring BootSparkKafkaETLELTAWSAzureGCPTerraformCloudFormationPythonSQLDockerKubernetes