Submit Resume

Big Data Engineer

  • Texas, Dallas

  • 11/18/2026

  • Contract

  • Active

Job Description:

  • Job Summary:
    We are seeking a Senior Big Data Engineer with strong experience in designing, developing, and maintaining scalable data engineering solutions. The ideal candidate will have hands-on expertise in Python, Apache Spark, PySpark, Airflow, Kafka, and GCP data services. The role involves building and optimizing data pipelines, processing large datasets, integrating RESTful APIs, and implementing CI/CD practices to ensure reliable and efficient data workflows.

    Key Responsibilities:
    • Design, develop, and maintain scalable data pipelines using Python, PySpark, Scala, and Apache Spark.
    • Develop and manage data workflows and orchestration using Apache Airflow.
    • Implement data processing solutions using Spark SQL and Spark Streaming.
    • Integrate Apache Kafka for real-time data ingestion and streaming.
    • Write, optimize, and maintain complex SQL queries for large-scale data processing.
    • Work with relational and NoSQL databases to support data storage and processing requirements.
    • Develop and maintain data pipelines using GCP services, including BigQuery and Dataproc.
    • Integrate RESTful APIs into data pipelines for data exchange and processing.
    • Implement and maintain CI/CD pipelines using GitLab CI.
    • Collaborate with cross-functional teams to deliver reliable, scalable, and high-performance data engineering solutions.

    Required Qualifications:
    • Strong hands-on experience with Python, PySpark, Scala, and Apache Spark.
    • Experience with Apache Airflow for workflow orchestration.
    • Proficiency in Spark SQL and Spark Streaming.
    • Experience working with relational and NoSQL databases.
    • Strong knowledge of Apache Kafka and real-time data processing.
    • Ability to write and optimize complex SQL queries.
    • Experience with Google Cloud Platform (GCP), particularly BigQuery and Dataproc.
    • Hands-on experience integrating RESTful APIs into data pipelines.
    • Experience with CI/CD implementation using GitLab CI.

    Preferred Qualifications:
    • Experience with backend development technologies, including Java and Spring Boot.

.

.

.