Job Description

Overview Iron EagleX (IEX), a wholly owned subsidiary of General Dynamics Information Technology, delivers agile IT and Intelligence solutions. Combining small-team flexibility with global scale, IEX leverages emerging technologies to provide innovative, user-focused solutions that empower organizations and end users to operate smarter, faster, and more securely in dynamic environments Responsibilities Job Description: We are seeking a Data Engineer SME to design, build, and maintain data collection, processing, and integration pipelines that centralize and structure data from various web-based applications. You will work closely with software developers, data analysts, and product teams to ensure seamless data flow, high-quality data storage, and efficient retrieval methods. This role is ideal for someone passionate about data-driven decision-making and optimizing data aggregation processes while exploring and identifying right-sized technologies and computing resources. This is not a Data Science, Big Data, ML-Ops, or AI-related role. Instead, this position focuses on wrangling disparate data sources into unified, consumable data APIs for internal and external stakeholders. This is a hybrid position in Crystal City, VA (3 days onsite, 2 remote). Job Duties Include (but not limited to): * Design, develop, and implement scalable data pipelines and ETL processes using Apache Airflow, with a focus on data for AI applications.
* Develop messaging solutions utilizing Kafka to support real-time data streaming and event-driven architectures.
* Build and maintain high-performance data retrieval solutions using ElasticSearch/OpenSearch.
* Implement and optimize Python-based data processing solutions.
* Integrate batch and streaming data processing techniques to enhance data availability and accessibility.
* Ensure adherence to security and compliance requirements when working with classified data.
* Work closely with cross-functional teams to define data strategies and develop technical solutions aligned with mission objectives.
* Deploy and manage cloud-based infrastructure to support scalable and resilient data solutions, including AWS S3.
* Optimize data storage, retrieval, and processing efficiency, focusing on big data and the use of Parquet files.
* Utilize Trino for distributed SQL query processing.
* Leverage OpenMetaData for data discovery and governance.
* Implement APIs using FastAPI for streamlined data interaction.
* Ensure proper implementation of PKI's & ACCM's.
* Integrate DIAS solutions within the data pipeline framework. Qualifications Required Skills & Experience: * Extensive experience with Apache Airflow for workflow orchestration.
* Strong programming skills in Python.
* Familiarity with OPA and Trino.
* Experience with ElasticSearch/OpenSearch for data indexing and search functionalities.
* Understanding of vector databases, embedding models, and vector search for AI applications.
* Expertise in event-driven architecture and microservices development.
* Hands-on experience with cloud services, particularly AWS S3 and MinIO, including data storage and compute resources.
* Strong knowledge of Reggo and its application within data environments.
* Understanding of big data technologies and file formats like Parquet.
* Expertise in data pipeline orchestration and workflow automation.
* Working knowledge of Linux environments and database optimization techniques.
* Strong understanding of version control with Git.
* Familiarity with OpenMetaData for data management.
* Experience implementing FastAPI for building scalable APIs.
* Due to US Government Contract Requirements, only US Citizens are eligible for this role. Education & Certifications: * Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience). Advanced degrees are a plus. Security Clearance: * An active TS/SCI security clearance is REQUIRED, and candidates must have or be willing to obtain a CI Poly. Candidates without this clearance will not be considered. Equal Opportunity Employer / Individuals with Disabilities / Protected Veterans

Job Tags

Contract work, Remote work,

Similar Jobs