Data Engineer

Job Title: Data Engineer: Streaming & Sensor Data
Job Location: Colorado Springs, CO - Denver, CO, also considered
Eligibility/Clearance: Top Secret

Job Description: We are seeking a Data Engineer to build the data
foundation for our machine learning applications. You will integrate
time-series measurements, including radar and OPIR data, from real-time
streams, application databases, and systems with varying extraction
capabilities. You will collaborate with senior technical staff and the
ML team to characterize available data, establish reliable pipelines,
and deliver cleaned, standardized datasets. You will help select
technologies and develop an architecture based on measured workloads,
application needs, and operational requirements. Our development
environment is primarily Python, with AWS as the preferred cloud
platform.

In this role, you will:
-Understand our data sources. Investigate available interfaces, message
formats, historical coverage, data volumes, arrival rates, and quality
limitations. Document findings to guide implementation and capacity planning.
-Build ingestion pipelines. Develop maintainable workflows for streaming
and historical data across databases, APIs, files, and application-specific interfaces.
Implement configurable filtering to capture the messages and measurements needed
downstream.
-Shape the data architecture. Evaluate processing, storage, and access
options against representative workloads, reliability needs, interoperability,
operational complexity, and cost. Document decisions and support the architecture’s evolution as
requirements become clearer.
-Deliver trusted datasets. Establish consistent schemas, timestamps,
units, identifiers, and source metadata while preserving measurement meaning. Implement
validation and traceability, and define dependable access methods with the ML team.
Support future feature, labeling, and training-data workflows.
-Operate reliable systems. Build tested, documented pipelines with
repeatable deployment, monitoring, and appropriate access controls. Handle
duplicate, missing, delayed, and out-of-order data, and implement recovery and reprocessing
procedures.

We’re looking for someone with:
-Experience designing, implementing, and operating production streaming
and batch data pipelines and their supporting storage systems.
-Strong software engineering skills, including maintainable code,
automated testing, version control, and deployment.
-Practical understanding of streaming behavior, including filtering,
consumer progress, buffering, failure recovery, and replay.
-Experience organizing time-series, telemetry, sensor, or similarly
complex data, with an understanding of schema evolution, partitioning, and efficient access
patterns.
-Demonstrated ability to investigate unfamiliar systems, evaluate
technical tradeoffs, and communicate recommendations while balancing reliability, performance,
maintainability, and cost.

Additional experience that would be valuable:
-Python-based data processing and AWS or comparable cloud
infrastructure.
-Radar, OPIR, or other sensor measurement data.
-Workflow orchestration, containers, infrastructure as code, and
automated delivery.
-Dataset versioning, feature generation, labeling, or training-data
preparation.

Arena Technical Resources, LLC, (ATR) is an Equal Opportunity Employer
(EOE) who will provide equal employment opportunity to employees and
applicants for employment without regard to  race, ethnicity, religion,
color, sex, pregnancy, national origin, age, veteran status, ancestry,
sexual orientation, gender identity or expression, marital status,
family structure, genetic information, or mental or physical disability.