Senior Data Engineer FinStream Analytics New York

Senior Data Engineer – Real-Time Analytics Platform

About Our Company

We are a leading technology innovator in the financial services sector, building cutting-edge platforms that process billions of data points daily. Our mission is to transform how institutions interact with financial markets through real-time analytics, predictive modeling, and scalable data infrastructure. With a global presence and a commitment to engineering excellence, we empower our clients with actionable insights that drive decision-making at market speed.

Role Overview: Your Impact on Our Mission

We are seeking a Senior Data Engineer to architect, build, and optimize the next generation of our real-time analytics platform. This is not just a maintenance role; you will be a key contributor in designing systems that handle massive-scale, low-latency data pipelines critical to our core business functions. You will work directly with quantitative analysts, data scientists, and product teams to translate complex business requirements into robust, scalable data solutions. If you are passionate about the challenges of streaming data, distributed systems, and creating engineering foundations that others build upon, this role offers a unique opportunity for significant technical ownership and impact.

Key Responsibilities: What You Will Build and Own

In this role, you will be entrusted with responsibilities that are central to our data strategy:

• Design and Development: Architect, develop, and maintain high-volume, low-latency real-time data pipelines using technologies like Apache Kafka, Apache Flink, and Spark Structured Streaming. You will ensure data quality and consistency from ingestion to consumption.

• Cloud Infrastructure & Optimization: Build and manage scalable data infrastructure on AWS (EMR, Glue, Kinesis, Redshift, S3). Your mandate includes continuously optimizing pipelines for performance and cost-efficiency, implementing monitoring, and automating operational processes.

• Cross-Functional Collaboration: Serve as the technical authority on data engineering for product and analytics teams. You will gather requirements, propose architectural solutions, and mentor junior engineers, fostering a culture of best practices in data handling.

• Data Modeling & Governance: Design and implement efficient data models for both real-time and batch processing contexts. You will champion data governance principles, ensuring security, compliance, and metadata management across our data ecosystem.

• Innovation & Problem-Solving: Proactively identify bottlenecks and innovate on our existing stack. You will troubleshoot complex issues in production and research new technologies to keep our platform at the forefront of the industry.

Essential Qualifications: The Expertise You Bring

Required Technical Skills & Experience:

• 5+ years of professional data engineering experience with a proven track record of building and deploying large-scale data processing systems.

• Expert-level proficiency in Python and/or Java/Scala, with deep understanding of software engineering principles, concurrency, and performance tuning.

• Hands-on experience with modern big data and streaming technologies: We use Apache Kafka, Flink, Spark, and Airflow extensively. Demonstrable experience with at least two of these is required.

• Deep cloud platform expertise, preferably with AWS services (EMR, Glue, Kinesis, Redshift, S3, Lambda, IAM). Certifications are a plus.

• Strong SQL skills and experience with data modeling for both transactional and analytical workloads (Star Schema, Data Vault).

• Experience with infrastructure-as-code tools such as Terraform or CloudFormation and containerization with Docker and Kubernetes.

Required Professional Competencies:

• Architectural Acumen: Ability to design systems that are not only functional but also scalable, maintainable, and fault-tolerant. You think in terms of trade-offs and long-term implications.

• Ownership & Initiative: You are a self-starter who drives projects to completion, takes ownership of outcomes, and thrives in an environment of autonomy and accountability.

• Collaborative Communication: Excellent verbal and written communication skills. You can explain complex technical concepts to non-technical stakeholders and collaborate effectively across disciplines.

• Problem-Solving Mindset: A systematic approach to debugging and solving intricate, open-ended problems under pressure.

Preferred Qualifications: What Will Make You Stand Out

• Experience in the financial services, fintech, or capital markets domain, with an understanding of market data, tick data, or risk analytics.

• Knowledge of data lakehouse architectures (Apache Iceberg, Delta Lake) and real-time database technologies.

• Experience implementing advanced data observability, quality, and lineage frameworks.

• Contributions to open-source projects or a public portfolio (GitHub) showcasing relevant work.

• Experience in a fast-paced, agile product development environment.

Our Technology Stack: What You Will Work With

• Streaming & Processing: Apache Kafka, Apache Flink, Apache Spark (Structured Streaming, PySpark), AWS Kinesis

• Orchestration & Workflow: Apache Airflow, Prefect

• Cloud & Infrastructure: AWS (EMR, EC2, S3, Redshift, Glue, Kinesis, Lambda, IAM), Terraform, Docker, Kubernetes

• Databases & Storage: PostgreSQL, Redis, Amazon Redshift, S3, Parquet/ORC formats

• Programming & Scripting: Python (Pandas, NumPy, FastAPI), Scala, SQL, Bash

• Monitoring & DevOps: Datadog, Prometheus, Grafana, GitLab CI/CD

Why Join Us? The Value We Offer You

We believe in supporting our team with comprehensive benefits and a culture that promotes growth and well-being.

Compensation & Financial Benefits:

• Highly competitive salary and annual performance bonus

• Equity/Stock Option participation, aligning your success with the company’s

• 401(k) plan with company match and comprehensive financial planning resources

Health & Well-being:

• Premium medical, dental, and vision insurance with multiple plan options (covered at 90% for employees and 75% for dependents)

• Flexible Spending Accounts (FSA) and Health Savings Accounts (HSA)

• Life and Disability Insurance (company-paid)

• Mental health support including counseling services and subscriptions to wellness platforms

Work-Life Integration & Flexibility:

• Hybrid Work Model: We value in-person collaboration and offer flexibility. This role is based in our New York City office with an expectation of 3 days per week in-office, with flexibility as needed.

• Unlimited Paid Time Off (PTO) with a culture that encourages you to use it

• Paid parental leave (16 weeks for primary caregivers, 8 weeks for secondary)

• Flexible working hours outside of core collaboration hours (10 AM – 4 PM ET)

Career Growth & Development:

• Clear career progression paths in both technical and leadership tracks

• Annual learning and development stipend ($3,000) for conferences, courses, and certifications

• Regular “Innovation Sprints” and dedicated time for exploring new technologies

• Access to industry-leading experts and opportunities for mentorship and speaking engagements

Culture & Office Perks:

• Work in a collaborative, low-ego environment where your ideas are valued

• Modern office space in Manhattan with panoramic views, standing desks, and collaboration areas

• Fully stocked kitchens with snacks, beverages, and daily catered lunch

• Regular team events, hackathons, and social outings

Our Hiring Process: What to Expect

We’ve designed our process to be thorough, respectful of your time, and mutually evaluative.

1. Initial Application Review: Our Talent Team reviews your application against the core requirements.

2. Recruiter Screen (30-minute video call): A preliminary conversation to discuss your background, the role, and answer your initial questions.

3. Technical Deep-Dive Interview (60-minute video call): A session with a senior team member focusing on your past projects, system design thinking, and hands-on problem-solving.

4. On-Site/Virtual Loop (3-4 hours):

   • Coding & Systems Design: A practical coding exercise and a broader system architecture discussion.

   • Collaboration & Culture Interview: A behavioral interview focusing on teamwork, mentorship, and navigating complex projects.

   • Meeting with the Hiring Manager: A deeper dive into your experience and a discussion about team fit and vision.

5. Final Offer & Conversation: We extend a formal offer and host a final conversation to answer any remaining questions before you make your decision.

• Total process typically takes 2-3 weeks from initial screen to offer.

How to Apply: Your Next Step

If you are excited by the challenge of building mission-critical data platforms, we encourage you to apply.

Please submit the following:

1. Your updated resume or LinkedIn profile.

2. A brief cover letter or note (in the application form) explaining why you are interested in this specific role and what unique perspective you would bring to our team.

3. (Optional but highly valued) A link to your GitHub profile, portfolio, or a description of a relevant project you are proud of.9