Senior Data Engineer

Team Overview

We're looking for a Senior Data Engineer to join our team working on products that power real-time analytics, business intelligence, and trading solutions across securities finance markets. This role blends core data engineering with hands-on exposure to machine learning, with  work focused on building and maintaining scalable data pipelines as well as supporting the development and deployment of AI/ML capabilities across our platform.

You'll contribute to products like EquiLend Competitive Bid and DataLend, helping shape how we deliver analytics at scale. If you have solid experience with streaming or batch data processing and want to extend your skills into ML-focused work, we'd like to hear from you.

What you’ll do

• Design, build, and maintain scalable ETL data pipelines using Python on Spark to process large datasets

• Develop and manage data workflows to support batch and real-time processing needs

• Deploy, monitor, and maintain data pipelines in production, ensuring robustness and reliability

• Build data integration solutions that connect disparate data sources into unified systems

• Support the development, monitoring, and enhancement of machine learning models, including models such as Predicted Short Interest

• Contribute to initiatives such as the DataLend Trade Analytics project

• Collaborate with analytics, product, and business teams to refine data models and support BI tools

• Monitor, troubleshoot, and optimise performance for ETL jobs and streaming pipelines to keep latency low and throughput high

• Improve data quality, reliability, and observability through monitoring and alerting practices

• Implement data governance practices, including validation checks, versioning, and compliance with data security requirements

• Partner with global engineering teams to standardise data processing frameworks and share best practices

What we need

• 5+ years of hands-on experience in data engineering, with a focus on building and optimising batch or streaming pipelines

• Strong proficiency in Python/PySpark or other object-oriented languages such as Scala or Java

• Advanced SQL skills for creating complex queries, database optimisation, and working with relational and non-relational databases

• Experience with stream processing technologies such as Kafka, Spark Streaming, AWS Kinesis, or Flink

• Hands-on experience with scheduling and orchestration tools like Airflow, Luigi, or similar

• Familiarity with building and deploying cloud-native solutions using AWS, GCP, or Azure

• Knowledge of data lake and data warehouse architectures, including tools like Snowflake or Redshift

• Strong understanding of distributed data processing concepts such as partitioning, sharding, and fault tolerance

• Experience with Git and CI/CD pipelines for automated deployment of data solutions

• Working knowledge of machine learning concepts, with experience supporting or maintaining ML models in production

• Familiarity with Python ML libraries such as pandas, NumPy, and scikit-learn

• A foundational understanding of data security, governance, and compliance requirements

Why EquiLend?

• Be recognised for your contribution through a discretionary annual cash bonus, linked to individual and company performance.

• Work flexibly with up to 100 remote working days per year.

• Take time to recharge with generous annual leave, plus public holidays.

• Access comprehensive health insurance for you and your dependents from your first day.

• Receive personal accident insurance from the start of your employment.

• Connect with colleagues at an annual company-sponsored team retreat, hosted at a new location each year.

• If relocating from more than 40 miles away, receive hotel accommodation and travel ticket reimbursement.

About EquiLend EquiLend is a global financial technology firm at the center of the securities finance industry. Since 2001, we have built the infrastructure that financial institutions rely on to trade, manage, and report. This includes products such as our NGT trading platform, Post Trade Suite, DataLend market intelligence, RegTech solutions, and EquiLend Clearing Services.

We operate across North America, EMEA, and Asia-Pacific, working with banks, broker dealers, asset managers, and custodians to bring greater efficiency, transparency, and automation to the securities finance lifecycle. Our teams combine deep market expertise with a focus on building technology that works at scale, under real-world conditions, for some of the world's most demanding financial institutions.

EquiLend is Great Place to Work Certified™ in the U.S., UK, Ireland, and India, and has been recognized as Global Data Provider of the Year and Regulatory Solution of the Year at the Securities Finance Times Industry Excellence Awards 2025.

Diversity & Inclusion We know that the best solutions come from teams that bring different perspectives, backgrounds, and ways of thinking to the table. At EquiLend, we are committed to building a workplace where everyone feels they belong and aim to reflect this in how we hire, develop, and support our people.

We are actively working to ensure our teams are as diverse as the global markets and clients we serve. That means creating an environment where different viewpoints are sought out and where people can do their best work regardless of background.

If you require any reasonable adjustments or accommodations at any stage of the recruitment process, please let us know and we will work with you to make it happen.

#LI-Hybrid