Data Engineer- Manager

Company profile

  State Street is one of the largest custodian banks, asset managers, and asset intelligence companies in the world. From technology to product innovation we’re making our mark on the financial services industry. For more than two centuries, we’ve been helping our clients safeguard and steward the investments of millions of people. We provide investment servicing, data & analytics, investment research & trading and investment management to institutional clients.

Key Responsibilities & Experience Range

• Software Development:
  • Specialize in seamless integration of various technologies to ensure cohesive application development.
  • Design, develop, test, and maintain data pipelines using Python, PySpark, and SQL.
  • Design, build, and manage Databricks workflows, pipelines, and data ingestion solutions using native Databricks features and platform capabilities
  • Develop containerized applications and jobs using Docker.
  • Write shell scripts to automate operational tasks and improve data workflow efficiency.
• Database management:
  • Demonstrate profound understanding of relational databases.
  • Implement design principles for efficient data storage and retrieval
  • Work with structured and unstructured datasets, applying strong data modeling concepts.
• Testing and Troubleshooting:
  • Ensure code quality by writing unit tests, automation scripts, and participating in code reviews.
  • Troubleshoot production issues, analyze logs, and contribute to fast resolution.
• Ownership and Collaboration:
  • Collaborate with cross functional teams to define requirements and provide scalable engineering solutions.
  • Develop oneself into a Subject Matter Expert (SME) on Technical & Domain areas.
  • A self-starter with excellent organizational, time and relationship management skills.
  • Able to lead projects with minimal guidance and should show complete ownership of deliverables.
• Delivery and Documentation:
  • Design and maintain CI/CD pipelines using tools such as Jenkins, Artifactory, Bitbucket/GitHub.
  • Develop containerized applications and batch jobs using Docker.
  • Review existing processes and procedures and recommend improvements.
  • Prepare and maintain technical and functional documentation as required.

 Skills & Knowledge

Qualification & Certifications

• Bachelor’s degree or equivalent related work experience
• As a key member of our development group, you will be responsible for designing, implementing and maintaining  scalable and high-performance application

Primary Skills (Must Have)

• 7+ years of experience in technology and/or data functions in financial services or consulting industry.
• Hands on experience with establishing data strategies and data transformation implementations with in a fast paced ,dynamically growing sector in the industry.
• Hands-on experience with Databricks and its ecosystem, strong understanding of Spark architecture, and proven ability to build and manage workflows, data pipelines, and applications on Databricks.
• Hands-on experience in developing, deploying, maintaining, and troubleshooting Python-based applications (e.g., Streamlit) on Databricks or similar cloud-based platforms.
• Hands on development in Python, PySpark, SQL, Shell Scripting.
• Exposure to cloud platforms, preferably AWS (S3, EMR, Lambda, ECS/EKS, Glue, IAM, etc.).
• Good understanding of data modeling, ETL frameworks, and distributed data processing
• Demonstrate experience in CI/CD tools such as Artifactory, Jenkins, Bitbucket etc.

• Good to have knowledge in Snowflake, AWS Iceberg / Apache Iceberg

• Working experience in all the phases of the project
• Strong written and verbal communication skills required to liaise with the business, developers, QA, and operations

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement