Director of Data Engineering

  • Competitive
  • New York, NY, USA
  • Permanent, Full time
  • S&P Global
  • 21 May 19

Director of Data Engineering

JobDescription :
The Director of Data Engineering will lead the engineering activities that will form the
new Platts' data platform, constructing innovative systems that will unite the collection,
parsing, enriching, linking and delivering mechanisms that form the large portfolio of
energy and pricing data for Platts. The Director of Data Engineering will lead the
development and execution of highly complex and large-scale data structures and
pipelines that link and enrich data, generating insights whose value are exposed via
APIs, analytical models and artificial intelligence, thus increasing the value of our data
sets for internal teams and customers of Platts' applications.
In short, the director of Data Engineering will be responsible for delivering on a
multi-year data strategy that will catapult Platts as one of the leading data
companies in the market.

The ideal candidate is an innovative, current practitioner with a strong data engineering
background, who will lead multiple data engineering teams globally and will work closely
with operations, technology, product management, analysts and other teams located in
our offices in London, NY, Denver, Dallas and India as well as work with the S&P Global
Chief Data Officer in shaping and refining the corporate standards to ensure they meet
the needs of Platts' data users and customers.

As Director of Data Engineering you will be responsible for the following facets of data
engineering:

Data Strategy & Architecture:
o Data Models (Relational, Time Series, Document & Graph Models)
o Knowledge/Semantic Models (Taxonomies, Ontologies, Reference &
Master Data, Knowledge Graphs)
o Operational Models (Multi-tenant, Cloud, Batch & Streaming models)
o Storage Models (NoSQL, Relational, Graph, Time Series, in-memory)
• Data Collection & Ingestion
o Data Wrangling
o Cleanup & Deduplication
o Data Scraping
• Data Processing & Automation
o Data Pipelines (ETL & ELT Models)
o Event driven Data Processing
o Change Data Capture
o Batch & Stream operations
o Data Quality Automation (using rules engines and statistical methods)

Data Access & Delivery
o APIs
o Analytics & Data Science
o Visualizations
o Data Exports
o Streams / Alerts / Events
o Channel Partner distributions
• Data Linking & Enrichment
o Data Tagging (Automated & Manual)
o Machine learning based data linking

Infrastructure & Operations
o Data Scalability
o Availability
o Performance
o Observability (Monitoring, Logging, Alerting)
o Telemetry
o Security
• Data Governance
o Standards
o Risk Management
o Access Policies
o Data Quality
o Data Lineage & Auditing
o Corporate Governance
o Continuous improvement

Additional Responsibilities:
Development Management
• Partner with product management and co-lead an agile transformation
• Partner with customer-facing peers to create and deliver on multi-year roadmaps
• Manage and build teams off-shore
• Lead the transformation from multiple applications/tools into a few platforms
• Migration legacy applications onto the platform
• Identify and consolidate duplicate product functionality, infrastructure
environments and teams
• Automate manual tasks

Agile Responsibilities
• Provide constant coaching to the agile teams in the program
• Provide mentoring and guidance to team members
• Have effective 1:1's with your direct reports
• Ensure managers and their team members work at capacity to ensure deadline
are meet
• Ensure agile teams estimate development effort by breaking down components
into work items
• Review their team's Scrum and ScrumBan boards weekly and ensure the teams
operate efficiently and meet sprint goals
• Continually work on process improvement

Technology Operations
• Work closely with application support to ensure escalated items are addressed
accordingly
• Ensure production operations of 99.99% uptime
• Partner with infrastructure to efficiently manage the AWS environments.
Implement elastic infrastructure to control costs.
• Lead technology audit compliance based on regulators and SOC2

Qualifications
• Degree in computer science or related quantitative field of study. Masters degree
is preferred
• 10+ years of experience in software industry with emphasis on Data Engineering
• Experience in software development with a major modern language (e.g. Java,
Scala, Python, etc.)
• Strong understanding of data structures, algorithms and data design.
• Expertise in data engineering topics (APIs, automation, big data, open source
packages, and secure systems architectures)
• Experience in AWS data storage, processing and analytical services.
• Experience with stream-processing systems, such as Kafka, Spark Streaming,
Kinesis.
• Experience with data lake/warehouse technologies (e.g. Athena, Redshift, EMR)
and distributed NoSQL databases (e.g. DynamoDB, MarkLogic).
• Experience with semantic web technologies (Ontologies, Taxonomies,
Knowledge Graphs) and Graph theory in general.
• Familiarity with data science techniques or machine learning.
• Good knowledge and experience in Agile development
• Proven technical abilities, but must be capable of communicating complex
analyses effectively
• Capable to lead multiple work streams simultaneously in a fast-paced
environment and partner with multiple business stakeholders
• Most of all, it requires an individual who is comfortable striking a balance
between cutting-edge and pragmatic solutions.