Associate, Data Engineer

  • Competitive
  • New York, NY, USA
  • Permanent, Full time
  • Morgan Stanley USA
  • 25 Apr 19

Associate, Data Engineer

Morgan Stanley Services Group Inc. seeks an Associate, Data Engineer in New York, New York

Evaluate Hadoop infrastructure requirements and design/deploy solutions including high availability, big data clusters, and elastic load tolerance. Develop automation as well as install and monitor Hadoop and data streaming ecosystem components. Dig deep into performance, scalability, capacity, and reliability problems to resolve issues. Data curation, prep, ETL (extract, transform, load), and analytics development of the many data sources ingested into streaming platform and Hadoop. Troubleshoot and debug Hadoop ecosystem run-time issues. Provide developer and operations documentation. Run proof of concept projects with customers.


Requirements :

Requires a Bachelor's degree in Information Technology, Computer Engineering, Computer Science, or related field of study and five (5) years of experience in the position offered or five (5) years of experience as a Technology Consultant, Software Engineer, or related occupation. Requires five (5) years of experience with: Java/J2EE development; Junit testing framework; and Eclipse development tools. Requires four (4) years of experience with: Linux/Unix system administration; Maven software project management; and distributed version control systems including Git and SVN. Requires three (3) years of experience with: Hadoop ecosystem including Map Reduce, Hive, SQOOP, and PIG. Requires two (2) years of experience with: Hadoop administration, distributed cluster management and operations with MapR, or Cloudera Hadoop distributions using Yarn for cluster Resource Management; building and scaling Hadoop based or Unix hosted database infrastructure for an enterprise including software, network, and storage; and NoSQL databases. Requires one (1) year of experience with: IntelliJ development tools, development on distributed execution and data streaming platforms including Scala, Apache Spark, and Apache Kafka; and ElasticSearch distributed search and analytics engine; and Hadoop administration.

Qualified Applicants :
To apply, visit us at Scroll down and enter 3129130 as the "Job Number" and click "Search jobs." No calls please. EOE
Morgan Stanley Use Only: *LI-DNI