Developer - Big Data

  • Competitive
  • Shanghai, Shanghai Shi, China
  • Permanent, Full time
  • Morgan Stanley
  • 15 Jan 18 2018-01-15

See job description for details


The Vault is the firms centralized security data lake for log collection and analytics. It serves as a key platform for fulfilling regulatory requirements via log retention, and also meets Cyber and Insider Threat requirements by providing big data analytics capabilities. This platform not only receives more than 200 log flows, and about 20TB data per day, but also provide interfaces for users to query against parsed/raw data on demand and for downstream applications to consume near real time log data for analytics purposes. This is a developer role, but the candidate also needs to have infrastructure experience working with big data platforms and components of the Hadoop ecosystem. The role encompasses the following responsibilities:

- Understand, maintain and improve existing big data systems/architecture
- Design, develop and deploy new server-side services or applications
- Explore cutting edge big data technologies and choose appropriate solution to solve problems
- Work closely with colleagues in global team and members from other technology teams to collaborate on delivering integrated solutions
- Provide technical support to operations team and clients

#LI-JZ2


Qualifications:


- relevant experience: 4+ years
- Proficient with Java/Python/Scala/Perl or other languages
- clear understanding of object-oriented fundamentals
- experience with big data technologies, like Kafka/Elasticsearch/Hbase/Spark/Hadoop and etc.
- experience developing, delivering and supporting software for Linux (or other UNIX ) operating system
- Knowledge of multi-threading and distributed computing
- familiar with software development lifecycle
- able to express ideas clearly and concisely, both in written and verbal English
- quick learner, self-motivated, maintain high standard of professional conduct and practice

#LI-JZ2