Big Data Engineer
- New York, NY, USA
- Permanent, Full time
- GMS Advisors
- 15 Dec 17 2017-12-15
Big Data Engineer on new initiative to build a large consolidated audit trail system.
This company is a market structure technology provider focused on delivering high performance trading technologies and big data solutions to financial market participants.
Their technology handles approximately 5% of the daily volume in U.S. equities & 10% of Canadian equities on the back of their ultra low latency HFT offering and their advanced gateway technology for Broker-Dealers. They also provide exchange / matching engine technology, and currently run two U.S. equities dark pools. The third pillar of their business is a Financial Big Data platform which powers the SEC’s MIDAS data and analytics system and the recently awarded Consolidated Audit Trail system.
They are looking for a self-motivated contributor who is passionate about supporting and building high-quality tools and systems. This person will help develop a new product in the domain of big data analytics on financial data. The role involves combining cutting-edge open source and proprietary technology to support back-end systems written in Apache Spark, Python, C++ and front-end systems using React. The role will also help improve supporting tools and infrastructure, create fast build systems, and improve workflows, with a persistent emphasis on pushing the limits of scalability and performance.
• Develop automation infrastructure for deploying multiple environments
• Using dynamic programming languages, such as Python and Linux command Language to architect, implement and integrate data analytics tools for financial markets
• Proactively assisting other developers in diagnosing and solving issues with development and build environments
• Establishing and improving infrastructure in support of production, QA, and development systems
• Implementation of system monitoring infrastructure and tools, including dashboards
• Monitoring systems for issues in real-time, responding to them as they occur, and working proactively to minimize future issues
• Level 3 support
• 5+ years of experience building large scale big data applications
• Proactive with a high attention to detail and ability to manage multiple projects simultaneously
• Strong working knowledge of the Linux operating system, including basic system administration and a high degree of comfort working from the command line
• Experience with version control software and SDLC principles
• Strong programming skills, including experience with development in dynamic programming languages, strong knowledge of Python
• Experience with Open Source software including Puppet, Jenkins, Nagios, GNU make, cmake, Redmine, OpenLDAP, FreeRADIUS, BIND, collectd, Graphite, Grafana, Hypertable, and PostgreSQL
• Experience with containers, orchestration or CI/CD pipelines is a bonus
• Financial experience is not required