Data Analyst / Arhcitect
- Highly competitive
- Mumbai, Maharashtra, India Mumbai Maharashtra IN
- Contract, Full time
- Pure Hong Kong , EA Licence No: 12S5954
- 14 Sep 18 2018-09-14
My client is a cutting edge AI powered financial markets trading platform with a global presence. This is a 3-month assignment to build out the company's Data Lake from existing MySQL Databases. There is an option to convert to a permanent contract for a great candidate. Immediate start available.
The platform processes all available structured (market data, financial data etc.) as well as unstructured data (news, analyst calls, annual reports, investor wires and press releases). These different data sources are sourced from many premium providers who use different technologies:
All the data is currently ingested into MySQL DBl.
The assignment is to aggregate all the structured data from all providers into an Universal Data Lake that will be built on Amazon Aurora, a MySQL and PostgreSQL compatible relational database.
- Experience in managing terrabytes scale data lakes
- Have significant development experience in technologies including Redshift, MySQL, or PostgreSQL.
- Demonstrated strength in data modeling, ETL development, and data warehousing
- Experience using big data technologies (e.g. Redshift, S3, Hive, Hbase, Spark, EMR, etc.)
- Knowledge of distributed systems as it pertains to data storage and computing
- Knowledge of performance tuning techniques for heterogeneous database system.
- Experience with database technical architecture and design for applications
- Experience with Data Conversion Strategies on various databases.
- Experience with Database Partitioning Strategies on various databases
- Bachelor's degree in Computer Science, Computer Engineering, Data Engineering, or a related field
- Knowledge of AWS technologies, especially Aurora
- 5+ years of working with financial markets data – market or fundamentals data
- Redshift/PostgreSQL/similar experience in a 24x7 production environment.
- Experience with Redshift, Postgres, or similar database system metrics and optimization strategies.
- Expertise in one or more of: Python, Perl, Bash or other scripting language.
- Entrepreneurial spirit, with a track record of delivering results
- Handle competing priorities in a fast-paced and demanding environment
- Effective communicator in both written and spoken English
- Master's degree in Computer Science, Computer Engineering, Data Engineering, or a related field