Amazon Redshift,Apache Spark,Databricks
• Experience in building data pipelines using spark (Scala) on Databricks
• Development experience in the Amazon Cloud Environment AWS (S3, EMR, Databricks, Amazon Redshift, Athena.)
• Experience in working with REST APIs
• Ability to perform data manipulations, load, extract from several sources of data into another schema.
• Ability to work with multiple file formats (JSON, xml, rdf, reports, etc.) and analyze data, if required for further processing
• Experience in DevOps is nice to have but knowledge about DevOps is required
• Understanding of core AWS services, and basic AWS architecture best practices.
• Snowflake experience will be a plus.
• Ability to understand requirements and changes to requirements