Loading...
Загрузка
Назад

Spark Cloud Data Engineer (AWS, remote)

Заработная плата: 5 200 – 7 000 долларов
Опыт работы: нет опыта
Занятость: полная занятость
Откликнуться

Описание вакансии

ClearScale (headquartered in San Francisco, California, USA) - AWS Premier Consulting Partner has been offering a full range of professional cloud computing services for over 10 years, including architecture design, DevOps automation, refactoring and cloud-native applications development, integration, migration, solving all sorts of security issues (from just a security check to preventing cyber-attacks) and 24/7 technical support using the best advanced technologies.

The list of our customers is diverse: from government companies (ClearScale is an official cloud partner of the State of California) and educational institutions (University of California, San Francisco) to well-known global brands (IBM, Samsung, GoPro, HP, Conde Nast, Carl Zeiss, etc.) The number of satisfied customers has been well over 850, some of which can be found on the company's website in the Case Studies section.

  • We were the third company to gain a new AWS competence: Applied AI and Machine Learning Operations (MLOps). Less than 15 partners have this competency!
  • We have confirmed status in the Database Freedom program (total less than 20 companies in)
  • Proven expertise as a Managed Services Provider (total less than 16 companies in). This means that ClearScale can perform a full cycle consultancy and service: from audits, system or software development to the 24/7. You can read more about us on the company page - Managed Services.

Since the very foundation of the company, we work 100% remotely from various cities and countries. We work on a high trust basis within the company, therefore we do not monitor work being done via taking screen captures, webcams or log keyboard typing, as many other companies do. The professional reputation of our employees is of the highest value.

Job Overview

ClearScale is looking for an AWS Spark Cloud Data Engineerto participate in building cost-efficient scalable data lakes for the wide variety of customers, from the small startups to the large enterprises.

Usually, our projects fit into one of the categories (but not limited to them):

  • Collect data from the IoT edge locations, store it in the Data Lake, orchestrate processes to ETL that data, slice it into the various data marts. Then put those data marts into the machine learning or BI pipelines
  • Optimize pre-existing Spark/Elasticsearch/Solr clusters, find the bottlenecks in the code and/or architecture, plan and execute improvements
  • Build a data deliver pipeline to ingest high volume of the real-time streams, detect anomalies, slice into the window analytics, put those results in the Elastic search system for the further dashboard consumption

Responsibilities

  • Analyze, scope and estimate tasks, identify technology stack and tools
  • Design and implement optimal architecture and migration plan
  • Develop new and re-architecture solution modules, re-design and re-factor program code
  • Specify the infrastructure and assist DevOps engineers with provisioning
  • Examine performance and advise necessary infrastructure changes
  • Communicate with client on project-related issues
  • Collaborate with in-house and external development and analytical team
  • Write ETL/ELT jobs

  • Design Data Lake House pipelines

Description of candidate requirements

ClearScale expects successful candidate to have most of the following qualifications and skills (not necessary all have to be presented):

  • We are looking for a candidate with 3+ years of experience in Data, Cloud or Software Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
  • Hands-on experience designing efficient architectures for high-load enterprise-scale applications or ‘big data’ pipelines
  • Practical experience in implementing of micro-services architectures (using Java, Python or Scala stack)
  • Hands-on experience with message queuing, stream processing and highly scalable ‘big data’ stores
  • Advanced knowledge and experience working with SQL and noSQL databases
  • Proven experience in re-design and re-architecting of the large complex business applications
  • Strong self-management and self-organizational skills
  • Successful candidates should have experience with any of the following software/tools (not all required at the same time):

    • Java/Scala/Python - strong knowledge
    • Big data tools: Kafka, Spark, Hadoop (HDFS3, YARN2,Tez, Hive, HBase)

    • Stream-processing systems: Kinesis Streaming, Spark-Streaming, Kafka Streams, Kinesis Analytics
    • NoSQL DBs: Kassandra, DynamoDB, Mongo
    • AWS cloud services: EMR, RDS, MSK, Redshift, DocumentDB, Lambda
    • Graph databases development and optimization (Neo4j, Neptune, Tinan)

    • Message queue systems: ActiveMQ, RabbitMQ, AWS SQS
    • Federated identity services (SSO): Okta, AWS Cognito
  • Valid AWS certificates would be a great plus

What we offer

# 1 Fair wage

  • Fixed hourly rate in USD
  • Full-time, 40 hours per week contract. It’s your main and permanent place of work, not a freelance job
  • Payments every 2 weeks
  • Annual Hourly Rate Review: received client's appreciation, obtained an AWS certificate, saved the world?The reward is based on facts, not on a personal attitude
  • Reference program: many engineers joined the team by the recommendation from their friends and former colleagues and we have a reward for this.

# 2 Independence

  • from the office location or office conditions because it's only up to you where you want to work from - home, office, co-working or outdoors in the woods. The only requirement - fast and reliable internet. At the same time, our engineers do not work evenings or weekends: we focus on your local timezone
  • from any annoyance or big team buzz - we are the community of professional engineers, valuing each others' time and effort
  • from corruption and bureaucracy - we operate in an honest and competitive environment and we are one of the AWS's top 10 key partners.

# 3 Professional Development

  • Work with innovative Silicon Valley companies and traditional American companies at the cutting edge of digital transformation
  • We work with the newest technologies in AWS cloud and open-source tools, Jira, Confluence, Lucidchart, Slack
  • Constant practice at writing and speaking in English
  • The team which is willing to share its experience
  • Paid AWS certification: we provide training material, paid time off and examination itself
  • Horizontal and vertical career growth - are you ready to be greater, faster, stronger? We keep growing and people keep growing with us
  • A nice bonus if you accept our offer quickly

Контакты

Иногда письма на почте теряются, а сотрудники могут пропустить ваш звонок из-за совещания. Вы можете связаться с работодателем напрямую по указанным контактам, но мы рекомендуем еще отправить отклик на сайте

Akimova Tamara, Senior Recruiter
+7 (905) 7363687
[email protected]
Откликнуться
Время карьеры
Найти работу мечты
Время карьеры
Зарегистрируйся сейчас и получай информацию об актуальных вакансиях, стажировках, событиях
Регистрация