JOB DESCRIPTION

Data Engineering Lead/ Project Manager for a global technology company

Data Engineer/Data Scientist

About
About the company
This company is a fast-growing, global technology company headquartered in Santa Clara, US, and
with offices in Chennai and Bangalore. Their mission is to simplify and innovate digital
marketing and data governance by building unique and large scale data products.
They are a world-class engineering team that encompasses front end (UI), back end (API/Java), mobile
(Android/IOS), and Big Data engineering to deliver compelling products.
Global Aspects
  • Foreign employees already employed
  • CEO/Executive is a foreigner
  • Many employees speak English
  • Communicate with foreign clients
  • Official Language is English
  • International sales
  • Actively pursuing diversity at the work place
  • Market research overseas
PR Aspects
They have been expanding their business overseas.
Other details
  • Flex time system
  • You can work from home
  • No dress code
JobDescription
You will be a key member of the engineering and project management team, responsible
for shaping and delivering data projects. You’ll have the opportunity to shape the next
generation of data analytics tech stack leveraging big data technologies. You will be
working closely with business stakeholders, product managers, and engineering teams to
meet the data requirements of various initiatives.

Responsibilities
● Design and implement distributed data processing pipelines using Spark, Hive,
Python, and other tools and languages prevalent in the Hadoop ecosystem on AWS.
Ability to design and implement end-to-end solutions.
● Work with architecture/engineering leads and other teams to ensure quality
solutions are implemented, and engineering best practices are defined and adhered
to. Create and maintain data documentation and definitions.
● Work with the product owner and scrum master to understand requirements, help
the team plan, and execute sprint tickets, work with other technical teams to
develop new features.
● Communicate project progress to stakeholders and senior management adequately
through presentations and periodic status reports.
● Identify and manage project risks throughout the project lifecycle and devise
mitigation strategies.
● Responsible for the documentation and carrying out of training for customer
service/technical support for new products.
● Build utilities, user-defined functions, and frameworks to better enable data flow
patterns.
● Build strong relationships with tech and sales teams distributed geographically.
● Help drive best practices in continuous integration and deliver data quality.
● Manage data flows and set up automation between various data sources.
● Help drive optimization, testing, and tooling of the data products.

※Please note that the position is only for those who are currently residing in Japan only
Requirements
Must
● B.E. in Computer Science/Engineering or equivalent. (MBA a plus)
● Strong demonstrable skills in two of the following programming languages – Python,
Scala or Java.
● Ability to understand technical issues, project requirements and communicate
effectively with technical teams.
● Ability to evaluate technology solutions, including build / buy / partner options.
● Minimum 7 years experience working on full life cycle Big Data production projects.
● Should have experience with AWS services like EMR, Lambda, S3, DynamoDb. Strong
Experience in processing Big Data and analyzing the data using Spark, Map Reduce,
Hadoop, Sqoop, Apache Airflow, HDFS, Hive, Zookeeper, etc.
● Familiarity with Docker, Airflow, or equivalent data pipeline and workflow
management tools, distributed Stream Processing frameworks for Fast & Big Data
like ApacheSpark, Flink, Kafka streams.
● Strong skills around developing RESTful APIs.
● Intermediate to advanced knowledge of SQL. Relational SQL and NoSQL databases,
including Postgres, MySql, Redshift, and Redis. Experience in SQL tuning, schema
design, or analytical programming.
● Experience with different storage formats like Parquet, Avro, Arrow, and JSON.
● Experience in unit, functional and automated testing.
● Strong expertise in troubleshooting production issues.
● Comfortable working across a wide array of technologies, fast-paced, and
results-oriented environment.
● Excellent verbal and writing skills to effectively collaborate with both business and
technical teams.
Japanese Level
  • None
Better
●Experience in digital marketing API development
●Japanese language skills to communicate with Japanese corporate clients
Conditions
Annual Expected salary (Minimum)
7500000
Annual Expected salary (Maximum)
12000000
Transporation cost coverage
  • Covered based on Company's set policy
Flextime
  • Yes
Holidays
  • End of Year and New Year Holiday
  • Saturday
  • National Holiday
  • Sunday
Insurances
  • Worker’s Accident Compensation Insurance (労災保険)
  • Health Insurance (健康保険)
  • Unemployment Insurance (雇用保険)
  • Welfare Pension Insurance (厚生年金保険)
Remote Work
  • Temporary remote work (During Covid-19)