Avenga is looking for a progressive Data Engineer, who knows that understanding the data is the key to excellent data science and is ready to push new knowledge. You will be the technical lead for a small team of eager and capable engineers. You will work directly with the product management team to understand the product roadmap deliverables, serve as a technical leader and mentor for your team mates, and contribute directly with architecture, code and documentation.
Vacancy level: Lead
Cooperation type: Full-time
Technology stack: Kafka, MWAA, Python, AWS
Share the opportunity
What you bring along:
- BS/MS in Computer Science or closely related discipline (math, computer engineering).
- A passion for writing good, clean, and reliable code.
- Knowledge of a range of database systems.
- Collaborative and enthusiastic approach to software development.
- Strong sense of project ownership and personal responsibility.
- Experience in designing, implementing, and managing data lakes
- Up to 7 years of experience working hands-on with large data sets
Knowledge of the following specific technologies:
- Apache Airflow. MWAA experience would be great to have, but not required.
- Apache Spark
- Athena and AWS Glue Databricks, or other lake-based data storage and query solutions
- Distributed messaging systems such as Apache Kafka
- Redshift experience is good to have, but not required
- Familiarity with row-based data formats such as Avro, JSON or CSV in addition to columnar ones such as Parquet
- Python and data analysis libraries such as Pandas, numpy, matplotlib, etc.
- AWS devops tools and techniques
This is your mission:
- Measurement and data analysis lie at the foundation of TVision’s data products, but those measurements need to be assembled with context to turn them into actionable insights for our customers.
- Reporting to the CTO, you will work closely with the data science and data analytics teams to design and build out a robust data storage, analysis, and delivery platform. Additionally, you will be responsible for implementing mission critical data integrations with partners and customers through APIs and file transfers.
- With data flowing in from many thousands of TVision measurement devices and partner integrations, and flowing out to customers and TVision’s SaaS products, you will build and orchestrate the processes that make it all work..
- We primarily develop in Python, use Postgres and Redshift as our main database technologies, and rely on Spark clusters for applying complex algorithms to large datasets in a timely fashion.
- All of our back end infrastructure resides in AWS. We use Apache Airflow (MWAA) for data pipeline orchestration, invoking partner APIs using the AWS Lambda service. Additionally, we use data lakes built with Athena and Glue for data analysis.
- You will be the technical lead for a small team of eager and capable engineers. You will work directly with the product management team to understand the product roadmap deliverables, serve as a technical leader and mentor for your team mates, and contribute directly with architecture, code and documentation.
- Our employees get regular performance and salary reviews;
- We offer an attractive social and benefits package;
- Everyone at Avenga is subject to professional growth via our mentorship program;
- The company provides extended training and certification opportunities;
- We foster professional freedom by offering free-of-charge English courses from A1 to C2;
- We stay up to date with the industry by embarking on tech talks, webinars, conferences, and hackathons;
- Avenga cares for its employees well-being by providing well-rounded health insurance coverage and fitness compensation programs;
- Our employees’ families are no less important, as we offer maternity aid and financial support for a number of family reasons, including marriage and childbirth;
- The company fosters a sense of professional belonging and an environment of togetherness: we achieve things together and celebrate our milestones.
Talent Acquisition & Resource Manager
More than just a place to work
Avenga is a perfect environment for anyone eager to build a successful meaningful career, as we work with both global and local projects.
Continuous care for people
Our employees are the company’s biggest asset. The reason we excel at projects is because we take proper care of our professionals who stand behind the solutions we deliver.
We are an international team of like-minded digital transformation proponents from Ukraine, Germany, Poland, the US and Malaysia.
We care for the planet and we care for people. This is why we aspire for sustainable excellence when it comes to setting up processes at Avenga and how we choose the projects we work on.
Mature experts you will cherish working with
Avenga is a self-development organism where seniors grow by mentoring juniors and vice versa.
Constant professional development
We believe there is always room for improvement. We will foster your growth by guiding and supporting you along your chosen path.