Senior Data Engineer
A critical member of the Big Data Analytical platform development team - he/she will be responsible for the design, development, and implementation of the data processing for ingestion, exporting and transformation areas of the data pipeline. The individual will work on one or many product teams at a time. He/she must have very strong communication and analytical skills and an ability to work as part of an agile team (product owner, developers, etc.) at the interface with traditional IT, where he/she interacts with the data architecture group to gather system knowledge necessary to execute his/her tasks.
● Building big data solution for batch processing and streaming data based on jointly defined requirements along the data pipeline under the concern of quality and having alert in place
● Works closely with both the platform engineer as well as the data architecture and data scientist group ● Work with Product Owners to help them leverage data solutions in their products
● Providing guidance on suitable options, designing, and creating data pipeline for the analytical solutions data lake / data warehouses to specific micro services
● Supporting the bug fixing and performance analysis along the data pipeline
● Bring experience of Data WareHouses, and Data Lakes to development team
● Be a strong advocate for a culture of process and data quality across development teams
● Follow an agile development methodology
● Keep abreast of new technologies in Big data area
● BS/MS degree in Computer Science, Engineering, Mathematics, Statistics or related subject
● Has Strong programming skills such as Scala, Python, SQL, Java
● 2 years of experience working on Big Data platforms like Hadoop, Hive etc.
● 2 years of experience working with parallel data processing such as Spark, Spark SQL, Spark Streaming and similar languages
● 2 years of coding experience in data transformation and data management for structured database and unstructured database.
● Experience with Cloud platforms such as Google Cloud platform, etc.
● Experience building pipeline using scheduler tools such as Airflow, Oozie, etc.
● Experience in Machine learning is plus
● Experienced with data modeling, design patterns, building highly scalable and secured analytical solutions
● Strong knowledge of Linux and shell scripting
● Telecom industry and data model knowledge is preferred
● Strong analytical and problem-solving skills
● Experience working in start-up environment or organizations with an agile culture
● Professional attitude and service orientation; superb team player
● Self-learning skill.
● Good communication in Thai and English