Data Engineer - Messaging Data Platform

Responsibilities

In this role, you’ll:

• Develop, construct, test, and maintain data architectures (e.g., databases, large-scale processing systems).
• Design and implement efficient data pipelines for the acquisition, storage, and analysis of large datasets.
• Collaborate with cross-functional teams to understand data requirements and implement solutions.
• Ensure the availability and integrity of data pipelines and systems.
• Optimize and fine-tune performance of data solutions.
• Troubleshoot and resolve data-related issues in a timely manner.
• Develop and maintain documentation for data processes and pipelines.
• Stay abreast of emerging technologies and industry trends related to data engineering.
• Create tools that make it easy for non-technical users to easily, and reliably, transform and query data.

Required:

• 3+ years of experience with Big Data processing tools and frameworks such as Apache Spark, PySpark, SparkSQL.
• Deep technical understanding of ETL tools, low-latency data stores, multiple data warehouses and data catalogs.
• Familiarity with data testing and verification tooling and best practices.
• Experience with cloud services (AWS preferred, Google, Azure etc.)
• Proficient in working with Key-Value, Streaming, and Search Database technologies, including AWS DynamoDB, Apache Kafka, and ElasticSearch.
• Readiness to participate in the on-call rotation.