The Role:

We are looking for hands-on Principal Data-Architect for a challenging and fun filled work of building a data architecture that is future proof for one of the customers in the financial domain.

Responsibilities:

• Own design and maintenance of all aspects of data solutions including modeling, developing, technical documentation, data diagrams and data dictionaries.

• Provide expertise in the development of standards, architectural governance, design patterns, and practices, evaluate best applicable solutions for different use cases

• Determines and develops architectural approaches and solutions, conducts business reviews, documents current systems, and develops recommendations

• Lead the data strategy and own the vision and roadmap of data products

• Work with stakeholders to ensure that data related business requirements for protecting sensitive data are clearly defined, communicated, and well understood and considered as part of operational prioritization and planning

• Develop, maintain, and optimize data infrastructure using Delta Lake, MLflow, and Databricks SQL to enhance data management, processing, and analytics.

• Utilize Snowflake’s features such as data sharing, zero-copy cloning, and automatic scaling to optimize data storage, accessibility, and performance. Ensure effective management of both semi-structured and structured data within Snowflake’s architecture.

• Implement and manage data storage solutions using Amazon S3, perform data warehousing with Amazon Redshift.

• Design and implement data integration workflows using AWS Glue to orchestrate and automate data movement and transformation.

• Design and implement scalable data pipelines using tools like Apache Kafka or Apache Airflow to facilitate real-time data processing and batch data workflows.

• Apply advanced analytics techniques, including predictive modeling and data mining, to uncover insights and drive data-driven decision-making.

Requirements:

• Overall experience of 15+ years

• Data architecture, data platform, data warehouse related experience of 10+ years

• Hands on experience with snowflake - 4+ years experience, Data bricks 4+ years

• Between snowflake and Data bricks (5+ years experience)

• Proficiency in features such as Delta Lake, MLflow, and Databricks SQL. Experience in managing Spark clusters and implementing machine learning workflows.

• Solid experience in emerging and traditional data stack components such as: batch and real time data ingestion, ETL, ELT, orchestration tools, on-prem and cloud DW, Python, structured, semi and unstructured databases

• Knowledge of features like Snowflake’s data sharing, zero-copy cloning, and automatic scaling. Experience in working with Snowflake’s architecture for semi-structured and structured data.

• Experience with services like Amazon S3, Amazon Redshift, and AWS Glue.

• Proficiency in tools such as Apache NiFi, Talend, Informatica, or Microsoft SQL Server Integration Services (SSIS).

• Experience in designing and implementing data pipelines using tools like Apache Kafka or Apache Airflow.

• Ability to perform data profiling, data quality assessments, and performance tuning.

• Experience in comparing and evaluating different data technologies based on criteria like performance, scalability, and cost.

• Skills in applying advanced analytics techniques, including predictive modeling and data mining.

• Expert with industry standard data practices, data strategies and data concepts

• Demonstrated experience in architecting/re-architecting complex data systems and data models.

• Demonstrated experience in overall system design, including database selection and solutioning.

Nice-to-Have Skills:

• Experience with data governance tools such as Collibra or Alation.

• Knowledge of data quality frameworks and standards, such as Data Quality Dimensions (completeness, consistency, etc.).

• Familiarity with tools like Apache Beam or Luigi for managing complex data workflows.

• Awareness of emerging data technologies such as data mesh, data fabric, and real-time data processing frameworks.