Data Architect With AI

Job Description

We are seeking a 12+ experienced Data Architect with strong expertise in Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Data Lake, SQL Data Warehouse, and modern cloud data platforms to define and implement scalable, secure, high-performance data architectures.

The ideal candidate will be responsible for translating business and technical requirements into robust enterprise data architecture and solution designs, defining data platforms, integration patterns, data models, governance frameworks, and analytics solutions. The role will involve working closely with business stakeholders, data engineers, BI teams, application teams, and cloud/infrastructure teams to deliver scalable and cost-effective data solutions.

The Data Architect will provide technical leadership across batch and real-time data processing, Lakehouse/Data Warehouse architecture, data integration, data modeling, data governance, security, Power BI, and cloud data platforms .

Key Responsibilities:

Data Architecture & Solution Design

• Design and define end-to-end enterprise data architectures using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Data Lake, Synapse, and related cloud services.

• Translate business requirements into scalable, secure, and maintainable data architecture and solution designs.

• Define architecture patterns for data ingestion, transformation, storage, processing, serving, analytics, and reporting.

• Design Lakehouse, Data Warehouse, and Medallion Architecture solutions based on business and technical requirements.

• Design and define AI/ML-ready data architectures to support Machine Learning, Generative AI, and advanced analytics use cases.

• Define data architecture patterns for Generative AI, LLM, RAG, embeddings, and vector search solutions

• Evaluate and recommend appropriate technologies, platforms, and architectural patterns based on performance, scalability, security, availability, and cost.

• Create architecture diagrams, data flow diagrams, solution designs, and technical documentation.

• Design secure and scalable integration of enterprise data platforms with AI/ML and Generative AI applications.

• Integrate AI agents with APIs, databases, cloud services (AWS, Azure), and enterprise applications.

• Good exposure to work on RFP’s, sales proposal, technical and estimate.

Microsoft Fabric & Databricks Architecture

• Define enterprise architecture using Microsoft Fabric, including OneLake, Lakehouse, Warehouse, Data Factory, Dataflows, Power BI, and related Fabric capabilities.

• Design scalable Azure Databricks and Delta Lake architectures for large-scale data processing and analytics.

• Define Bronze, Silver, and Gold data-layer strategies and standards.

• Design Databricks solutions using Apache Spark, Structured Streaming, Delta Lake, Unity Catalog, and optimized data processing patterns.

• Define standards for partitioning, file sizes, schema evolution, incremental processing, data retention, and performance optimization.

• Guide teams in optimizing Databricks workloads for performance and cloud cost efficiency.

Data Integration & ETL/ELT Architecture

• Design data integration architectures for data originating from databases, APIs, files, SaaS applications, Dynamics 365, and streaming sources.

• Define appropriate batch, micro-batch, and real-time/streaming ingestion patterns.

• Establish ETL/ELT patterns using ADF, Fabric Data Factory, Databricks, and other cloud-native services.

• Define frameworks for incremental loads, CDC, SCD Type 1/Type 2, reconciliation, error handling, and data recovery.

• Design reusable and scalable data ingestion frameworks.

Data Modelling & Data Architecture

• Define conceptual, logical, and physical data models based on business requirements.

• Design dimensional models, star schemas, fact and dimension tables, and analytical data models.

• Define enterprise data standards, naming conventions, data domains, and master/reference data structures.

• Design models optimized for Power BI, analytics, reporting, and downstream applications.

• Define strategies for managing historical data, slowly changing dimensions, data retention, and archival.

Data Governance, Security & Compliance

• Define enterprise data governance and security architecture across the data platform.

• Implement and govern Microsoft Purview, Unity Catalog, RBAC, data classification, lineage, auditing, and access-control strategies.

• Define security patterns for row-level security, column-level security, masking, encryption, secrets management, and secure data access.

• Establish data ownership, data stewardship, metadata, lineage, and data quality standards.

Performance, Scalability & Cost Optimization

• Define architecture strategies to support high-volume and high-performance data workloads.

• Identify and resolve architectural bottlenecks across ingestion, storage, transformation, database, and reporting layers.

• Optimize Databricks/Spark, Delta Lake, Fabric, SQL, and Power BI workloads.

• Define appropriate compute sizing, workload isolation, partitioning, caching, indexing, and optimization strategies.

• Perform capacity planning, scalability assessment, and cost optimization for cloud data platforms.

Cloud & Technology Architecture

• Design cloud-based data solutions primarily on Microsoft Azure, with knowledge of AWS/GCP data platforms.

• Define integration patterns across Azure, AWS, APIs, SaaS applications, and on-premises systems.

• Evaluate new technologies and recommend solutions based on business requirements and architectural standards.

• Define cloud architecture covering availability, scalability, networking, security, disaster recovery, and operational support.

• Strong understanding of data governance, security, metadata, lineage, and data quality.

• Experience with cloud architecture, scalability, performance optimization, and cost management.

• Good understanding of CI/CD, DevOps, Infrastructure as Code, monitoring, and deployment strategies.

Key Competencies

• Enterprise Data Architecture

• Solution Architecture

• Cloud Data Architecture

• Databricks & Lakehouse Architecture

• Data Modeling

• Data Governance & Security

• Performance & Cost Optimization

• Technical Leadership

• Stakeholder Management

• Architecture Documentation & Design

Qualifications

Educational qualification:

B.E/B.Tech/MCA/PhD or equivalent Qualification

Experience :

• 12+ experienced Data Architect with strong expertise in Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Data Lake, SQL Data Warehouse, and modern cloud data platforms to define and implement scalable, secure, high-performance data architectures.

Mandatory/requires Skills :

• Design and define end-to-end enterprise data architectures using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Data Lake, Synapse, and related cloud services.

• Translate business requirements into scalable, secure, and maintainable data architecture and solution designs.

• Define architecture patterns for data ingestion, transformation, storage, processing, serving, analytics, and reporting.

• Design Lakehouse, Data Warehouse, and Medallion Architecture solutions based on business and technical requirements.

• Design and define AI/ML-ready data architectures to support Machine Learning, Generative AI, and advanced analytics use cases.

• Define data architecture patterns for Generative AI, LLM, RAG, embeddings, and vector search solutions

• Evaluate and recommend appropriate technologies, platforms, and architectural patterns based on performance, scalability, security, availability, and cost.

• Create architecture diagrams, data flow diagrams, solution designs, and technical documentation.

• Design secure and scalable integration of enterprise data platforms with AI/ML and Generative AI applications.

…