Strong experience in ADF( Azure data factory), Azure SQL, Synapse, Spark/Databricks= 5+ Yrs
Excellent written and verbal communication, intellectual curiosity, a passion to understand and solve problems, consulting & customer service
Structured and conceptual mindset coupled with strong quantitative and analytical problem-solving aptitude
Exceptional interpersonal and collaboration skills within a team environment
PySpark
Parquet files
Big data and modern data lake
Demonstrable experience in enterprise level data platforms involving implementation of end-to-end data pipelines
Hands-on experience with Azure
Experience with column-oriented database technologies (e.g., Synapse), NoSQL database technologies (e.g., DynamoDB, Cosmos DB, etc.) and traditional database systems (e.g., SQL Server, Oracle, MySQL)
Experience in data pipelines and solutions for both streaming and batch integrations using tools/frameworks like Azure Data Factory, Azure functions and Stream analytics
Metadata definition and management via data catalogs, service catalogs, and stewardship tools such as OpenMetadata, and Azure Purview
Test, plan, creation and test programming using automated testing frameworks, data validation and quality frameworks, and data lineage frameworks
Data modeling, querying, and optimization for relational, NoSQL, timeseries, graph databases, data warehouses and data lakes
Data processing programming using SQL, Python, and similar tools
Logical programming in Python, Spark, PySpark, Java, Javascript, and/or Scala
Cloud-native data platform design with a focus on streaming and event-driven architectures
Participate in integrated validation and analysis sessions of components and subsystems on production servers
Data ingest, validation, and enrichment pipeline design and implementation
SDLC optimization across workstreams within a solution
Bachelor’s degree in Computer Science, Engineering, or related field