Data Architect
About Purgo AI
Purgo AI delivers LLM based agentic design, development and deployment of data applications over cloud data warehouses. Purgo AI dramatically improves productivity, reduce costs and shortens time to delivery for data engineering teams.
Purgo AI is based in Palo Alto, CA and funded by The Hive and Capital One Growth Ventures. The company’s co-founder and CTO, Sang Kim, has been an engineering leader across VMWare and Blackberry. Early users of the product include leading enterprises in the life sciences, media and financial services verticals. Purgo AI’s in-built Vertical Solutions have saved users months of time, cost and efforts by getting to production within days. Prominent execs from several cloud data warehouse leaders are advisors to Purgo AI. The product integrates with leading cloud data warehouses including Databricks, Snowflake, and others.
Purgo AI helps automate the building ETL/ELT pipelines and BI applications on cloud data warehouses. The product offers data engineering teams an end-to-end requirements-to-production design, development, test & deployment of ETL/ELT pipelines.
- Purgo AI Vertical Solutions come pre-built with comprehensive Process Definition Libraries written by subject matter experts in natural language as easy-to-edit Jira tickets.
- Business analysts or product owners specify ETL/ELT user requirements as Jira tickets through Purgo AI’s Jira app either by easily editing in-built process definitions or creating new ones.
- This triggers the generation of a design (Behavior-driven design (BDD)) specification. The design creates test harnesses with test scripts and test data for QA.
- Purgo AI then generates source code from integrated code-generation LLMs by using the design specifications without needing any human prompting.
- For enterprise context the product integrates seamlessly with Github and GitLab (to interpret existing/legacy source code and pipelines), and data warehouse catalogs (for schema).
- The generated code is subject to pre-generated quality assurance tests, and test fails re-trigger generation of the source code. The final source code is ready for end deployment over the cloud data warehouse after inspection/approval by the business analyst team.
The entire process has end-to-end traceability through Purgo AI generated log entries across Jira and GitHub systems. Across the end-to-end development process, Purgo AI produces intermediate development artifacts enabling developers transparent, white-box review.
In addition to ETL/ELT pipelines for Business Intelligence applications, in the future Purgo AI will provide a range of requirements-to-production workflows, viz. cloud migration and on-demand machine learning.
About the role
Purgo AI is seeking a Data Architect with a unique opportunity to champion the use of generative AI in designing, developing, testing, maintaining, and migrating cloud data applications. This is a hands-on role requiring close collaboration with Purgo AI’s product and engineering teams—as well as customers and partners—to drive adoption of Purgo AI’s innovative software design lifecycle. The successful candidate will be instrumental in advancing generative AI integration with cloud data warehouses, ensuring successful implementation across both partners and customers.
Responsibilities
- Identify and analyze customer data engineering requirements for Purgo’s AI Agent use cases.
- Design and architect solutions that leverage Purgo AI’s generative AI-powered software lifecycle on leading cloud data warehouses (e.g., Snowflake, Databricks).
- Define and document best practices for data engineering for Purgo AI Agent
- Conduct technical assessments, build proofs-of-value, and showcase automated solutions that drive optimized customer adoption paths.
- Stay current on trends in cloud data platforms and data engineering, GenAI and related technologies to inform solution design.
- Architect and implement security, compliance, and data governance standards across all Purgo AI solutions.
About you
- Over 15 years of experience working on SaaS applications focused on cloud data warehousing and data engineering
- Developed cloud data engineering tools at companies such as Fivetran, dbt, Nexla, Collibra, Alation, ThoughtSpot, Qlik, or Looker
- Extensive background in data warehousing, ETL processes, AI modeling, and data integration for platforms like Databricks and Snowflake
- Proficiency in programming in PySpark, Python, SQL, and Scala.
- Hands-on experience with cloud platforms like AWS, Azure, or Google Cloud.
- Exceptional problem-solving skills with the ability to thrive in a fast-paced, collaborative environment.
- Strong communication and presentation skills to effectively convey technical concepts to diverse stakeholders.
Interested candidates should reach out to sang@purgo.ai.
Purgo AI is an affirmative action employer and welcomes candidates who will contribute to the diversity of the company.